Trigger AI platform using Harvester output

[VimanyuAgg]

Once harvestor has ingested a listing - this should trigger AI platform. This might need tweaks in harvestor to onboard/ingest new data schema.

[supriyapuri]

Currently the AI team is operating on this base listing json: https://drive.google.com/file/d/1nZv2ZByyX1JEqhNRqOl86JctBeilTV8C/view?usp=drive_link

[supriyapuri]

I'm looking to identify the exact parts of this listing json that they are actually using.

[supriyapuri]

I was able to locate the logs for the job running under GCP. Below are the extracted features used by the LLM, along with a sample output.

Extracted fields from listing: dict_keys(['listing_id', 'checking_in_and_out_house_rule', 'during_your_stay_house_rule', 'before_you_leave_house_rule', 'house_rules_parsed_successfully', 'place_description', 'the_space_description', 'guest_access_description', 'other_things_to_note_description', 'registration_number_description', 'description_parsed_successfully', 'property_type', 'location', 'listing_under_title', 'review_count', 'star_rating', 'listing_lat', 'listing_lng', 'room_type', 'person_capacity', 'accuracy_rating', 'is_superhost', 'title', 'metadata_parsed_successfully', 'must_climb_stairs_property_info', 'property_info_parsed_successfully'])
2025-04-29 23:52:11,715 - INFO - Selected columns extracted: {'listing_id': '1000078195635658573', 'property_type': 'Entire home', 'room_type': 'Entire home/apt', 'title': 'Entire home in Vancouver, Canada', 'listing_under_title': 'Home in Vancouver · ★4.95 · 4 bedrooms · 4 beds · 4 baths', 'place_description': "Discover the perfect blend of city convenience and residential tranquility in our spacious 4-bedroom, 4-bathroom home in Vancouver. Whether you're traveling with family or friends, our well-appointed oasis promises a comfortable and memorable stay. Just 10 mins to Kits Beach, 15 to Downtown & 15 to UBC. Enjoy our backyard & deck for relaxation. No parties; ideal for a peaceful stay. Experience Vancouver from a prime location & cozy retreat. Book now for a memorable stay. (BN 25-157873)", 'during_your_stay_description': None, 'guest_access_description': 'You will have full access to both the main and top floors of our home, ensuring a spacious and private experience. The layout includes all the bedrooms, bathrooms, living areas, the dining room, and the fully equipped kitchen.<br /><br />Please note, the basement unit of our property is a separate entity. It has its own entrance and is completely independent of the main house. During your stay, the owners may reside in this basement unit but rest assured your privacy and comfort remain our top priority.<br /><br />The backyard is a shared space. The deck adjoining the main house is exclusively for guest use, providing a private outdoor space to unwind.', 'other_things_to_note_description': "Our property includes a separate guest unit in the basement with its own entrance, which may be occupied by the owners during your stay. Please be assured, while the owners are on-site, they do not have access to the main house or share any of the home's amenities, ensuring your complete privacy and exclusive use of the facilities.", 'registration_number_description': '25-157873', 'the_space_description': "Welcome to our inviting home, boasting four bedrooms and four bathrooms, designed to offer comfort and convenience for your stay. Our space has two master bedrooms, each featuring an ensuite bathroom, providing privacy and ease. All bedrooms and three of the bathrooms are conveniently located upstairs.<br /><br />On the ground floor, you'll find a well-appointed bathroom, adding to the accessibility of our home. Our property features two cozy living room areas, perfect for relaxation or socializing. One of these living spaces is enhanced with a massage chair, offering a touch of luxury and relaxation.<br /><br />The dining room comfortably seats six and is graced with a piano, setting the stage for memorable meals and perhaps some musical entertainment. Our fully equipped kitchen is ready for your culinary adventures, stocked with all the necessary appliances to make your stay feel like home.<br /><br />Step outside to enjoy our spacious backyard and deck, ideal for unwinding and enjoying the outdoors. <br /><br />For your convenience, there is free street parking available. Our home is designed to be your comfortable retreat, whether you're visiting for work or leisure. We look forward to welcoming you to our peaceful haven.", 'during_your_stay_house_rule': ['8 guests maximum', 'No pets', 'Quiet hours: 8:00\u202fPM - 7:00\u202fAM', 'No parties or events', 'Commercial photography allowed', 'No smoking', 'Additional rules', "Please note, our home is located in a quiet neighbourhood where the community values peace and tranquility. To respect this, we strictly prohibit parties and loud noise. It's important to be aware that our neighbours are quite attentive and won't hesitate to contact the police if disturbances occur. We appreciate your understanding and cooperation in maintaining a peaceful environment for everyone."], 'must_climb_stairs_property_info': 'Must climb stairs', 'pet(s)_live_on_property_property_info': None, 'potential_for_noise_property_info': None, 'some_spaces_are_shared_property_info': None, 'person_capacity': 8}

[VimanyuAgg]

May 20:
For the above listing id :

• Confirm if all the data is present in the listing json used by harvestor
• Update harvester to include all the new fields. Update the model and run.
• Update AI platform to take the data from harvester.
• Trigger AI Platform based on harvester listing.

[supriyapuri]

There is a mismatch in the dictionary of the extracted fields and the selected columns extracted json.

Out of the 23 required attributes, 13 are present in the provided data, while 10 are missing. The missing attributes that are not in the "Selected columns extracted" json are as follows:

• checking_in_and_out_house_rule
• before_you_leave_house_rule
• location
• review_count
• star_rating
• listing_lat
• listing_lng
• accuracy_rating
• is_superhost

[supriyapuri]

Many of the fields in the provided JSON are not standalone attributes but serve as nested or extended information under broader parent attributes. For example, the during_your_stay_house_rule field expands on general house rules by including specific behavioral expectations, while guest_access_description and the_space_description offer detailed narratives tied to the overarching property details. These types of entries are like sub-sections that support and expand the main topics. While they make the listing more informative, they don’t always match directly with the high-level attributes listed for comparison. So, it's important to recognize that much of the content is more descriptive and nested, rather than flat one-line values.

[VimanyuAgg]

Jun 10:

• Supriya to add the required attributes needed by the AI platform.

[supriyapuri]

The attributes being used in the JSON "the selected columns extracted json":

• listing_id
• property_type
• room_type
• title
• listing_under_title
• place_description
• the_space_description
• guest_access_description
• other_things_to_note_description
• registration_number_description
• during_your_stay_house_rule
• must_climb_stairs_property_info
• person_capacity

[supriyapuri]

Started working on creating the mapping for the attributes from Airbnb regulation to ai and need some infomration to move ahead:

Where are the attributes saved - AWS or any other db location?
What is the format - json, key value pair?
I am also assuming that we are doing a simple rename from the attributes scraped under regulation to ai..

[supriyapuri]

Hey Vimanyu and Mangat - thanks a lot for sharing the github link. I created this document highlighting what approach I am planning to take for mapping and where I am blocked at. There are a few attributes where we will need more info from the harvester and I have highlighted that. Please feel free to provide any feedback that you have.

Airbnb regulation and AI mapping

Let me know if you cannot access it.

[supriyapuri]

Airbnb regulation and AI map.docx

[supriyapuri]

Hey Vimanyu & Mangat! Based on the docker logs I was able to create a mapping code for the features in Ai part but still a few features are missing. Please check the not scraped ones.
` # --- Target fields ---

    "listing_id": row["airbnb_listing_id"],
    "property_type": None,  # Not scraped
    "room_type": row["room_type"],  # direct mapping
    "title": row["title"] if pd.notnull(row["title"]) else row["name"],
    "listing_under_title": row["title_text"],  # closest available field
    "place_description": None,  # not scraped
    "the_space_description": None,  # not scraped
    "guest_access_description": None,  # not scraped
    "other_things_to_note_description": None,  # not scraped
    "registration_number_description": None,  # not scraped
    "during_your_stay_house_rule": None,  # not scraped
    "must_climb_stairs_property_info": None,  # not scraped
    "person_capacity": int(row["person_capacity"]) if pd.notnull(row["person_capacity"]) else None,
    "registration_number": row["registration_number"],
    "is_bath_shared": row.get("bath_is_shared", None),  # from scraped dict
    "baths_text": row["baths_text"],
    "scrapped_at": row["scrapped_at"] if pd.notnull(row["scrapped_at"]) else datetime.today().date(),

`

Would appreciate it if you could look into this.. I will create the PR request with the updated code that I have for now.