Airbnb Data Scraper for Apify This project is an Apify actor designed to scrape detailed data from Airbnb listings, including SEO metadata, pricing details, and other key information. The actor supports dynamic date generation for flexible booking queries and stores the extracted data in an Apify dataset. --- ## Features - Dynamic Date Generation: Automatically generates check-in and check-out dates based on user input. - Comprehensive Data Extraction: - SEO Metadata: Title, meta description, canonical URL, and more. - Pricing Details: Breakdown of prices, including cleaning fee, service fee, and taxes. - Property Information: Room type, location, capacity, rating, and reviews. - Error Handling: Gracefully handles API errors and continues processing other listings. - Data Storage: - Outputs data to the Apify dataset. - Optionally saves data to a local CSV file for offline use. --- ## Input Parameters The actor accepts the following input parameters: | Parameter | Description | Example Value | |----------------------|-------------------------------------------------|-------------------------------------------| | startUrls | List of Airbnb listing URLs to scrape. | [{"url": "https://www.airbnb.com/rooms/12345"}] | | checkInDate | Starting date for check-in. | "2024-11-21" | | Stay_Days | Duration of each stay in days. | 1 | | numberOfDays | Total number of days to iterate. | 10 | | adults | Number of adults for the booking. | 2 | | children | Number of children for the booking. | 0 | | pets | Indicates if pets are included. | 0 | --- ## Output Data The scraper outputs a dataset with detailed information for each listing and date range: | Field Name | Description | |-------------------------|------------------------------------------------------------| | Check-In Date | Generated check-in date. | | Check-Out Date | Generated check-out date. | | Booking Available | Indicates if instant booking is available. | | Title | SEO title of the listing. | | Meta Description | SEO meta description of the listing. | | Canonical URL | Canonical URL of the listing page. | | Room Info | General description of the room. | | Property Type | Type of property (e.g., apartment, house). | | Room Rating | Star rating of the listing. | | Total Reviews | Total number of reviews. | | Location | Location of the listing. | | Capacity | Maximum number of guests the property can accommodate. | | Image URL | URL of the primary listing image. | | Price Breakdown Title | Title of the price breakdown section. | | Strike Through Price | Original price before discounts. | | Primary Price | Current price after discounts. | | Cleaning Fee | Cleaning fee for the booking. | | Service Fee | Service fee for the booking. | | Taxes | Tax amount for the booking. | | Total Price | Total price for the stay, including fees and taxes. | --- ## How It Works 1. Dynamic Date Generator: - Generates a sequence of check-in and check-out dates based on the input checkInDate, Stay_Days, and numberOfDays. 2. Data Extraction: - Makes a GraphQL API call to Airbnb using dynamically generated queries. - Extracts specific SEO and pricing details using a recursive function to traverse JSON paths. 3. Data Storage: - Stores data in the Apify dataset for further processing or integration. - Optionally saves the data to a local CSV file. --- ## Example Input json { "startUrls": [ { "url": "https://www.airbnb.com/rooms/12345" }, { "url": "https://www.airbnb.com/rooms/67890" } ], "checkInDate": "2024-11-21", "Stay_Days": 1, "numberOfDays": 5, "adults": "2", "children": "0", "pets": "0" } --- ## Example Output | Check-In Date | Check-Out Date | Title | Total Price | Room Rating | Total Reviews | |---------------|----------------|-------------------------|-------------|-------------|---------------| | 2024-11-21 | 2024-11-22 | Cozy Apartment in City | $120 | 4.8 | 120 | | 2024-11-22 | 2024-11-23 | Spacious Condo by Beach | $200 | 4.9 | 85 | --- ## Logging and Debugging The script uses Python's logging module to log: - Progress of the scraping process. - Errors encountered during API calls or data processing. Logs are displayed in the console and can be used to debug issues. --- ## License This project is licensed under the MIT License.