Airbnb Lodging URLs Scraper An Apify scraper to extract all Airbnb lodging URLs from a city or region. ## 🎯 Features - Complete extraction : Retrieves all lodging URLs from an Airbnb search - Detailed property data : Extracts title, price, host type, rating, reviews, travel dates, and images - Smart image selection : Gets property images (not host profile photos) - Price intelligence : Handles discounts and price variations - Date extraction : Accurate travel date detection - Automatic navigation : Browses through all result pages - Smart scrolling : Loads all available listings - Clean URLs : Removes unnecessary search parameters - Robustness : Error handling and multiple fallbacks ## 📊 Extracted Data The scraper extracts detailed property information: json { "url": "https://www.airbnb.fr/rooms/1169471441730393904", "title": "Appartement \"Flore\"", "price": "626 €", "hostType": "Hôte particulier", "rating": "4,97", "reviewCount": "30", "travelDates": "7–12 nov.", "imageUrl": "https://a0.muscache.com/im/pictures/hosting/Hosting-1169471441730393904/original/e25c6d0f-cfbf-44c8-b68d-e0febce169ae.jpeg?im_w=720", "extractedAt": "2025-10-25T12:43:09.521Z" } ### Detailed Fields - url : Property URL - title : Property title or name - price : Current price per night (with discount if applicable) - hostType : Type of host ("Hôte particulier" or "Hôte pro") - rating : Property rating (e.g., "4,97") - reviewCount : Number of reviews (e.g., "30") - travelDates : Travel dates for the booking (e.g., "7–12 nov.") - imageUrl : Property main image URL - extractedAt : Extraction timestamp ## 🚀 Usage ### Input Format The scraper accepts two input formats: #### 1. City name only json { "input": "Verneuil-sur-Seine" } #### 2. Full Airbnb search URL (with /homes at the end) json { "input": "https://www.airbnb.fr/s/verneuil-sur-seine/homes" } The scraper automatically detects the format and constructs the appropriate search URL. ### Examples #### City name (Recommended) json { "input": "Lyon" } #### Full URL with /homes json { "input": "https://www.airbnb.fr/s/lyon/homes" } #### City with spaces and special characters json { "input": "Verneuil-sur-Seine" } #### Full URL for complex cities json { "input": "https://www.airbnb.fr/s/verneuil-sur-seine/homes" } ## ⚡ Performance - Automatic navigation : Browses through all available pages - Smart detection : Automatically stops when no more pages - Clean URLs : Removes search parameters - Deduplication : Avoids duplicate URLs ## 🛠️ Technical Architecture ### Technologies Used - Apify SDK : Automation framework - PuppeteerCrawler : Navigation and extraction - Puppeteer : Browser control - Node.js : JavaScript runtime ### Extraction Process 1. Navigation : Access to Airbnb search page 2. Cookie management : Automatic cookie acceptance 3. Property extraction : Retrieval of lodging data (URL, title, price, host type, rating, reviews, dates, images) 4. Smart image selection : Property images (not host photos) 5. Price processing : Handles discounts and price variations 6. Date extraction : Accurate travel date detection 7. Automatic navigation : Moving to next pages 8. Cleaning : Removal of unnecessary parameters 9. Deduplication : Elimination of duplicates ## 📈 Typical Results - 250+ properties extracted per city - Complete data : URL, title, price, host type, rating, reviews, dates, and images - 15-20 pages browsed on average - Execution time : ~1-2 minutes per city (optimized) - Accuracy : Extraction of all available properties with detailed information ## ⚙️ Recommended Configuration ### Optimal Settings for Apify: MEMORY: 8 GB (recommended) TIMEOUT: 7200s (2 hours) RESTART ON ERROR: ON ### Why 8GB Memory? - Puppeteer + Chrome : ~4-5GB base consumption - Airbnb pages : ~1-2GB for complex layouts - Navigation buffer : ~1GB for smooth transitions - Total recommended : 8GB for optimal performance ### Performance Comparison: | Memory | Speed | Stability | Recommendation | |--------|-------|-----------|----------------| | 4GB | ❌ Slow | ❌ Crashes | ❌ Not recommended | | 6GB | ⚠️ Medium | ⚠️ Unstable | ⚠️ Minimum | | 8GB | ✅ Fast | ✅ Stable | ✅ Recommended | | 16GB | ✅ Very fast | ✅ Very stable | ✅ Optimal | ### Alternative (Budget Option): MEMORY: 6 GB (minimum) TIMEOUT: 5400s (1.5 hours) RESTART ON ERROR: ON ### ⚠️ Important: Manual Configuration If the default settings don't apply automatically, you can manually configure them in the Apify console: 1. Go to your Actor in the Apify console 2. Click on "Settings" tab 3. Set the following values: - Memory: 8192 MB (8 GB) - Timeout: 7200 seconds (2 hours) - Restart on error: ON This ensures optimal performance and prevents memory-related crashes. ## 🔍 Selectors Used The scraper uses multiple selectors to maximize extraction: - [data-testid="card-container"] a[href*="/rooms/"] - a[href*="/rooms/"] - [data-testid="listing-card-link"] ## 🚀 Deployment The scraper is ready for deployment on Apify Cloud: - Complete configuration : package.json, Dockerfile, actor.json - Optimized code : Based on Airbnb HTML analysis - Documentation : Complete README with examples ## 📝 Important Notes - Search URLs : Use Airbnb search URLs (e.g., /s/Paris--France) - Pagination : The scraper automatically detects next pages - Deduplication : Duplicate URLs are automatically removed - Cleaning : Search parameters are removed from URLs ## 📞 Support For any questions or issues: - Check execution logs - Verify Airbnb HTML structure - Adapt selectors if necessary - Verify that the search URL is valid