Zillow Real Estate Agent Scraper Extract verified real estate agent leads from Zillow with complete contact information including emails, phone numbers, and performance metrics. ## 🚀 Features - High-Speed Scraping: Up to 150 concurrent requests based on your plan - Smart Deduplication: Automatic filtering of previously scraped agents - Persistent Storage: Your data is saved permanently across runs - Cost Tracking: Real-time credit monitoring and cost estimation - Quality Metrics: Data quality scoring for each scraped agent - Crash-Safe: Incremental saves prevent data loss - Production Ready: Health checks, error handling, and retry logic ## 📋 What Data You Get Each agent profile includes: - Contact Information: Name, Phone, Email, Address, City, Zip Code - Profile Link: Direct URL to Zillow profile - Performance Metrics: Total sales, Last 12 months sales, Total reviews - Quality Score: Data completeness indicator ## 💰 Pricing & Requirements ### You Need: - Apify Account (free tier available) - ScrapingBee API Key - Get yours at scrapingbee.com ### ScrapingBee Plans: | Plan | Concurrent | Credits/Month | Price | Cost per 1K Agents | |------|-----------|---------------|-------|---------------------| | Freelance | 10 | 150K | $49/mo | ~$3.30 | | Startup | 50 | 1M | $99/mo | ~$3.30 | | Business | 100 | 3M | $249/mo | ~$3.30 | | Business+ | 150 | 10M | $799/mo | ~$3.30 | Estimated based on ~33 credits per agent ### Cost Calculation: - Per Agent: ~30-40 ScrapingBee credits - 1,000 Agents: ~$3.30 - 10,000 Agents: ~$33.00 ## 🎯 Quick Start ### 1. Get Your ScrapingBee API Key 1. Sign up at scrapingbee.com 2. Choose your plan based on volume needs 3. Copy your API key from the dashboard ### 2. Configure the Actor Required Fields: - Zip Codes: List of US zip codes to scrape (e.g., 10001, 10002, 10003) - ScrapingBee Plan: Select your subscription plan - ScrapingBee API Key: Paste your API key Optional Fields: - Max Agents Per Zip: How many agents to scrape per zip code (default: 150) - Database Name: Custom name for your persistent dataset (default: zillow-agents-database) ### 3. Run the Actor Click Start and monitor progress in real-time. Results appear in the Output tab as they're scraped. ## 📊 Example Input json { "zip_codes": ["10001", "10002", "10003"], "max_agents_per_zip": 150, "scrapingbee_plan": "startup", "scrapingbee_api_key": "YOUR_API_KEY_HERE", "database_name": "zillow-agents-database" } ## 📤 Example Output json { "profile_url": "https://www.zillow.com/profile/john-smith", "name": "John Smith", "phone": "(555) 123-4567", "email": "john.smith@realty.com", "address": "123 Main St, New York, NY 10001", "city": "New York", "zipcode": "10001", "total_sales": "45", "last_12_months_sales": "12", "total_reviews": "28" } ## 🔄 How Deduplication Works The actor maintains a permanent database in your Apify account: 1. First Run: Scrapes fresh agents, saves to database 2. Subsequent Runs: Automatically skips previously scraped profiles 3. Cross-Run Memory: Your database persists indefinitely 4. Smart Filtering: Checks profile URLs before scraping Example: Run 1: Scrape zip 10001 → Get 150 agents → Save to database Run 2: Scrape zip 10001 → Skip 150 existing → Get 150 NEW agents Total in database: 300 unique agents ## ⚙️ Advanced Features ### Dynamic Page Calculation The actor automatically calculates optimal pages to search based on your target: - 50 agents needed → ~5 pages searched - 150 agents needed → ~16 pages searched - 500 agents needed → ~50 pages searched - Capped at 100 pages for safety ### Health Check Before scraping starts: - ✅ Verifies ScrapingBee API key is valid - ✅ Tests API connectivity - ✅ Checks available credits - ⚠️ Warns if credits are low (<10,000) ### Credit Monitoring During scraping: - 📊 Tracks credits used vs. remaining - 💰 Shows real-time cost calculation - ⚠️ Alerts on high consumption rate - 📈 Provides per-agent cost metrics ### Data Quality Scoring Each agent receives a quality score (0-100%): - Name: 25 points - Phone: 25 points - Email: 25 points - Address: 15 points - Sales Data: 10 points Scores displayed per zip code in final statistics. ## 📈 Performance ### Speed Benchmarks: | Plan | Agents/Hour | Daily Capacity | |------|-------------|----------------| | Freelance (10) | ~300 | ~7,200 | | Startup (50) | ~1,500 | ~36,000 | | Business (100) | ~3,000 | ~72,000 | | Business+ (150) | ~4,500 | ~108,000 | Approximate, depends on Zillow response times ### Reliability: - Auto-retry: Up to 4 attempts per failed request - Error handling: 500, 403, 429 errors handled gracefully - Crash recovery: Data saved incrementally every 50 agents ## 🛡️ Safety Features 1. Rate Limiting: Respects ScrapingBee plan limits 2. Exponential Backoff: Smart retry delays 3. Error Recovery: Continues scraping if individual requests fail 4. Batch Saves: Prevents data loss on crashes 5. Credit Safeguards: Monitors and alerts on low credits ## 💡 Use Cases - Real Estate Marketing: Build targeted agent contact lists - Lead Generation: Find agents in specific markets - Market Research: Analyze agent distribution and performance - Recruiting: Identify top-performing agents - CRM Integration: Export to your sales pipeline ## 🔧 Troubleshooting ### "Invalid API Key" Error - Double-check your ScrapingBee API key - Ensure no extra spaces when copying - Verify key is active in ScrapingBee dashboard ### "Low Credits" Warning - Check your ScrapingBee usage at app.scrapingbee.com - Upgrade plan or wait for monthly reset - Reduce max_agents_per_zip to conserve credits ### No New Agents Found - Area may be fully scraped (check your database) - Try different zip codes - Increase max_agents_per_zip to search deeper ### Scraping is Slow - Verify you selected the correct ScrapingBee plan - Check if Zillow is experiencing issues - Consider upgrading to a higher-tier plan ## 📞 Support - Issues: Report bugs via GitHub issues - Questions: Contact through Apify platform - Feature Requests: Open a discussion on GitHub ## 🔐 Privacy & Legal - This actor scrapes publicly available information from Zillow - Users are responsible for compliance with Zillow's Terms of Service - Respect privacy laws and regulations in your jurisdiction - Use scraped data ethically and responsibly ## 📝 Changelog ### Version 2.0 (Current) - ✨ Added ScrapingBee plan selector - ✨ Dynamic max pages calculation - ✨ API health check on startup - ✨ Real-time credit monitoring - ✨ Enhanced logging with quality scores - ✨ Comprehensive statistics dashboard - ✨ Incremental data saves (every 50 agents) - 🐛 Fixed crash recovery issues - 🐛 Improved error handling ### Version 1.0 - Initial release with basic scraping functionality ## 📄 License This actor is provided as-is for use on the Apify platform. By using this actor, you agree to comply with all applicable laws and the terms of service of both Apify and Zillow. --- Ready to generate leads? Start scraping now → --- Built with ❤️ for the Apify community