Facebook Ads Library Scraper Fast and reliable scraper for Facebook Ads Library that can extract thousands of ads from any Facebook Page. Built with automatic rate limit handling, proxy rotation, and intelligent session management. ## ✨ Key Features - 🚀 High Performance: Scrapes ~5-10 ads/second with optimized GraphQL requests - 🔄 Automatic Rate Limit Handling: Automatically switches IPs and renews sessions when rate limited - 🌍 Smart Proxy Rotation: Rotates through multiple countries (US, UK, CA, AU, DE) for better success rates - 💪 Robust & Reliable: Multiple retry attempts with exponential backoff - 📊 Unlimited Scraping: Can scrape thousands of ads from any page (tested with 15,000+ ads) - ⚡ Real-time Streaming: Results saved immediately to dataset - 🎯 Flexible Filtering: Filter by country, status, ad type, and more ## 📊 What You Can Extract Each ad includes comprehensive data: - Ad Content: Text, titles, descriptions, call-to-action - Media: Images, videos, carousel content - Targeting: Demographics, locations, interests - Performance: Impressions, spend ranges (when available) - Metadata: Creation time, status, page info - URLs: Ad snapshot links, landing pages ## 🎯 Use Cases - 📈 Competitive Analysis: Monitor competitor ad strategies - 🔍 Market Research: Discover industry trends and messaging - 📱 Social Media Monitoring: Track brand campaigns - 💼 Business Intelligence: Analyze advertising tactics - 🎨 Creative Inspiration: Study successful ad creatives - 📉 Spend Tracking: Monitor advertising investments ## 🚀 Quick Start ### Option 1: Using Page ID json { "pageId": "109806077081081", "country": "US", "activeStatus": "all", "maxAds": 5000 } ### Option 2: Using Full URL json { "url": "https://www.facebook.com/ads/library/?active_status=all&ad_type=all&country=ALL&search_type=page&view_all_page_id=109806077081081", "maxAds": 5000, "requestDelay": 100 } ### Option 3: Unlimited Scraping json { "pageId": "109806077081081", "requestDelay": 100 } Leave maxAds empty to scrape all available ads ## 📝 Input Parameters | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | url | string | * | - | Full Facebook Ads Library URL | | pageId | string | * | - | Facebook Page ID (alternative to URL) | | country | string | No | "ALL" | Country code (US, GB, ALL, etc.) | | activeStatus | string | No | "all" | Filter: all, active, or inactive | | adType | string | No | "all" | Filter: all, political, housing, employment, credit | | maxAds | integer | No | unlimited | Maximum number of ads to scrape | | requestDelay | integer | No | 500 | Delay between requests in milliseconds | | useProxy | boolean | No | true | Enable Apify proxy rotation after rate limiting | | maxSessionRenewals | integer | No | 10 | Max times to renew session when rate limited | * Either url or pageId is required ## 💡 How It Works ### Smart Architecture 1. Browser Initialization: Launches Playwright browser to capture authentication tokens from Facebook 2. Token Extraction: Intercepts GraphQL requests to extract necessary authentication headers 3. Direct API Requests: Makes fast, direct GraphQL API calls using captured tokens 4. Cursor-Based Pagination: Efficiently navigates through results using Facebook's cursor system 5. Rate Limit Detection: Automatically detects rate limiting (error code 1675004) 6. Intelligent Recovery: When rate limited: - Closes current browser session - Waits with exponential backoff (30s-120s) - Creates new session with different proxy IP - Rotates through 5 countries for IP diversity - Retries up to 6 times before falling back to no-proxy - Resumes exactly where it left off ### Rate Limit Handling Flow Rate limit detected → Wait 30-120s → Try proxy (US) → Failed? → Wait 10s → Try proxy (GB) → Failed? → Wait 20s → Try proxy (CA) → Failed? → Wait 30s → Try proxy (AU) → Failed? → Wait 40s → Try proxy (DE) → Failed? → Wait 50s → Try without proxy → Failed? → Stop with error ## 📤 Output Format Results are saved to Apify dataset in JSON format: json { "ad_archive_id": "1234567890", "page_id": "109806077081081", "page_name": "Example Brand", "ad_creation_time": "2024-01-15T10:30:00+0000", "ad_creative_bodies": ["Check out our new collection!"], "ad_creative_link_titles": ["Shop Now"], "ad_creative_link_descriptions": ["Free shipping on orders over $50"], "ad_snapshot_url": "https://www.facebook.com/ads/library/?id=1234567890", "currency": "USD", "spend": { "lower_bound": "100", "upper_bound": "500" }, "impressions": { "lower_bound": "10000", "upper_bound": "50000" }, "scrapedAt": "2024-11-17T08:30:00.000Z" } ## 🎛️ Advanced Configuration ### High-Speed Scraping For maximum speed (with higher rate limit risk): json { "pageId": "109806077081081", "requestDelay": 100, "maxSessionRenewals": 15 } ### Conservative Scraping For slower but more reliable scraping: json { "pageId": "109806077081081", "requestDelay": 1000, "maxSessionRenewals": 5 } ## ⚡ Performance - Speed: 5-10 ads per second (varies by rate limiting) - Reliability: 99% success rate with automatic recovery - Scale: Tested with 15,000+ ads per run - Efficiency: ~10 ads per GraphQL request ## 🛡️ Best Practices 1. Start with limits: Test with maxAds: 100 first 2. Monitor logs: Check Actor logs for rate limit handling 3. Adjust delays: Increase requestDelay if frequently rate limited 4. Enable proxies: Keep useProxy: true for automatic recovery 5. Be patient: Let the actor handle rate limits automatically 6. Export data: Download dataset after completion ## ❓ Common Issues ### "Failed to capture GraphQL tokens" Solution: This usually happens with blocked proxy IPs. The actor will automatically retry with different IPs. If all retries fail, it may indicate Facebook is blocking the proxy provider. ### Frequent Rate Limiting Solution: - Increase requestDelay to 1000ms or higher - Increase maxSessionRenewals for more retry attempts - Ensure useProxy: true is enabled ### Duplicate Ads Solution: The actor automatically deduplicates ads by ad_archive_id. Duplicates in output are rare. ## 📊 Cost Estimation Typical costs on Apify (approximate): - 1,000 ads: ~0.01-0.02 credits - 10,000 ads: ~0.10-0.20 credits - 100,000 ads: ~1.00-2.00 credits Costs vary based on rate limiting, proxy usage, and request delays ## 🔧 Technical Details - Platform: Apify Actor (Node.js) - Browser: Playwright with Chromium - Proxy: Apify Residential Proxies with country rotation - API: Facebook GraphQL API (AdLibrarySearchPaginationQuery) - Storage: Apify Dataset with immediate persistence ## 📞 Support & Feedback Having issues or suggestions? - Check Actor logs for detailed error messages - Review input parameters for correctness - Ensure Page ID or URL is valid - Contact Apify support for platform issues ## ⚖️ Legal & Ethical Use This actor scrapes publicly available data from Facebook's Ad Library. Users are responsible for: - Complying with Facebook's Terms of Service - Respecting data privacy and applicable laws - Using scraped data ethically and legally - Not overwhelming Facebook's servers Note: This tool is for research, analysis, and competitive intelligence purposes only. ## 🚀 Version History - v1.0: Initial release with basic GraphQL scraping - v2.0: Added automatic rate limit handling and proxy rotation - v3.0: Implemented country rotation and session management - Current: Robust retry logic with 6-attempt fallback system --- Made with ❤️ for the Apify community Extract insights, stay competitive, and make data-driven decisions with Facebook Ads Library Scraper.