SEMrush Scraper - Domain Authority, Traffic & SEO Data Fast, reliable extraction of domain authority, traffic data, rankings, and competitor insights from SEMrush. No SEMrush account required. Perfect for SEO tracking, competitor monitoring, and market research. ## Features Extract 20+ SEO metrics from any domain in under 20 seconds: ### Traffic Metrics - Monthly visits - Total website traffic - Bounce rate - Visitor engagement percentage - Pages per visit - Average pages viewed per session - Visit duration - Time spent on site (in seconds) - Traffic change - Month-over-month growth/decline percentage ### Traffic Sources - Organic search visits - Traffic from search engines - Paid search visits - Traffic from paid ads - Direct traffic % - Percentage of direct visitors - Desktop traffic % - Desktop vs mobile breakdown - Mobile traffic % - Mobile device traffic share ### Rankings & Authority - Domain authority - SEMrush authority score (0-100) - Global rank - Worldwide ranking - Country rank - Ranking in primary country - Industry category - Business category classification ### Backlinks & SEO - Total backlinks - Number of incoming links - Referring domains - Unique domains linking to site ### Competitive Intelligence - Top 5 competitors - Similar websites by audience - Top 5 organic keywords - Best performing search terms with position & volume - Top 5 countries - Geographic traffic distribution with percentages ## Input json { "domains": [ "amazon.com", "apple.com", "make.com" ], "maxConcurrency": 3, "navigationTimeout": 60, "pageLoadDelay": 5000, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["DATACENTER"] } } ### Parameters - domains (required) - Array of domain names to analyze (e.g., ["example.com"]) - maxConcurrency (optional) - Number of pages to scrape simultaneously (default: 3, max: 10) - navigationTimeout (optional) - Maximum time to wait for page navigation in seconds (default: 60, max: 300). Increase if you get timeout errors. - pageLoadDelay (optional) - Additional wait time for dynamic content in milliseconds (default: 5000, max: 15000) - proxyConfiguration (optional) - Proxy settings (free Apify Datacenter proxies included by default) ## Output json { "domain": "make.com", "scrapedAt": "2025-11-10T20:43:27.750Z", "domain_authority": 55, "monthly_visits": 8300000, "bounce_rate": 37.22, "pages_per_visit": 6.68, "avg_visit_duration_seconds": 770, "traffic_change_percent": -2.91, "organic_search_visits": 657960, "paid_search_visits": 123320, "direct_visits_percent": 79.72, "desktop_traffic_percent": 76.92, "mobile_traffic_percent": 23.08, "global_rank": 6757, "country_rank": 7264, "country": "United States", "industry_category": "Computer Software and Development", "total_backlinks": 2370000, "referring_domains": 26590, "competitors": [ {"domain": "n8n.io"}, {"domain": "zapier.com"}, {"domain": "pipedream.com"} ], "organic_keywords": [ { "keyword": "make", "position": 1, "search_volume": 74000 } ], "top_countries": [ { "country": "United States", "traffic_percent": 16.96, "visits": 1410000 } ] } ## Use Cases ### 1. Competitor Analysis Monitor your competitors' traffic, keywords, and backlink profiles to identify opportunities. ### 2. SEO Monitoring Track domain authority, organic traffic, and keyword rankings over time. ### 3. Market Research Discover industry leaders, traffic patterns, and top-performing keywords in your niche. ### 4. Lead Generation Identify high-traffic websites in your target industry for partnership opportunities. ### 5. Portfolio Tracking Monitor multiple websites' performance metrics from a single dashboard. ## Performance - Speed: ~35-40 seconds per domain - Success rate: 95%+ on most domains - Concurrency: Process up to 10 domains simultaneously - Data freshness: SEMrush updates monthly (scrapes latest available data) ## Notes - No SEMrush account required - Scrapes publicly available data - Free Apify Datacenter proxies included - Works great for most use cases - Rate limiting: Respect SEMrush's infrastructure (max 10 concurrent requests) - Some metrics may be unavailable for low-traffic domains - Data is as fresh as SEMrush's latest update (typically monthly) ## Technical Details - Runtime: Node.js with Playwright - Browser: Chromium (headless mode) - Rendering: Full JavaScript execution for dynamic content - Extraction: Regex pattern matching on rendered page text - Proxy support: Free Apify Datacenter proxies (default) or custom proxies ## Support Having issues? Check these common solutions: 1. Timeout errors: Increase the navigationTimeout parameter (e.g., 120 seconds) or reduce maxConcurrency 2. Slow loading sites: Increase the pageLoadDelay parameter (e.g., 8000ms) to allow more time for dynamic content 3. Missing data: Some domains may have limited public data on SEMrush 4. Proxy blocks: Try increasing navigationTimeout or reducing maxConcurrency. Datacenter proxies work well for SEMrush. 5. Rate limiting: Reduce maxConcurrency (e.g., set to 1 or 2) to slow down request rate For bugs or feature requests, please contact support. ## Example Integrations ### Zapier/Make.com Trigger analysis when competitors launch new pages or campaigns. ### Google Sheets Export data automatically to spreadsheets for reporting. ### Slack/Discord Get alerts when competitors' traffic changes significantly. ### Custom Dashboards Feed data into Tableau, Power BI, or custom analytics platforms. --- No SEMrush subscription required. Get started analyzing domains today!