Site Researcher - Full Website Media Scraper Extract all images, videos, and data from any website in one click. ## What It Does - 🗺️ Discovers all pages via sitemap crawling - 📊 Extracts structured data (JSON-LD, meta tags, Open Graph) - 🔧 Detects tech stack (Shopify, WordPress, React, etc.) - 📷 Downloads all images to Key-Value Store - 🎬 Downloads all videos found on pages - 📄 Outputs complete research dossier as JSON ## Use Cases - Competitor research - Analyze any website's structure and content - B-roll production - Download all media for video production - Market analysis - Extract product data from e-commerce sites - Content inspiration - Study design and tech choices ## Input | Field | Type | Default | Description | |-------|------|---------|-------------| | startUrl | String | Required | Website URL to research | | deepCrawl | Boolean | true | Crawl all sitemap pages (up to 50) | | downloadMedia | Boolean | true | Download images and videos | | mediaLimit | Integer | 100 | Max media files to download | ## Output json { "domain": "example.com", "crawled_at": "2024-12-13T12:00:00Z", "pages": [ { "url": "https://example.com/products/item", "title": "Product Name", "description": "...", "og_data": { "title": "...", "image": "..." }, "json_ld": [ { "@type": "Product", ... } ], "headings": [ { "level": 1, "text": "..." } ], "images": [ { "src": "...", "alt": "..." } ], "videos": [ { "src": "...", "type": "video/mp4" } ] } ], "sitemap_urls": ["..."], "tech_stack": { "shopify": true, "cloudflare": true }, "media": { "images": [...], "videos": [...] } } ## Media Files Downloaded media is saved to the Actor's Key-Value Store: - images/filename.jpg - videos/filename.mp4 ## Pricing Pay-per-result: ~$0.01 per page crawled + $0.005 per media file downloaded. ## Support Questions? Open an issue or contact the developer.