Website Recovery Actor Recover and reverse-engineer website files from a live or deployed site. This actor downloads the complete website including HTML, CSS, JavaScript, images, fonts, and all other assets, then rewrites URLs so everything works locally. Perfect for: - Recovering lost source files from Netlify, Vercel, GitHub Pages, or any deployed site - Creating offline copies of websites - Migrating sites to new hosting - Archiving web content - Analyzing site structure and assets ## Features - Full JS Rendering: Uses Puppeteer to capture JavaScript-rendered content (React, Vue, Next.js, etc.) - Complete Asset Download: Downloads all CSS, JS, images, fonts, videos, and other media - CDN Support: Optionally downloads assets from external CDNs (Google Fonts, Cloudflare, jsDelivr) - URL Rewriting: Converts absolute URLs to relative paths so the site works locally - Structure Preservation: Maintains original URL path structure for easy navigation - CSS Asset Extraction: Parses CSS files to find and download additional assets (fonts, background images) - Inline Style Extraction: Optionally extracts inline <style> tags to separate files - Concurrent Downloads: Processes multiple pages and assets in parallel for speed - Proxy Support: Access geo-restricted or blocked sites using Apify Proxy ## How It Works 1. Crawling: The actor visits your start URL and follows links to discover all pages on the same domain 2. Rendering: Each page is fully rendered using a headless Chrome browser to capture dynamic content 3. Asset Extraction: All linked assets (CSS, JS, images, fonts) are identified and queued for download 4. Download: Assets are downloaded in parallel batches 5. URL Rewriting: All URLs in HTML and CSS files are rewritten to use relative local paths 6. Output: Files are saved to Apify's Key-Value Store with a manifest file listing everything ## Input Configuration | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | startUrl | string | (required) | The URL of the website to recover | | maxDepth | integer | 5 | How deep to follow links (0 = only start page) | | maxPages | integer | 100 | Maximum number of pages to crawl | | downloadAssets | boolean | true | Download CSS, JS, images, fonts, etc. | | rewriteUrls | boolean | true | Convert URLs to relative paths | | includeExternalAssets | boolean | true | Download assets from CDNs | | maxConcurrency | integer | 5 | Parallel page processing | | downloadTimeout | integer | 30000 | Asset download timeout (ms) | | userAgent | string | Chrome UA | Browser user agent | | waitForSelector | string | - | CSS selector to wait for (JS sites) | | extractInlineStyles | boolean | false | Extract inline CSS to files | | preserveStructure | boolean | true | Maintain URL path structure | | proxyConfiguration | object | - | Proxy settings | ## Output The actor outputs all files to the Key-Value Store: - HTML pages with rewritten URLs - CSS files with rewritten asset paths - JavaScript files - Images (PNG, JPG, GIF, WebP, SVG, ICO) - Fonts (WOFF, WOFF2, TTF, OTF) - Videos and audio files - __MANIFEST__ - JSON file listing all downloaded files with their original URLs ### Downloading Your Files After the run completes: 1. Go to the Storage tab 2. Click on Key-Value Store 3. Click Export to download all files as a ZIP 4. Extract the ZIP and you have your recovered website! ## Example Usage ### Basic Recovery json { "startUrl": "https://your-site.netlify.app" } ### Full Site with All Assets json { "startUrl": "https://example.com", "maxDepth": 10, "maxPages": 500, "downloadAssets": true, "includeExternalAssets": true, "rewriteUrls": true } ### JavaScript-Heavy Site (React/Vue/Next.js) json { "startUrl": "https://my-react-app.vercel.app", "waitForSelector": "#root", "maxDepth": 5, "maxPages": 100 } ### Single Page Only json { "startUrl": "https://example.com/specific-page", "maxDepth": 0, "maxPages": 1 } ## Tips for Best Results 1. For JS-rendered sites: Use waitForSelector to ensure content loads before capture 2. Large sites: Increase maxPages and maxConcurrency 3. Slow sites: Increase downloadTimeout 4. Blocked sites: Enable proxy configuration 5. CDN-heavy sites: Keep includeExternalAssets enabled ## Limitations - Cannot recover server-side logic (APIs, databases, authentication) - Some minified/bundled JS may be difficult to understand - Dynamic content that requires user interaction won't be captured - Login-protected content requires additional setup ## Technical Details - Built with Apify SDK 3.x and Crawlee - Uses Puppeteer for browser automation - Cheerio for HTML parsing - Respects robots.txt (can be disabled) ## Cost Estimate Typical usage costs approximately: - $1-3 per 100 pages with full asset download - Larger sites with many assets may cost more ## License ISC License ## Support For issues or feature requests, please open an issue on the repository.