Website Image Scraper

# Website Image Scraper This Apify Actor crawls a website and extracts image URLs (e.g. .jpg, .png, .svg, etc.) without downloading them and without using Puppeteer or headless browsers. It is designed to be fast, lightweight, and resource-efficient — using plain HTTP requests and HTML parsing instead of browser automation. You can absolutely use it with less Memory to save some money. --- ## Features * Extracts all image URLs from <img src="..."> and CSS background-image styles * Recursively crawls internal links up to a specified depth and using a specified scope * Avoids duplicate URLs and pages * Supports modern formats (e.g. .webp, .svg) * Minimal (no Puppeteer, no Crawlee) * Respects robots.txt rules --- ## Input Provide a JSON input with the following fields: json { "startUrl": "https://example.com", "maxCrawlDepth": 1, "maxConcurrency": 10, "imageExtensions": ["jpg", "jpeg", "png", "gif", "webp", "svg"], "respectRobotsTxt": true, "userAgent": "Mozilla/5.0 (compatible; ApifyBot/1.0; +https://apify.com/bot)" } | Field | Type | Description | | ------------------ | ---------- | ------------------------------------------------------------ | | startUrl | string | The URL where crawling starts (required) | | maxCrawlDepth | number | How deep to follow internal links (default: 1) | | maxConcurrency | number | Number of concurrent requests (default: 10) | | imageExtensions | string[] | List of file extensions to extract (default: common formats) | | respectRobotsTxt | boolean | Whether to respect robots.txt rules (default: true) | | userAgent | string | User-Agent string to use for requests | --- ## Output The Actor outputs a JSON dataset with the following structure: json { "totalUrlsProcessed": 12, "totalImagesFound": 58, "images": [ { "url": "https://example.com/image1.jpg", "sourcePage": "https://example.com", "foundAt": "2025-07-17T20:30:00.000Z" } ] } This dataset can be exported in the Apify UI or via API. --- ## Limitations * Does not parse JavaScript-rendered content (no Puppeteer) * Only follows http(s) links found in <a href="..."> * Images must have absolute URLs (or relative URLs resolvable from page) to be captured * robots.txt parsing requires valid and well-formed robots.txt files --- ## Use Cases * Quickly analyze image usage across a site * Monitor websites for new or changed images * Create sitemaps of image assets * Pre-scan domains before downloading assets ## License Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0