🚀 Amazon Search Scraper One-line tagline: Instantly extract product listings and product-page metadata from Amazon search queries — fast, secure, and ready for business use. --- ## 📖 Summary Amazon Search Scraper retrieves product data from Amazon search results and product pages and returns clean, business-ready JSON records for analysis. Designed for speed and reliability, it helps teams discover product details, prices, reviews, images, and availability for competitive research, analytics, and monitoring. --- ## 💡 Use cases / When to use * Competitive price monitoring and alerting * Market/product research and sourcing * Creating product catalogs and feeds * Gathering product images and descriptions for analytics * Quickly prototyping e-commerce dashboards --- ## ⚡ Quick Start — Console (one-click) 1. Open this Actor in Apify Console. 2. Fill the Queries input (single keyword or array of keywords). 3. (Optional) Enable Proxy Configuration if scraping at scale. 4. Click Run. Results appear in the default dataset/OUTPUT in seconds. > Friendly microcopy: “Plug in a search term, click Run, and get structured product data instantly.” --- ## ⚙️ Quick Start (CLI + API) CLI bash # Run an actor with JSON input via apify-cli apify run --actor <your-actor-id> --input input.example.json Python (apify-client) python from apify_client import ApifyClient client = ApifyClient('<APIFY_TOKEN>') run = client.actor('your-user/amazon-search-scraper').call(run_input={"queries": ["wireless earbuds"]}) print('Started run:', run['id']) --- ## 📝 Inputs (fields & schema) Console JSON input example (see input.example.json file): json { "queries": ["wireless earbuds", "gaming mouse"], "headless": true, "requestDelay": [1.0, 2.0] } Fields * queries — string or array — required — Search keywords or Amazon product URLs. The actor accepts either a search keyword (will run site search) or a direct Amazon product URL for product-page scraping. * headless — boolean — optional — Run browser in headless mode (default: true). * requestDelay — array [min, max] — optional — Delay range (seconds) between product-page requests to reduce rate. --- ## ⚙️ Configuration | 🔑 Name | 📝 Type | ❓ Required | ⚙️ Default | 📌 Example | 🧠 Notes | | -----------------: | --------------: | ----------: | ---------: | ----------------------: | --------------------------------- | | queries | string/array | ✅ Yes | — | ["wireless earbuds"] | Search terms or product URLs | | headless | boolean | ⚙️ Optional | true | false | Turn off to debug visually | | requestDelay | array (min,max) | ⚙️ Optional | [1.0, 2.0] | [0.5, 1.0] | Avoids aggressive scraping | | proxyConfiguration | object | ⚙️ Optional | {} | {"useApifyProxy": true} | Use residential proxies for scale | Example Console setup: paste "wireless earbuds" into queries and press Run Actor. --- ## 📄 Outputs (Dataset / KV examples) Each dataset item is a JSON object with attributes similar to: json { "asin": "B09EXAMPLE", "title": "Wireless Earbuds XYZ", "brand_name": "BrandCo", "url": "https://www.amazon.com/...]", "price": "$59.99", "currency": "$", "thumbnail": "https://...jpg", "images": ["https://...jpg", "https://...jpg"], "stars": 4.3, "review_count": "1.2k", "availability": "In Stock", "description": "Key bullet points...", "categories": "Electronics > Audio", "search_keyword": "wireless earbuds" } > Note: The actor pushes results to the default dataset and also returns the full result array in the key-value store under the standard OUTPUT key. --- ## 🔑 Environment Variables * <APIFY_TOKEN> — required to call the Actor programmatically via API. * <PROXY_USER:PASS@HOST:PORT> — placeholder for custom proxy credentials. Security note: Store secrets in Apify Console Secrets — do not paste them into input fields. --- ## ▶️ How to Run Console 1. Go to the Actor page in Apify Console. 2. Paste your queries (single string or array) into the Input field. 3. (Optional) Configure proxies under the Proxy Configuration editor. 4. Click Run. CLI bash apify run --actor your-user/amazon-search-scraper --input input.example.json API (Python) See Quick Start (above) — use client.actor(...).call(run_input=...) and read the returned run ID. --- ## ⏰ Scheduling & Webhooks * Use Apify Console scheduling to run this Actor at any interval (hourly, daily, weekly). * Configure webhooks on run completion to forward JSON output to your endpoint for real-time processing. --- ## 🕾️ Logs & Troubleshooting * Check the Console logs for step-by-step run info and any per-item warnings. * Common issues: * No results: verify that queries are valid and spelled correctly. * Request timeouts: increase requestDelay or enable Proxy Configuration. * Selector changes on Amazon: refresh the run and adjust queries — the actor is resilient but web UIs change frequently. --- ## 🔒 Permissions & Storage Notes * This Actor collects publicly visible product information only. It does not perform account actions. * Results are stored in Apify Datasets/Key-Value stores in your account and follow Apify’s standard retention and access controls. --- ## 🔟 Changelog / Versioning * v0.1.0 — Initial public release: search + product-page scraping, structured dataset output. --- ## 🖌 Notes / TODOs * TODO: Consider adding a CLI flag / input for limiting the number of product pages per query (reason: some queries return many results). * TODO: Add optional CSV export in output settings (reason: convenient for BI ingestion). --- ## 🌍 Proxy Configuration If you will run many searches or large-scale scraping, configure Apify Proxy or custom proxies. Enable Apify Proxy (Console): * In the Actor run form, open Proxy configuration and enable Use Apify Proxy (choose RESIDENTIAL for best results). Custom proxy example (as secret): * Use <PROXY_USER:PASS@HOST:PORT> format and store as a Console Secret. Reference it in the Proxy Configuration editor. Environment variables (examples) HTTP_PROXY=<PROXY_USER:PASS@HOST:PORT> HTTPS_PROXY=<PROXY_USER:PASS@HOST:PORT> Reminder: Store proxy credentials in Secrets and do not paste them into public inputs. TODO: Consider proxy rotation for large-scale scraping. --- ## 📚 References 1. Apify Actor README guidelines — https://docs.apify.com/console/actors/README 2. Apify Input/Output schemas — https://docs.apify.com/platform/input-output 3. Apify CLI & API usage — https://docs.apify.com/console/actors/run --- ## 🤔 What I inferred from main.py * The Actor accepts queries (keywords or Amazon URLs) and uses an automated browser to fetch search results and product pages. * It extracts product metadata, images, prices, ratings, and availability and returns structured JSON items. * It respects throttling delays and can be configured to use proxies for scale. * Results are pushed to the default dataset and the key-value store under OUTPUT. --- --- # input.example.json json { "queries": [ "wireless earbuds", "gaming mouse" ], "headless": true, "requestDelay": [1.0, 2.0] } --- # CONFIG.md (optional) ## Quick config notes * Secrets: Add <APIFY_TOKEN> and any proxy credentials to Console Secrets. * Scaling: For repeated large runs, enable Apify Proxy (RESIDENTIAL) and consider running with scheduling + webhooks to automate downstream processing. ## Suggested settings in Console * Proxy configuration: use Apify Proxy → RESIDENTIAL * Dataset retention: enable automatic export to your storage of choice --- Generated by: Neuro Scraper