🌟 The Fastest Instagram Metadata Scraper & Downloader in Apify One-line hero: Fast, privacy-safe Instagram metadata scraper + optional downloader — plug-and-play on Apify Console, enterprise-grade reliability, results in seconds. --- ## 📖 Summary This Actor extracts clean metadata (captions, hashtags, owner, thumbnails, durations, short human-friendly dates) from Instagram post/reel URLs and — optionally — provides direct download links or performs secure downloads with optional merging. Designed for fast, resilient runs and easy Console usage. > Instant outcome: feed a list of start URLs → get normalized JSON records in Dataset / Key-Value store in seconds. --- ## 💡 Use cases / When to use * Collect metadata for moderation, analytics, or cataloging (captions, hashtags, owner info). * Quickly fetch download links or download video/audio artifacts (optional, best-effort). * Backfill datasets for BI pipelines or social media research. * Lightweight monitoring of a set of public posts / reels for changes. --- ## ⚡ Quick Start (Console — one-click) 1. Open this Actor in Apify Console. 2. Paste your list of startUrls in the Input editor (see example below). 3. Click Run → results appear in the Dataset and under Key-Value store key OUTPUT. (Hero screenshot recommended: Console input + Run button — add a GIF for best conversion.) --- ## ⚙️ Quick Start (CLI + API) CLI (one-liner) bash # Run this Actor on Apify platform using Apify CLI # Replace <OWNER/ACTOR_NAME> and use your input file or pipe JSON echo '{"startUrls": ["https://www.instagram.com/reel/SHORTCODE/"]}' | apify call <OWNER/ACTOR_NAME> --silent --output-dataset Python (apify-client) — compact snippet python from apify_client import ApifyClient client = ApifyClient('<APIFY_TOKEN>') run = client.actor('OWNER/ACTOR_NAME').call(input={ 'startUrls': ['https://www.instagram.com/reel/SHORTCODE/'], 'mode': 'scrape', }) print('Run started:', run['id']) --- ## 📝 Inputs (fields & schema) Below is a compact Console JSON input example (also in input.example.json). The Actor accepts a JSON object. The platform's Input Schema (if provided by you) will auto-generate the Console UI. json { "mode": "scrape", "startUrls": [ "https://www.instagram.com/reel/SHORTCODE/", "https://www.instagram.com/p/SHORTCODE2/" ], "desired_resolution": "1080p", "download": false, "merge_if_ffmpeg": false, "hide_media_links": true, "preserve_thumbnails": true, "maxConcurrency": 3, "diagnostic": false, "suppress_errors": true } --- ## ⚙️ Configuration (friendly table) | 🔑 Name | 📝 Type | ❓ Required | ⚙️ Default | 📌 Example | 🧠 Notes | | -------------------- | ------- | ----------- | ---------- | -------------------------------------------------------------- | ------------------------------------------------ | | mode | string | ✅ Yes | "scrape" | "scrape" / "download" / "both" | Determines scrape vs. download behavior | | startUrls | array | ✅ Yes | None | ["https://.../reel/SHORTCODE/"] | List of post/reel URLs | | proxyConfiguration | object | ⚙️ Optional | {} | {"useApifyProxy": true} | Use Console proxy UI or custom proxy object | | preferred_proxy_type | string | ⚙️ Optional | "auto" | "residential" | Prefer residential proxies if needed | | force_residential | boolean | ⚙️ Optional | false | true | Alias for preferred proxy behavior | | download | boolean | ⚙️ Optional | false | true | If true, performs (best-effort) downloads | | desired_resolution | string | ⚙️ Optional | "1080p" | "720p" | Requested resolution for downloads | | merge_if_ffmpeg | boolean | ⚙️ Optional | false | true | If true and ffmpeg available, merges audio/video | | cookie_file | string | ⚙️ Optional | None | "cookies.txt" | Path in actor storage to cookie file (if used) | | hide_media_links | boolean | ⚙️ Optional | true | true | Redact raw media URLs in pushed records | | preserve_thumbnails | boolean | ⚙️ Optional | true | false | Keep thumbnail URLs even when hiding media | | maxConcurrency | integer | ⚙️ Optional | 3 | 5 | Controls parallelism (1–10) | | diagnostic | boolean | ⚙️ Optional | false | true | Enables verbose logs for debugging | | suppress_errors | boolean | ⚙️ Optional | true | false | Show errors in logs vs. suppress them | Console tip: Paste a single JSON object in the Input field (or select a saved Task) and click Run. --- ## 📄 Outputs (Dataset / KV examples) Where results land: * Metadata & download summaries are pushed to the Dataset (and final batch also stored to Key-Value under key OUTPUT). Example output record (schema-friendly) json { "original_url": "https://www.instagram.com/reel/SHORTCODE/", "id": "SHORTCODE", "ownerUsername": "creator_name", "ownerFullName": "Creator Name", "description": "Caption text...", "hashtags": ["fun", "reel"], "mentions": ["other_user"], "likesCount": 1234, "likesDisplay": "1.2k", "commentsCount": 12, "commentsDisplay": "12", "videoViewCount": 5432, "viewsDisplay": "5.4k", "upload_date_iso": "2024-10-04T12:34:56Z", "upload_date": "4th October 2024", "upload_time_ago": "1 Month Ago", "duration_seconds": 33.123, "duration_display": "0 Minutes 33 Seconds", "thumbnail": "https://.../thumbnail.jpg", "download_links_summary": { "available": { "merged_video": false, "video_without_audio": true, "audio": true } }, "randomExtraInfo": { "scrapeBatch": "Reel-Set-07", "regionCode": "US" } } > Note: raw media URLs may be redacted depending on hide_media_links and preserve_thumbnails settings (privacy-safe by default). --- ## 🔑 Environment Variables (placeholders) * APIFY_TOKEN — Set in Console integrations or use platform authentication. (Placeholder: <APIFY_TOKEN>) * HTTP_PROXY / HTTPS_PROXY — Only for custom proxy setups (use secrets, do not store plaintext). Example: <PROXY_USER:PASS@HOST:PORT> --- ## ▶️ How to Run (Console, CLI, API) 1. Console — Open Actor → paste JSON input → Click Run. Results appear in Dataset; key-value OUTPUT stores the full batch. 2. CLI — Use the apify CLI: echo '{"startUrls":["https://.../reel/SHORTCODE/"]}' | apify call <OWNER/ACTOR_NAME> --silent --output-dataset. 3. API / apify-client (Python) — See snippet above; use ApifyClient('<APIFY_TOKEN>').actor('OWNER/ACTOR_NAME').call(input=...). --- ## ⏰ Scheduling & Webhooks * Scheduling: Use Apify Console Scheduler to run this Actor periodically (e.g., hourly/daily). Choose your saved input task and frequency in Console. * Webhooks: Configure a run onFinish webhook in Console to receive run status and dataset IDs for downstream processing. --- ## 🕾️ Logs & Troubleshooting * Where to find logs: Open the run in Apify Console → Logs tab (filter by diagnostic to see verbose traces). * Common issues & quick fixes: * No startUrls provided → add startUrls to input. * Rate-limit / access errors → enable proxy or try preferred_proxy_type: "residential". * Downloads failing → ensure download is true and the run has network access; check diagnostic logs. --- ## 🔒 Permissions & Storage Notes * Storage used: Dataset for item records; Key-Value store under key OUTPUT for the result batch; temporary actor storage for downloads. * Privacy & safety: By default, raw media URLs are redacted (hide_media_links: true) and thumbnails preserved. Store credentials (cookies, proxy auth) only as Console secrets — never in plain input. --- ## 🔟 Changelog / Versioning * v1.0.0 — Initial release: metadata scraping + best-effort downloader, proxy-friendly, diagnostic mode. Use semantic versioning for future releases. --- ## 🖌 Notes / TODOs * TODO: confirm output schema — derived from runtime behavior but please verify against your integration. * TODO: add demo GIF/screenshots (Console Run + sample record) for higher conversion. --- ## 🌍 Proxy Configuration Enable Apify Proxy (quick): In Console → Integrations → Proxy → select Apify Proxy (recommended for reliability and rate-limit handling). Custom proxies: Provide proxyConfiguration in input or set HTTP_PROXY / HTTPS_PROXY as secrets. Example placeholders: json { "proxyConfiguration": { "useApifyProxy": false, "proxyUrls": ["http://<PROXY_USER:PASS@HOST:PORT>"] } } Environment variables (examples): bash export HTTP_PROXY="http://<PROXY_USER:PASS@HOST:PORT>" export HTTPS_PROXY="http://<PROXY_USER:PASS@HOST:PORT>" Security note: Store proxy credentials as Console secrets and do not place them directly in the input JSON. Advanced: TODO: Consider proxy rotation for large-scale scraping — use session-based proxy URLs to reduce rate-limiting. --- ## 📚 References * Apify Actor README guidance (official) * Actor input schema & Console Input generation (official) * Apify CLI & apify-client examples (official) --- ## 🤔 What I inferred from main.py * Network activity present (HTML metadata fetch + downloader) — Proxy configuration is supported and recommended. * Inputs: mode, startUrls, proxyConfiguration, preferred_proxy_type, download, desired_resolution, merge_if_ffmpeg, cookie_file, hide_media_links, preserve_thumbnails, maxConcurrency, diagnostic, suppress_errors. * Outputs: Dataset items (normalized metadata), Key-Value store key OUTPUT holds full batch. * The Actor attempts a primary extractor (metadata-first) with an HTML fallback and best-effort download link extraction; downloads are optional and non-fatal. --- Why this Actor? — Trusted, production-ready, and privacy-conscious: runs in Console with zero setup, yields normalized JSON ready for analytics pipelines, and keeps sensitive links hidden by default. Run this Actor on Apify Console — get instant results in seconds. --- # --- # input.example.json json { "mode": "scrape", "startUrls": [ "https://www.instagram.com/reel/SHORTCODE/" ], "desired_resolution": "1080p", "download": false, "hide_media_links": true, "preserve_thumbnails": true, "maxConcurrency": 3 } # CONFIG.md Quick config & proxy tips * Use Console Proxy integration for reliable runs. * If using custom proxies, add them as Console secrets and reference via proxyConfiguration or HTTP_PROXY env var. * If you enable downloads and merging, ensure ffmpeg is available in your run environment; merging is optional and automatic only when enabled. Advanced TODO * TODO: Add formal INPUT_SCHEMA.json for Console auto UI. * TODO: Add example Console screenshot and animated GIF for the README hero. --- End of generated README + examples. --- # actor.json json { "name": "merged-instagram-metadata-downloader", "title": "Merged Instagram Metadata & Downloader Actor", "version": "1.0.0", "main": "src/main.py", "buildTag": "latest", "memory": 4096, "timeout": 36000 } --- # INPUT_SCHEMA.json json { "$schema": "http://json-schema.org/draft-07/schema#", "title": "Merged Instagram Metadata & Downloader Actor - Input Schema", "type": "object", "properties": { "mode": { "type": "string", "enum": ["scrape", "download", "both"], "default": "scrape", "description": "Operation mode" }, "startUrls": { "type": "array", "items": { "anyOf": [{"type":"string"},{"type":"object","properties":{"url":{"type":"string"}}}] }, "description": "List of post/reel URLs (string or {\"url\": \"...\"})" }, "desired_resolution": { "type": "string", "default": "1080p" }, "download": { "type": "boolean", "default": false }, "merge_if_ffmpeg": { "type": "boolean", "default": false }, "cookie_file": { "type": "string" }, "hide_media_links": { "type": "boolean", "default": true }, "preserve_thumbnails": { "type": "boolean", "default": true }, "maxConcurrency": { "type": "integer", "minimum": 1, "maximum": 10, "default": 3 }, "diagnostic": { "type": "boolean", "default": false }, "suppress_errors": { "type": "boolean", "default": true }, "proxyConfiguration": { "type": "object" }, "preferred_proxy_type": { "type": "string", "enum": ["auto", "residential"], "default": "auto" }, "force_residential": { "type": "boolean", "default": false }, "proxy_country": { "type": "string" } }, "required": ["mode", "startUrls"], "additionalProperties": false } --- Notes: * actor.json gives minimal runtime metadata for Console packaging. Adjust memory and timeout based on your workload. * INPUT_SCHEMA.json will enable Console UI generation (auto form fields) when uploaded with the Actor. Update descriptions as needed. --- End of generated README + examples.