Steam Workshop Scraper - Extract Mods, Stats & Community Data Scrape Steam Workshop data at scale. Extract mod metadata, download statistics, descriptions, dependencies, changelogs, and community discussions from any Steam game's Workshop. Perfect for game developers, modders, researchers, and automation workflows. ## 🎯 What You Can Scrape ### Workshop Items Data - Mod Information: Titles, authors, Steam IDs, creation & update dates - Engagement Metrics: Subscriber counts, favorites, views, trending scores - Content: Full BBCode descriptions, plain text versions, multiple languages - Media Assets: Preview images, screenshots, video thumbnails - Relationships: Required mods, dependencies, mod collections - Tags & Categories: User-defined tags for filtering and discovery ### Community Content (Optional) - Version History: Change notes, update logs, patch notes - Discussions: Thread titles, posts, timestamps, authors - Trend Analysis: Track popularity changes over time ## 💡 Use Cases ### For Game Developers - Monitor popular mods in your game's Workshop - Identify trending content and community preferences - Track mod compatibility and dependency chains - Analyze player engagement with Workshop content ### For Modders & Creators - Research popular mod features and descriptions - Find complementary mods for dependencies - Track competitor mods and market trends - Analyze successful mod marketing strategies ### For Researchers & Analysts - Study gaming community behavior - Analyze user-generated content trends - Track mod ecosystem evolution - Sentiment analysis from discussions ### For Automation & Integration - Auto-update mod databases - Trigger notifications on new releases - Generate mod recommendation systems - Feed data into Discord bots or websites ## 🚀 Quick Start (3 Minutes) ### Step 1: Get Your Steam API Key (Free) 1. Visit steamcommunity.com/dev/apikey 2. Sign in with your Steam account 3. Enter any domain name (e.g., apify.com) 4. Copy your API key ### Step 2: Configure & Run json { "steamApiKey": "YOUR_KEY_HERE", "appId": [550], "maxItems": 10 } That's it! The Actor will automatically: - Discover trending Workshop items for Left 4 Dead 2 (App ID 550) - Fetch complete metadata via Steam API - Enrich with descriptions and media from pages - Save structured data to your dataset ### Step 3: View Results Access your dataset in multiple views: - Overview: All fields with clickable Steam URLs - Popular Items: Sorted by subscriber count - Dependencies: Mod relationships and requirements ## 📊 Popular Games Supported | Game | App ID | Workshop Items | |------|--------|----------------| | Left 4 Dead 2 | 550 | 187,000+ | | RimWorld | 294100 | 45,000+ | | Cities: Skylines | 255710 | 250,000+ | | Counter-Strike 2 | 730 | 1,000,000+ | | Garry's Mod | 4000 | 500,000+ | | Skyrim | 72850 | 28,000+ | | Ark: Survival | 346110 | 85,000+ | Find your game's App ID: Visit store.steampowered.com/app/APPID/ in the game's URL ## ⚙️ Configuration Options ### Basic Settings Steam API Key (required) Your Steam Web API authentication key App IDs (required) One or more game Workshop IDs to scrape. Examples: - [550] - Left 4 Dead 2 - [294100, 255710] - RimWorld + Cities: Skylines Max Items (default: 1000) Maximum items to scrape per game. Range: 1-50,000 ### Filtering Options Search Text Find mods containing specific keywords in title/description Tags Filter by Workshop tags: ["weapons", "QoL", "graphics"] Updated Since Only scrape items updated after date: "2025-01-01" Sort Order - trend - Currently trending (default) - top - Most subscribed all-time - new - Recently published - updated - Recently updated ### Data Enrichment Include Descriptions (default: true) Scrape full BBCode and text from pages (API provides stripped text only) Include Media (default: true) Extract preview images, screenshots, video URLs Include Dependencies (default: true) Find required mods and mod relationships Include Change Notes (default: false) Scrape version history and changelogs (slower) Include Discussions (default: false) Extract community threads and posts (significantly slower) ### Advanced Options Proxy Configuration Use Apify's residential proxies to avoid blocking (recommended) Request Interval Delay between page requests: 1000-10000ms (default: 1200) Debug Mode Enable verbose logging for troubleshooting ## 📦 Output Format Each Workshop item is saved as a structured JSON record: json { "appId": 550, "publishedFileId": "3578582211", "url": "https://steamcommunity.com/sharedfiles/filedetails/?id=3578582211", "title": "TW-201-S Morita I (Combat Rifle)", "author": { "name": "Adorabirb", "steamId": "76561198073176857", "profileUrl": "https://steamcommunity.com/profiles/76561198073176857" }, "stats": { "subscriptions": 69, "favorites": 47, "views": 430, "created": 1759370331, "updated": 1759370331 }, "tags": ["Weapons", "Rifle"], "dependencies": [], "media": { "preview": "https://...", "additionalPreviews": ["https://...", "https://..."] }, "description": { "bbcode": "[h1]Features[/h1][list][*]Custom animations...", "text": "Features\n- Custom animations...", "language": "en" }, "changeNotes": [], "discussions": [], "hash": "9bf0c7e658befa85f36232f43e93285f55e28665", "scrapedAt": "2025-10-02T15:36:18.280Z" } ### Key Fields Explained - publishedFileId: Unique Workshop item ID (use as primary key) - hash: Content hash for change detection (changes when item updates) - stats.updated: Unix timestamp of last modification - dependencies: Array of required mods - media.additionalPreviews: All screenshots/videos - description.bbcode: Raw BBCode for rendering - scrapedAt: ISO timestamp of when data was collected ## 💰 Pricing Pay-Per-Event Model: You only pay for successfully scraped records. - Event: record-ingested (one per Workshop item) - No charges for: Failed requests, API calls, or retries - Predictable: Cost scales with the number of items scraped See current pricing in the Apify Console when starting a run. Pricing varies based on enrichment options enabled (descriptions, discussions, etc.). ## 🔗 Integration Examples ### n8n Workflow javascript // 1. Trigger: Schedule (daily at 8am) // 2. Apify: Run Actor { "actorId": "YOUR_ACTOR_ID", "input": { "steamApiKey": "{{$credentials.steamApiKey}}", "appId": [550], "maxItems": 100, "filters": { "updatedSince": "{{$now.minus({days: 1}).toISO()}}" } } } // 3. HTTP Request: Get dataset items // 4. Google Sheets: Upsert by publishedFileId // 5. Slack: Notify new mods > 100 subscribers ### Zapier Integration 1. Trigger: New run finished (Apify webhook) 2. Action: Get dataset items (HTTP) 3. Filter: Only items with stats.subscriptions > 1000 4. Action: Add to Airtable 5. Action: Post to Discord channel ### API Integration bash # Start a run curl -X POST https://api.apify.com/v2/acts/YOUR_ACTOR_ID/runs \ -H "Authorization: Bearer YOUR_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"steamApiKey":"KEY","appId":[550],"maxItems":50}' # Get results curl https://api.apify.com/v2/datasets/DATASET_ID/items ## ⚡ Performance & Limits ### Speed - API Discovery: ~1 second per 100 items - Basic Scraping: ~9 seconds per item average - With Discussions: ~15-20 seconds per item - Throughput: 15-20 items/minute ### Limits - Steam API: 100,000 calls/day per key - Batch Size: 100 items per GetDetails call - Concurrent Pages: 3 browser contexts - Max Items: 50,000 per run (soft limit) ### Optimization Tips - Use itemIds to scrape specific items only - Disable discussions unless needed (5x faster) - Filter by updatedSince for incremental updates - Run on schedule instead of scraping all items repeatedly ## 🛡️ Anti-Blocking & Reliability ### Built-in Protection - ✅ Residential Proxies: Apify's rotating proxy network - ✅ Rate Limiting: 1.2s delay between requests (configurable) - ✅ Retry Logic: 3 attempts with exponential backoff - ✅ Circuit Breaker: Automatic pause on repeated failures - ✅ Stealth Mode: Browser fingerprint masking - ✅ User-Agent Rotation: Randomized headers ### Compliance - Respects robots.txt (Workshop paths are allowed) - API-first approach (minimizes page scraping) - Configurable delays and concurrency - Graceful error handling ## 🐛 Troubleshooting ### "403 Forbidden" Error Cause: Invalid or missing Steam API key Fix: Get a new key from steamcommunity.com/dev/apikey ### "No items found" Causes: - Wrong App ID - Overly restrictive filters (tags/search) - Game has no Workshop content Fix: Visit steamcommunity.com/app/APPID/workshop/ to verify Workshop exists ### "Item page not found" Warnings Normal behavior: Some items are deleted/hidden by authors Impact: None - Actor continues with remaining items ### Slow Performance Solutions: - Disable includeDiscussions (default: off) - Disable includeChangeNotes if not needed - Reduce maxItems for testing - Increase requestIntervalMs if getting rate limited ### Dataset Schema Not Showing Verified Fix: Files are correctly placed in .actor/ directory with proper storages.dataset configuration ## 📚 FAQ Q: Do I need a Steam account? A: Yes, to get an API key (free, takes 2 minutes) Q: Can I scrape private Workshop items? A: No, only public items are accessible Q: How often should I run this? A: Daily or weekly for trends, hourly for time-sensitive monitoring Q: Can I scrape multiple games at once? A: Yes! Use "appId": [550, 294100, 255710] Q: What happens if I hit the API limit? A: Actor logs a warning and stops gracefully. Increase delay or split into multiple runs. Q: Can I get historical data? A: No, only current state. Run regularly to build history. Q: Is this legal? A: Yes - scrapes public data, respects robots.txt, uses official API where available. ## 🆘 Support & Feedback - Email: kontakt@barrierefix.de - Issues: Report bugs or request features via email - Updates: Follow actor changelog in Apify Console ## 🏷️ Keywords Steam Workshop API, Steam mods scraper, gaming data extraction, Workshop statistics, mod metadata, Steam community data, user-generated content analysis, game Workshop scraper, Steam API integration, mod dependency tracker, Workshop trend analysis, gaming analytics, mod recommendation data, Steam Workshop automation, L4D2 mods, RimWorld Workshop, Cities Skylines mods, Counter-Strike Workshop, Steam mod database, Workshop items export, gaming community insights --- ## 🔗 Explore More of Our Actors ### ⚽ Sports & Entertainment | Actor | Description | |-------|-------------| | Transfermarkt Scraper | Extract football player transfers and market values | ### 📰 Content & Publishing | Actor | Description | |-------|-------------| | Notion Marketplace Scraper | Scrape Notion templates and marketplace listings | | Ghost Newsletter Scraper | Extract Ghost newsletter content and subscriber data | | Farcaster Hub Scraper | Scrape Farcaster decentralized social network data | | Google Play Reviews Scraper | Extract app reviews from Google Play Store | --- --- Built with: Apify SDK v3, Crawlee v3, Playwright, TypeScript, Zod Version: 1.0.0 Last Updated: October 2025 ## Legal Disclaimer / Rechtlicher Hinweis EN: This actor is a general-purpose tool for analyzing publicly accessible web data. The user bears sole responsibility for ensuring their specific use complies with: - Applicable laws (GDPR/DSGVO, copyright law) - The target website's Terms of Service - Apify's Terms of Service The provider (barrierefix) expressly disclaims liability for any unauthorized or unlawful use. By using this actor, the user agrees to indemnify the provider against any third-party claims arising from their use of the data. DE: Dieser Actor ist ein allgemeines Werkzeug zur Analyse öffentlich zugänglicher Webdaten. Der Nutzer trägt die alleinige Verantwortung dafür, dass seine spezifische Nutzung den geltenden Gesetzen (DSGVO, Urheberrecht), den Nutzungsbedingungen der Zielwebsite und den Apify-Nutzungsbedingungen entspricht. Der Anbieter (barrierefix) schließt jegliche Haftung für unbefugte oder rechtswidrige Nutzung ausdrücklich aus. Mit der Nutzung dieses Actors erklärt sich der Nutzer bereit, den Anbieter von allen Ansprüchen Dritter freizustellen, die aus seiner Datennutzung entstehen. --- This tool is not affiliated with Steam/Valve. All trademarks belong to their respective owners.