Steam Store Scraper

Extract comprehensive game data from Steam Store including prices, discounts, reviews, genres, screenshots, system requirements, and detailed metadata. Fast, reliable, and production-ready scraper built for gaming platforms, price comparison sites, and market research.

## Overview

This scraper enables you to collect detailed information about games available on Steam Store, the world's largest digital distribution platform for PC gaming. Extract pricing data, user reviews, game specifications, developer information, and much more with a single configuration.

Key capabilities: Multi-strategy data extraction (JSON API + HTML parsing), automatic fallback mechanisms, concurrent processing for high performance, comprehensive game metadata collection, and flexible filtering options.

## Features

• Comprehensive Data Extraction - Game titles, descriptions, pricing, discounts, reviews, genres, tags, screenshots, and system requirements
• Multi-Strategy Scraping - Prioritizes JSON API for speed, automatically falls back to HTML parsing when needed
• Advanced Filtering - Sort by price/reviews/release date, filter by language/category/tags/price range
• High Performance - Concurrent request processing with configurable concurrency levels (1-10 parallel requests)
• Detailed Game Information - Optional deep scraping for full descriptions, screenshots, system requirements, and metadata
• Production Ready - Built-in error handling, proxy support, timeout management, and comprehensive logging
• Flexible Configuration - Customize results count (1-1000), pagination (1-20 pages), and search parameters

## Use Cases

Gaming Platforms & Aggregators

Build comprehensive game databases, create price comparison tools, aggregate game reviews and ratings across platforms

Price Tracking & Monitoring

Track game prices over time, monitor discount patterns, alert users about price drops and sales

Market Research & Analytics

Analyze gaming industry trends, study pricing strategies, identify popular genres and tags, research developer/publisher portfolios

Gaming News & Media

Track new releases, monitor upcoming games, collect game specifications for reviews and articles

Competitive Intelligence

Monitor competitor game launches, analyze pricing strategies, track review scores and player sentiment

E-commerce & Retail

Import Steam game data for resale platforms, create automated catalogs, maintain up-to-date pricing information

## Quick Start

Basic Configuration - Extract 50 Games

{ "maxResults": 50, "maxPages": 3, "collectDetails": true, "language": "english", "maxConcurrency": 3 }

Filter by Price - Games Under $20

{ "maxResults": 100, "maxPrice": "20", "sortBy": "Price_ASC", "collectDetails": true }

Custom Search URL - Specific Steam Category

{ "startUrl": "https://store.steampowered.com/search/?sort_by=Reviews_DESC&supportedlang=english&category1=21", "maxResults": 75, "collectDetails": true }

Best Rated Games - Sorted by Reviews

{ "sortBy": "Reviews_DESC", "maxResults": 100, "maxPages": 5, "collectDetails": true, "language": "english" }

## Input Parameters

Parameter	Type	Required	Description
startUrl	String	No	Custom Steam Store search URL. If provided, overrides other search parameters. Use this for advanced filtering or specific searches.
sortBy	String	No	Sort order for results. Options: _ASC (relevance), Released_DESC (newest first), Price_ASC, Price_DESC, Name_ASC, Reviews_DESC. Default: _ASC
category	String	No	Steam category ID filter. Common values: 998 (All), 21 (Action), 23 (Indie), 9 (Strategy), 3 (RPG), 1 (Action). Leave empty for all categories.
maxPrice	String	No	Maximum price filter (e.g., "20" for $20). Leave empty to include all prices. Useful for finding budget games or filtering expensive titles.
tags	String	No	Filter by Steam tag IDs (comma-separated). Example: "1695" for Open World. Combine multiple tags for precise filtering.
language	String	No	Supported language filter. Options: english, spanish, french, german, italian, japanese, korean, portuguese, russian, schinese, tchinese. Default: english
collectDetails	Boolean	No	Enable to collect comprehensive game details including full descriptions, system requirements, screenshots, and metadata. Increases runtime but provides richer data. Default: true
maxResults	Integer	No	Maximum number of games to scrape (1-1000). Higher values increase runtime and compute usage. Default: 100
maxPages	Integer	No	Maximum search result pages to process (1-20). Each page contains approximately 25 games. Default: 5
maxConcurrency	Integer	No	Number of parallel requests (1-10). Higher values = faster scraping but may trigger rate limits. Recommended: 3-5. Default: 3
proxyConfiguration	Object	Recommended	Proxy settings. Recommended to use Apify Proxy (residential or datacenter) to avoid blocking and ensure reliable access. Default: {"useApifyProxy": true}

## Output Data

Each game entry contains comprehensive information structured as JSON. The data includes pricing, reviews, technical specifications, and metadata useful for gaming platforms, price trackers, and market research.

Sample Output

{ "appId": "730", "title": "Counter-Strike 2", "type": "game", "description": "For over two decades, Counter-Strike has offered an elite competitive experience...", "shortDescription": "Competitive tactical FPS", "price": "Free", "originalPrice": null, "discountPercent": null, "isFree": true, "releaseDate": "27 Sep, 2023", "developer": "Valve", "publisher": "Valve", "reviewSummary": "Very Positive", "reviewScore": null, "positiveReviews": "485000", "genres": "Action, Free to Play", "categories": "Multi-player, Steam Achievements, Steam Trading Cards", "tags": "FPS, Shooter, Multiplayer, Competitive, Action", "screenshots": [ "https://cdn.akamai.steamstatic.com/steam/apps/730/ss_1.jpg", "https://cdn.akamai.steamstatic.com/steam/apps/730/ss_2.jpg" ], "headerImage": "https://cdn.akamai.steamstatic.com/steam/apps/730/header.jpg", "website": "https://www.counter-strike.net", "supportInfo": "https://support.steampowered.com", "platforms": "Windows, Mac, Linux", "requiredAge": "0", "dlcCount": null, "achievements": "167", "systemRequirements": { "windows": "OS: Windows 10, Processor: Intel Core i5 2500k...", "mac": "OS: macOS Big Sur 11.0..." }, "url": "https://store.steampowered.com/app/730/", "source": "json-api-detailed", "scrapedAt": "2024-12-18T10:30:00.000Z" }

Output Fields Description

Field	Type	Description
appId	String	Unique Steam application identifier
title	String	Official game title
type	String	Content type (game, dlc, demo, etc.)
description	String	Detailed game description (collected when collectDetails=true)
price	String	Current price with currency symbol or "Free"
originalPrice	String	Original price before discount (if on sale)
discountPercent	String	Discount percentage (e.g., "50%")
releaseDate	String	Game release date
developer	String	Game developer/studio name(s)
publisher	String	Game publisher name(s)
reviewSummary	String	Steam review score (e.g., "Very Positive", "Mixed")
genres	String	Game genres (comma-separated)
tags	String	Popular user-defined tags
screenshots	Array	List of screenshot URLs (up to 5)
platforms	String	Supported platforms (Windows, Mac, Linux)
systemRequirements	Object	Minimum and recommended system specs (when collectDetails=true)
url	String	Direct link to game's Steam Store page
source	String	Data extraction method used (json-api-detailed, html-detailed, etc.)
scrapedAt	String	ISO timestamp when data was extracted

## Configuration Examples

Price Comparison Tool

{ "sortBy": "Price_DESC", "maxResults": 200, "collectDetails": false, "maxConcurrency": 5 }

New Releases Research

{ "sortBy": "Released_DESC", "maxResults": 100, "maxPages": 5, "collectDetails": true }

Budget Gaming Collection

{ "maxPrice": "10", "sortBy": "Reviews_DESC", "maxResults": 150, "collectDetails": true }

Specific Language Support

{ "language": "japanese", "maxResults": 50, "sortBy": "Reviews_DESC", "collectDetails": true }

## Performance & Optimization

Recommended Settings by Use Case

Use Case	Max Results	Details	Concurrency	Est. Runtime
Quick Test	10	false	3	~20 seconds
Price Monitoring	100	false	5	~1 minute
Market Research	200	true	3	~4 minutes
Full Catalog	500	true	5	~8-10 minutes
Comprehensive Analysis	1000	true	3	~15-20 minutes

Best Practices

• Start Small: Begin with 10-20 games to test your configuration and validate output format
• Use Proxies: Enable Apify Proxy (residential recommended) to avoid rate limiting and ensure reliable access
• Optimize Concurrency: Start with 3, increase to 5-7 for faster scraping if no errors occur
• Balance Detail vs Speed: Set collectDetails=false for quick price checks, true for comprehensive data
• Monitor Costs: Each detailed game fetch requires additional requests - estimate ~2-3 requests per game with details
• Schedule Regular Runs: For price tracking, schedule daily/weekly runs to monitor changes over time
• Filter Strategically: Use maxPrice, category, and sortBy to target specific game segments
• Batch Processing: For large datasets (500+ games), split into multiple runs to avoid timeouts

Cost Estimation

• Compute Units: ~0.01-0.02 per game (basic), ~0.03-0.05 per game (with details)
• Proxy Costs: Additional charges apply when using Apify Proxy
• 100 Games (Basic): ~$0.10-0.15 platform cost
• 100 Games (Detailed): ~$0.30-0.40 platform cost
• 500 Games (Detailed): ~$1.50-2.00 platform cost

## Integration & Data Export

Export scraped data in multiple formats for seamless integration with your applications and workflows.

Supported Export Formats

• JSON: Structured data ideal for APIs and web applications
• CSV: Spreadsheet-friendly format for analysis in Excel, Google Sheets
• Excel: Direct export to .xlsx files with formatted columns
• HTML Table: Ready-to-use HTML for embedding in web pages
• RSS Feed: Subscribe to updates for continuous monitoring

API Access

Access your scraped data programmatically using Apify API. Perfect for automated workflows, scheduled data updates, and integration with external systems.

// Fetch dataset items via Apify API const response = await fetch( 'https://api.apify.com/v2/datasets/YOUR_DATASET_ID/items' ); const games = await response.json(); 

## Technical Details

Scraping Strategy

This scraper implements a multi-layered approach for maximum reliability and performance:

1. JSON API Priority: First attempts to fetch search results via Steam's JSON API endpoint for speed and reliability
2. HTML Fallback: Automatically switches to HTML parsing if JSON API is unavailable or returns errors
3. Detailed Extraction: When collectDetails=true, fetches comprehensive game data from individual store pages
4. Concurrent Processing: Processes multiple games simultaneously with configurable concurrency limits
5. Error Handling: Graceful error recovery with detailed logging for troubleshooting

Rate Limiting & Reliability

• Built-in rate limiting to respect Steam Store servers
• Automatic retry mechanism for failed requests
• Proxy rotation support for high-volume scraping
• Timeout management to prevent hanging requests
• Graceful shutdown on approaching time limits

Data Quality

• Validates all extracted data for consistency
• Removes duplicate games based on App ID
• Cleans and normalizes price data
• Extracts structured JSON-LD when available
• Provides source attribution for each data point

## Troubleshooting

Common Issues & Solutions

No results returned

Check if your filters are too restrictive (maxPrice, category, tags). Try removing filters or increasing maxPages parameter. Verify proxy configuration is enabled.

Slow performance

Reduce maxConcurrency to 1-2 if experiencing timeouts. Set collectDetails=false for faster scraping. Use more specific filters to reduce total games processed.

Missing detailed data

Ensure collectDetails=true in your configuration. Some games may have limited data on Steam Store. Check the 'source' field to see which extraction method was used.

Rate limiting errors

Enable Apify Proxy in proxyConfiguration. Reduce maxConcurrency to 2-3. Add delays between runs if scheduling multiple consecutive executions.

Incomplete price data

Some games don't display prices in all regions. Price may show as "Free" for free-to-play games. Check originalPrice field for pre-discount pricing.

## Limitations

• Regional Restrictions: Game availability and prices may vary by region/country
• Dynamic Content: Some game pages use heavy JavaScript - limited data may be extracted in edge cases
• Rate Limits: Steam may throttle requests during peak times - use proxies and appropriate concurrency
• Age-Restricted Content: Some mature content may require authentication - not accessible through this scraper
• Private Games: Unreleased or hidden games won't appear in public search results

## Support & Feedback

Need help or have suggestions? We're here to assist you in getting the most out of this scraper.

• Issues: Report bugs or technical problems through the Apify Console
• Feature Requests: Share ideas for new features or improvements
• Documentation: Refer to this README and Apify documentation for detailed guidance
• Community: Join Apify Discord community for discussions and tips

## Privacy & Legal

This scraper is designed for ethical data collection from publicly available Steam Store pages. Users must comply with:

• Steam's Terms of Service and acceptable use policies
• Local and international data protection regulations (GDPR, CCPA, etc.)
• Apify Terms of Service and fair use guidelines
• Respect for website rate limits and server resources

Disclaimer: This scraper is provided for legitimate business and research purposes. Users are responsible for ensuring their use complies with all applicable laws and regulations. The scraper extracts only publicly available information visible on Steam Store pages.

## Updates & Maintenance

This scraper is actively maintained to ensure compatibility with Steam Store changes and optimal performance.

• Regular Updates: Selectors and scraping logic updated to match Steam Store structure
• Bug Fixes: Prompt resolution of reported issues and errors
• Feature Enhancements: New features added based on user feedback and market needs
• Performance Optimization: Continuous improvements for speed and reliability

Last updated: December 2024 | Version 1.0.0

"max_pages": 4 } ``` ## 📋 Requirements & Limitations ### Data Freshness - Jobs updated in real-time from APEC.fr - Listings typically available for 30-60 days - Salary data available for ~60% of positions ### Geographic Coverage - France-wide coverage - All 101 departments supported - Major cities: Paris, Lyon, Marseille, Toulouse, Nice, Nantes, Bordeaux ### Language Support - Primary language: French - Some international companies list in English - Location names in French format