Commodity Price Scraper 📈 Real-time commodity prices. Track crude oil, gold, silver, natural gas, and more from leading financial websites. Get current prices, trading data, contract details, and technical indicators — all extracted automatically. Currently supports: Investing.com Coming soon: More financial data sources ## ✨ Features - 💰 Real-Time Prices – Current price, change, and change percentage - 📊 Trading Data – Previous close, open, day's range, 52-week range, volume - 📋 Contract Details – Settlement date, tick size, contract size, months, rollover dates - 📈 Technical Indicators – Summary signals (Buy/Sell/Neutral) - ❓ FAQ Extraction – Optionally extract FAQ questions and answers about the commodity - 📦 Multiple Commodities – Scrape multiple commodities in a single run - 🔐 Login Support – Authenticate with email/password for full data access - 🎭 Stealth Mode – Uses Camoufox (stealthy Firefox fork) to avoid detection - 🔄 Multi-Site Architecture – Easily extensible to support more financial sites ## 🚀 Getting Started ### Basic Usage Scrape multiple commodity prices from Investing.com: json { "site": "investing.com", "commodities": ["crude-oil", "gold", "natural-gas"] } Or scrape a single commodity: json { "site": "investing.com", "commodities": ["crude-oil"] } ## 📦 Supported Commodities | Commodity | Slug | Description | |-----------|------|-------------| | Crude Oil WTI | crude-oil | West Texas Intermediate crude | | Brent Oil | brent-oil | North Sea Brent crude | | Natural Gas | natural-gas | Henry Hub natural gas | | Gold | gold | Gold futures | | Silver | silver | Silver futures | | Copper | copper | Copper futures | | Platinum | platinum | Platinum futures | | Palladium | palladium | Palladium futures | | Heating Oil | heating-oil | No. 2 heating oil | | Gasoline RBOB | gasoline-rbob | Reformulated gasoline | > 💡 Tip: You can use any commodity slug from Investing.com. Just use the URL path (e.g., https://www.investing.com/commodities/crude-oil → crude-oil). ## 🔑 Authentication Authentication is optional but provides full data access on Investing.com. json { "site": "investing.com", "commodities": ["gold", "silver"], "skipAuthentication": false, "investingEmail": "your-email@example.com", "investingPassword": "your-password" } ## ⚙️ Input Parameters | Parameter | Type | Required | Description | Default | |-----------|------|----------|-------------|---------| | site | String | ✅ | Website to scrape from | investing.com | | commodities | String[] | ✅ | List of commodity slugs (e.g., ["crude-oil", "gold"]) | ["crude-oil"] | | skipAuthentication | Boolean | ❌ | Skip login (some data may be limited) | true | | investingEmail | String | ❌ | Investing.com email | — | | investingPassword | String | ❌ | Investing.com password | — | | includeFaq | Boolean | ❌ | Extract FAQ questions and answers | false | | maxConcurrency | Integer | ❌ | Max concurrent browsers (1-10) | 1 | | proxyConfiguration | Object | ❌ | Proxy settings (not required, works without proxies) | Disabled | | debugScreenshots | Boolean | ❌ | Save screenshots for debugging | true | ## 📊 Output Example json { "name": "Crude Oil WTI", "symbol": "CL", "url": "https://www.investing.com/commodities/crude-oil", "scrapedAt": "2024-01-15T14:30:00.000Z", "source": "investing.com", "price": "72.45", "currency": "USD", "change": "-0.53", "changePercent": "(-0.73%)", "priceType": "Real-time derived", "prevClose": "72.98", "open": "72.98", "dayRange": "71.85-73.12", "weekRange52": "63.64-95.03", "volume": "218,634", "yearChange1": "-12.21%", "settlementDate": "01/22/2024", "settlementType": "Physical", "month": "02, 2024", "baseSymbol": "CL", "pointValue": "1 = $1000", "tickSize": "0.01", "tickValue": "10", "contractSize": "1,000 Barrels", "months": "FGHJKMNQUVXZ", "lastRolloverDay": "01/18/2024", "technicalSummary": "Sell", "faq": [ { "question": "What Is the Current Price of Crude Oil?", "answer": "The current price of Crude Oil WTI is $72.45 per barrel." }, { "question": "What Is the Daily Trading Range for Crude Oil?", "answer": "Today's trading range for Crude Oil is $71.85 - $73.12." } ] } > 💡 Note: The faq field is only included when includeFaq is set to true. ## 🎯 Use Cases - Trading & Investment – Track real-time commodity prices for trading decisions - Portfolio Monitoring – Monitor commodities in your investment portfolio - Market Research – Analyze price trends and technical indicators - Data Analytics – Build datasets for price analysis and forecasting - Price Alerts – Integrate with workflows to trigger alerts on price changes - Business Intelligence – Track commodity costs for business planning ## ⚙️ How It Works 1. Navigate – Opens the commodity page on Investing.com 2. Handle Cookies – Automatically rejects tracking cookies (OneTrust) 3. Authenticate – Logs in with your email and password 4. Extract – Scrapes all available price and trading data 5. Output – Saves structured data to the dataset ## 🛡️ Proxy Support Proxies are disabled by default and typically not needed. The scraper uses Camoufox (a stealthy Firefox fork) which works well without proxies in most cases. If you experience blocking issues, you can enable proxies: json { "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] } } > 💡 Tip: Try running without proxies first. Only enable them if you encounter blocking. ## 💡 Tips - Check commodity slugs – Use the exact slug from the site's URL - Enable debug screenshots – Helpful for troubleshooting issues - Multiple commodities – Add as many commodities as you need to the commodities array - Speed up scraping – Increase maxConcurrency (up to 10) to run multiple browsers in parallel ## 🔮 Coming Soon - More Sites – Support for additional financial data sources - Historical Data – Extract historical price data - Price Alerts – Built-in price threshold notifications - Forex & Indices – Support for currency pairs and market indices ## 📜 Legal & Ethical Use This Actor is intended for legitimate data collection purposes. Please: - Respect each site's Terms of Service - Use reasonable request rates - Comply with applicable data protection laws - Don't use for market manipulation or illegal activities ## 🤝 Support Having issues? Found a bug? Want a feature? - Open an issue on the Actor's page - Contact the developer through Apify --- Built by nocodeventure • Made with ❤️ for the Apify community