Streem Article Scraper – Webhook Support A fast and reliable tool for extracting full-text content and metadata from Streem article links. This scraper is designed to support media teams, analysts, and researchers who need consistent, structured article data—automatically delivered via webhook. --- ## 🚀 Features - ✅ Multi-URL Support – Process one or many Streem article links in a single run - ⚙️ Concurrent Scraping – Handle multiple articles in parallel for faster results - 🔄 Webhook Integration – Automatically send results to your endpoint after completion - 🧹 Clean Content Output – Removes excess HTML/CSS, returns plain, readable text - 🔒 Optional Webhook Auth – Add a token to secure your webhook endpoint - 📄 Consistent Output – Each result includes success/error status, article data, and input URL --- ## 🔧 How It Works ### 1. Provide Input You can supply either: - urls: A list of Streem article links - url: A single Streem article link Additional options: - concurrency: Number of articles to process at the same time (default: 3) - webhook_url: (Optional) Where the data will be sent after scraping - webhook_method: POST or PUT (default: POST) - webhook_token: (Optional) Used as a bearer token to secure your webhook --- ### 2. Processing For each URL, the scraper will: - Fetch the page - Extract the clean article text and metadata - Package the data into a consistent JSON result - Flag errors while keeping the structure intact --- ### 3. Delivery When a webhook_url is provided, all results are sent automatically to your system—no manual downloading or polling required. --- ## 📥 Input Example ```json { "urls": [ "https://links.streem.com.au/dubbo-photo-news-20250702-XVg7X1h8q7U4Hnh7EbRnI8cEjRRve", "https://links.streem.com.au/the-kimberley-echo-20250702-2716altLN5CQHRh" ], "concurrency": 3, "webhook_url": "https://your-api.com/webhook/streem-articles", "webhook_token": "my-secret-token-123", "webhook_method": "POST" }