💎ESG Scraper: Sustainability Reports & PDF Disclosures

# 🌱 ESG Scraper: Sustainability Reports, Articles & PDF Disclosures Extractor Enterprise-grade ESG web scraper that automatically extracts sustainability articles, corporate reports, climate news, and PDF disclosures — clean, structured, and ready for investors, compliance teams, or AI training. High-quality ESG & Sustainability Web Scraper for Investors, Analysts, and AI Teams Automatically collects ESG articles, sustainability reports, corporate disclosures, climate news, and PDF reports from any website — clean, structured, ready for analysis or AI. Built for: - Sustainable investors & analysts - Compliance and risk teams - AI/ML engineers building ESG models - Researchers and NGOs tracking climate & governance trends ✅ Smart ESG keyword filtering ✅ Full clean article text extraction ✅ PDF sustainability reports parsing ✅ Rich metadata (date, author, description) ✅ ESG-relevant images and related links ✅ AI-ready dataset splitting (overview / full-text / images) 👉 Runs on Apify • No code required • Pay only for compute used --- ## 🚀 Why This Scraper ✔ Purpose-Built for ESG Data Intelligently filters pages using custom ESG keywords (climate, emissions, governance, CSR, net zero, etc.). ✔ Excellent PDF Handling Full text extraction from sustainability and ESG reports (PDF) with metadata where available. ✔ Clean & Noise-Free Output Removes ads, navigation, scripts — only meaningful content remains. ✔ Rich Structured Data Title, publication date, author, description, ESG keywords, internal links, relevant images. ✔ AI & ML Ready Optional splitting into specialized datasets for RAG, LLM fine-tuning, or training. ✔ Fast & Efficient Powered by Crawlee + Cheerio — excellent for static and content-heavy sites (news, corporate pages, PDFs). For heavily JavaScript-rendered sites, results may vary. ✔ Safe & Controlled Crawling Automatic domain restriction, depth limit (max 3 levels), request limits. --- ## 💼 Use Cases - ESG portfolio screening and risk monitoring - Training ESG-focused LLMs or RAG systems - Regulatory compliance and disclosure tracking - Competitive intelligence on corporate sustainability - Academic research on climate and governance trends --- ## 📊 Supported Sources - ESG news sections (Reuters, Bloomberg, FT, Guardian, etc.) - Corporate sustainability / ESG pages - Annual sustainability reports (PDF) - Climate, emissions, governance disclosures --- ## ⚙️ How It Works 1. Provide start URLs (news sections, corporate pages, PDF links) 2. Set custom ESG keywords and limits 3. Run the Actor 4. Download clean, structured ESG datasets --- ## 🧩 Input Configuration ### Example JSON Input json { "startUrls": [ { "url": "https://www.reuters.com/sustainability/" }, { "url": "https://www.weforum.org/stories/technological-innovation/" } ], "allowedDomains": ["reuters.com"], "useApifyProxy": false, "maxRequestsPerCrawl": 500, "esgKeywords": [ "ESG", "sustainability", "climate", "emissions", "net zero", "governance" ], "extractContent": true, "extractMetadata": true, "followLinks": true, "useSeparateDatasets": true, "cleanDefaultDataset": true, "proxyUrls": [ { "url": "http://user:pass@host:port" } ] } Key Options - startUrls — one or more starting pages or direct PDF links (required) - allowedDomains — restrict crawling to specific domains. If empty, automatically limited to domains from startUrls - maxRequestsPerCrawl — control cost and crawl size - esgKeywords — custom list for relevance filtering (default includes common ESG terms) - extractContent / extractMetadata — toggle full text or metadata extraction - followLinks — enable internal crawling (limited to depth 3 for safety) - useSeparateDatasets — recommended for large runs and AI workflows - cleanDefaultDataset — clear previous run data --- ## 📂 Output Datasets When useSeparateDatasets: true (recommended): - esg-overview (primary) — lightweight metadata for fast analysis - esg-full-content — long articles (>5000 characters) - esg-images — ESG-relevant images with context - Default dataset — minimal preview records (for Apify UI visibility) When useSeparateDatasets: false - Single dataset with full detailed records --- ### Example Output Record (Full Mode) json { "url": "https://www.reuters.com/sustainability/example", "title": "Companies strengthen climate commitments", "scrapedAt": "2025-12-15T10:30:45Z", "publishedDate": "2025-12-10", "author": "Jane Doe", "description": "Major firms enhance ESG targets...", "content": "Full clean article text...\n\nParagraphs preserved...", "esgKeywords": ["climate", "emissions", "sustainability"], "relatedLinks": [ { "url": "https://www.reuters.com/sustainability/esg-guide", "text": "ESG Explained" } ], "images": [ { "url": "https://reuters.com/chart-netzero.jpg", "alt": "Net zero emissions progress" } ] } ### PDF Example json { "url": "https://company.com/sustainability-2024.pdf", "title": "Annual Sustainability Report 2024", "content": "Full extracted report text...", "esgKeywords": ["sustainability", "carbon", "governance"], "type": "PDF", "author": "Corporate Sustainability Team", "publishedDate": "2024-03-15" } --- ## 🏁 Getting Started 1. Click “Try for free” on Apify 2. Paste ESG/sustainability URLs or direct PDF links 3. Customize keywords and limits 4. Run and download your dataset --- ## 📧 Support - Email: kidaxxb@gmail.com - Response within 24 hours - Issues: Use Apify Issues tab Tags: ESG, sustainability, web scraping, PDF extraction, climate data, corporate governance, RAG, LLM training, sustainable investing, compliance monitoring Built with ❤️ on Apify