Substack Newsletter Scraper 📰 Extract complete data from Substack newsletters including posts, authors, engagement metrics, and full article text. ## What does this Actor do? 🎯 This actor scrapes Substack newsletters and extracts 13 data points per post: ### Content Data - Post headline & subheading - Full article text (or preview for paid posts) - Post URL ### Author Information - Author name - Author profile URL ### Publishing Data - Publication date - Free vs Paid status ### Engagement Metrics - Number of likes ❤️ - Number of comments 💬 - Number of restacks 🔄 ## Perfect for 💡 - Content Researchers - Analyze trends across newsletters - Competitive Analysis - Track what competitors publish - Data Scientists - Build training datasets - Writers - Research popular topics - AI Developers - Collect training data ## Input json { "substackUrls": [ "https://tedhope.substack.com", "https://theankler.com" ], "scrapingMethod": "sitemap", "maxPostsPerSubstack": 100, "batchSize": 20 } | Field | Type | Required | Description | |-------|------|----------|-------------| | substackUrls | Array | Yes | List of Substack URLs | | scrapingMethod | String | No | "sitemap" or "archive" | | maxPostsPerSubstack | Number | No | Limit posts (0 = unlimited) | | batchSize | Number | No | Substacks per batch | ## Output json { "substack_url": "https://example.substack.com", "post_url": "https://example.substack.com/p/post-title", "headline": "Amazing Post Title", "subheading": "Subtitle here", "author_name": "John Doe", "author_url": "https://substack.com/@johndoe", "date": "December 10, 2024", "free_or_paid": "Free", "likes": 156, "comments": 23, "restacks": 12, "article_text": "Full article content...", "content_type": "full" } ## Features ✨ - Two scraping methods - Sitemap (fast) or Archive (fallback) - Complete metadata - 13 fields per post - Engagement metrics - Likes, comments, restacks - Smart extraction - Handles paywalled content - Structured output - Clean JSON/CSV format ## Pricing 💰 Based on Compute Units (CUs): | Scale | Approximate Cost | Time | |-------|-----------------|------| | 10 Substacks | $0.01-0.05 | 2-5 min | | 100 Substacks | $0.50-1.00 | 30-60 min | | 1,000 Substacks | $5-10 | 5-10 hours | Start with Apify's free tier - includes $5 monthly credit! ## Tips 💪 1. Test first - Try 2-3 Substacks initially 2. Use sitemap - Faster and more reliable 3. Set limits - Use maxPostsPerSubstack for large newsletters 4. Schedule runs - Set up weekly/monthly automation ## FAQ ❓ Q: Can I scrape paywalled content? A: You'll get previews for paid posts, not full content. Q: How long does it take? A: ~10-30 seconds per Substack with 100 posts. Q: What if sitemap doesn't work? A: Use "archive" method as fallback. ## Support 📧 - 📖 Apify Documentation - 💬 Contact Support ## About 👨‍💻 Built by MUSTAPHA LIAICHI - Automation & Web Scraping Specialist --- Happy Scraping! 🚀