Reddit Scraper - Cost-Effective ## 🤖 What does Reddit Scraper do? Reddit Scraper enables you to extract structured data from www.reddit.com, including posts, comments, and media content. It allows targeted data scraping based on specific subreddits, search queries, and sorting preferences. ## 💡 Why scrape Reddit? Reddit is one of the largest discussion platforms on the internet, with millions of active users and a wide variety of topics. Scraping Reddit data can be valuable for: - 📊 Analyzing community discussions and trends. - 🔍 Monitoring keyword mentions across subreddits. - 📈 Researching user behavior and sentiment analysis. - 📰 Gathering content for aggregation platforms or analysis tools. ## 🚀 How to use Reddit Scraper 1. Select Action: Choose between search or explore modes. 2. Set Fetch Preferences: Decide whether to fetch posts, comments, or both. 3. Define Action Parameters: Provide subreddit names, queries, and filters. 4. Adjust Proxy Settings: Configure proxies for large-scale scraping. 5. Run the Scraper: Start the process and monitor progress. ### 🔍 Search vs Explore Modes - Search Mode: Uses Reddit's search bar feature to find posts and comments based on search terms and filters. - Explore Mode: Focuses on subreddit and topic listings. It already includes post data by default, even if fetchPosts is disabled. ## 📝 Input Parameters | Parameter | Type | Description | | -------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | action | string | Action type (search or explore) | | maxItems | integer | Maximum number of posts to scrape | | fetchMode | string | Specify the type of data to fetch
- list (summary data from listing)
- post (full post data)
- comments (full post data + comments) | | searchSub | string | Subreddit to search within | | searchQuery | string | Search term for posts or comments | | searchType | string | Type of content (posts, comments, media) | | searchSort | string | Sorting criteria (relevance, hot, top, new, comments) | | searchTime | string | Time filter (hour, day, week, month, year) | | exploreSub | string | Subreddit to explore | | exploreTopic | string | Topic to explore | | exploreSort | string | Sorting criteria (hot, new, top, rising) | | exploreTime | string | Time filter (only used for top sorting) (hour, day, week, month, year) | | proxy | struct | Proxy configuration | ### Example Input json { "action": "search", "maxItems": 50, "fetchMode": "comments", "searchSub": "golang", "searchQuery": "web scraping", "searchSort": "top", "searchTime": "month", "proxy": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] } } ## 📊 Output The scraper outputs structured JSON data containing the most relevant fields from Reddit posts and comments. ### Example Output json [ { "id": "t3_1hlkdi4", "created_at": "2024-12-24T19:13:00.649Z", "url": "https://www.reddit.com/r/golang/comments/1hlkdi4/go_concurrency_problems_intermediate_level/", "title": "Go Concurrency Problems intermediate level", "author": "Confident-Option-341", "author_id": "t2_9m4khn4mj", "author_avatar": "https://www.redditstatic.com/avatars/defaults/v2/avatar_default_3.png", "subreddit": "r/golang", "subreddit_id": "t5_2rc7j", "score": 7, "type": "text", "content": "Hi ! I have recently started interviewing for golang and it seems that the expectation has gone up for the concurrent aspects in golang. Are there any resources/interview problems available that would give one enough insight on the concurrency problems to practice? Would be grateful for any help! Thanks in advance!", "content_link": "", "comment_count": 2, "language": "en", "comments": [ { "id": "t1_m3nlfno", "parent_id": "", "url": "https://www.reddit.com/r/golang/comments/1hlkdi4/comment/m3nlfno/", "author": "jrandom_42", "author_id": "t2_9qnz2", "author_avatar": "https://styles.redditmedia.com/t5_di19t/styles/profileIcon_snoo562a14d7-08eb-4fdf-825c-9af5475305ba-headshot.png", "score": 5, "award_count": 0, "type": "text", "content": "Just out of interest OP, what questions have you faced in interviews in relation to Go concurrency?" }, { "id": "t1_m3mwev9", "parent_id": "", "url": "https://www.reddit.com/r/golang/comments/1hlkdi4/comment/m3mwev9/", "author": "yourAsad", "author_id": "t2_8vub84jf", "author_avatar": "https://styles.redditmedia.com/t5_3er6gh/styles/profileIcon_wxj52p5lbs2b1.jpg", "score": 7, "award_count": 0, "type": "text", "content": "Yes, I faced the same issue when I gave an interview at a tech company. To improve, I started reading books, watching YouTube videos, and taking a Udemy course on Go concurrency. One of the books I read was Concurrency in Go: Tools and Techniques for Developers, which is recommended by the Go community itself" } ] } ] ## 💳 How much will it cost to scrape reddit.com? Apify provides you with $5 free usage credits every month on the Apify Free plan. You can try and test Reddit-Scraper for free with the Free plan for a limited time. However, if you need to get more data regularly from reddit.com, you should get an Apify subscription. We recommend our $49/month Personal plan - this plan covers the costs of Reddit-Scraper and numerous executions. ## 🌍 Proxy Configuration Reddit Scraper supports proxy configurations to avoid IP blocking and ensure uninterrupted scraping. Use Apify Residential Proxies for reliable results. ## 🛠️ Troubleshooting - Double-check proxy settings. - Ensure subreddit and query parameters are correctly formatted. - Monitor logs for error messages. - Open an issue if you encounter any problems. Happy Scraping! 🚀