Google News Scraper ## What data does Google News Scraper collect? Google News Scraper allows you to extract news metadata such as title, link, source, publication datetime and image. ## Query-based search The Actor provides a simple interface where you can define your search query, language, region, and date range. Unlike manually browsing Google News, which limits you to approximately 100 results per search, Google News Scraper allows you to retrieve significantly more results without restrictions. To bypass the usual result limits, the scraper can automatically filter news articles by date. If you set maxItems to a value greater than 100, the scraper will fetch news articles day by day until it reaches the specified limit. For more precise control over the date range, you can use the dateFrom and dateTo fields in the YYYY-MM-DD format. Alternatively, you can specify an open-ended date range using the openEndedDateRange field with values such as: - 1h – retrieves news from the past hour - 7d – retrieves news from the past week - 1y – retrieves news from the past year You can combine numbers with the h, d, or y shortcuts to customize your date range as needed. ### Advanced search filters You can also use advanced search operators in your queries, such as intitle, inurl, site, exclude operator -, exact match with double-quotes "", AND, OR and more. Example queries with advanced operators: | Query | Explained | | ------------------------------------------------------ | ------------------------------------------------------------ | | intitle:"AI" AND site:bbc.com | Finds articles with "AI" in the title from BBC. | | site:reuters.com "stock market" -crypto | Finds stock market articles on Reuters, excluding crypto-related ones. | | "Samsung Galaxy S25" AND (review OR comparison) | Searches for reviews or comparisons of Samsung Galaxy S25. | | site:nytimes.com intitle:"election" after:2025-01-01 | Retrieves recent NY Times articles with "election" keyword in the title. | | inurl:blog OR inurl:news "climate change" | Searches for climate change mentions in blog or news URLs. | You may already know some of the operators from Google search or Apify's Google Search Scraper. For more information, see Google Guide on Search Operators. ## Topic-based search Google News Scraper also supports topic-based search. You can search for news from one of the topics predefined in topics input field: - WORLD 🌎 - NATION 🚩 - BUSINESS 🪙 - TECHNOLOGY 💻 - ENTERTAINMENT 🎸 - SPORTS 🏒 - SCIENCE 🧪 - HEALTH 🧑‍⚕️ ### Hashed Topics The scraper also supports hashed topics, allowing you to extract news based on specific topic IDs from Google News. Simply copy the topic ID from the Google News URL and paste it into the topicsHashed input field. For example, if you want to find news about your home country but published by foreign sources or in different languages, you can: 1. Click on your country in Google News. 2. Copy the topic ID from the URL. 3. Paste it into the topicsHashed array. 4. Set the language field to your preferred language and region (e.g., DE:de). This way, you can for example retrieve the latest news from the US but published by German sources like zdf.de or spiegel.de. Similarly, you can search for news about a non-English-speaking country and get results from international sources such as reuters.com, bbc.co.uk, independent.co.uk, or variety.com. ### Topic Sections The scraper also allows you to refine your search by targeting specific sections within broader topics. For example, you can extract news from the NHL section under Sports or Artificial Intelligence within the Technology category. Imagine you need to scrape AI-related news from the section Technology > Artificial Intelligence, which is available at the following URL: https://news.google.com/topics/CAAqJggKIiBDQkFTRWdvSUwyMHZNRGRqTVhZU0FtVnVHZ0pWVXlnQVAB/sections/CAQiQ0NCQVNMQW9JTDIwdk1EZGpNWFlTQW1WdUdnSlZVeUlOQ0FRYUNRb0hMMjB2TUcxcmVpb0pFZ2N2YlM4d2JXdDZLQUEqKggAKiYICiIgQ0JBU0Vnb0lMMjB2TURkak1YWVNBbVZ1R2dKVlV5Z0FQAVAB?hl=en-US&gl=US&ceid=US%3Aen Simply copy the {TOPIC_ID}/section/{SECTION_ID} part of the URL and paste it into the topicsHashed array to target that specific section: json { "topicsHashed": [ "CAAqJggKIiBDQkFTRWdvSUwyMHZNRGRqTVhZU0FtVnVHZ0pWVXlnQVAB/sections/CAQiQ0NCQVNMQW9JTDIwdk1EZGpNWFlTQW1WdUdnSlZVeUlOQ0FRYUNRb0hMMjB2TUcxcmVpb0pFZ2N2YlM4d2JXdDZLQUEqKggAKiYICiIgQ0JBU0Vnb0lMMjB2TURkak1YWVNBbVZ1R2dKVlV5Z0FQAVAB" ] } ## Example output To obtain a detailed output with decoded article links and image URLs, set the fetchArticleDetails input field to true. Extracted news are then stored in a following format: json [ { "title": "A win at last: Big blow to AI world in training data copyright scrap", "link": "https://www.theregister.com/2025/02/12/thomson_reuters_wins_ai_copyright/", "guid": "CBMifkFVX3lxTFByV0JDX09JaDR4bmNfX3AzWVFRbjMwdWNKcmtWQ255MEVGa2lCTHBpTE5JbFFWMG5nVFE1T2M0alVfY0N2N3ZFdFJDV3FrUjhVeDZ2dkQxQUNwb1FPcm9kMV9KYXk0ZUNnN1BDNDlZUXZtY3ktemgwWkpNU3Nxd9IBgwFBVV95cUxPc2tobXlKUmcwR0wyR0podklDS2MzMDhpQTd6N2NVbnRqYndPSVVmckV1Tk82TkNDck1JbmFuUU0yUjByTDE4VFRGVC1jTUc2M0VPOE1wX1RUZWpfOG9DamVmX21lWERuU0JGcHJyVDZQdEFDTHNNZjFQV1BQNkN0SjhaSQ", "source": "The Register", "sourceUrl": "https://www.theregister.com", "publishedAt": "2025-02-12T01:45:00.000Z", "loadedUrl": "https://www.theregister.com/2025/02/12/thomson_reuters_wins_ai_copyright/", "rssLink": "https://news.google.com/rss/articles/CBMifkFVX3lxTFByV0JDX09JaDR4bmNfX3AzWVFRbjMwdWNKcmtWQ255MEVGa2lCTHBpTE5JbFFWMG5nVFE1T2M0alVfY0N2N3ZFdFJDV3FrUjhVeDZ2dkQxQUNwb1FPcm9kMV9KYXk0ZUNnN1BDNDlZUXZtY3ktemgwWkpNU3Nxd9IBgwFBVV95cUxPc2tobXlKUmcwR0wyR0podklDS2MzMDhpQTd6N2NVbnRqYndPSVVmckV1Tk82TkNDck1JbmFuUU0yUjByTDE4VFRGVC1jTUc2M0VPOE1wX1RUZWpfOG9DamVmX21lWERuU0JGcHJyVDZQdEFDTHNNZjFQV1BQNkN0SjhaSQ?oc=5", "image": "https://regmedia.co.uk/2021/08/02/shutterstock_robot_justice.jpg" }, { "title": "Web Scraping Optimization: Tips for Faster, Smarter Scrapers", "link": "https://hackernoon.com/web-scraping-optimization-tips-for-faster-smarter-scrapers", "guid": "CBMiiAFBVV95cUxQUXh5WVZ2RkNpNG9ndjF6V3hMRHBRTGRSVnNkelpwZDY2TWJzejBSMGZrRC1rSm5DZ1BxanpoeFFGdDRjWGpZR0tOUG9FY0kyeWFXOE9MSzBobTg1ajRiZzVhSWhtbm5nSVNJVWExSDBSaEFjUUJkT1JRRDJHSDBrMU9jU2ZZN3RN", "source": "hackernoon.com", "sourceUrl": "https://hackernoon.com", "publishedAt": "2024-11-15T08:00:00.000Z", "loadedUrl": "https://hackernoon.com/web-scraping-optimization-tips-for-faster-smarter-scrapers", "rssLink": "https://news.google.com/rss/articles/CBMiiAFBVV95cUxQUXh5WVZ2RkNpNG9ndjF6V3hMRHBRTGRSVnNkelpwZDY2TWJzejBSMGZrRC1rSm5DZ1BxanpoeFFGdDRjWGpZR0tOUG9FY0kyeWFXOE9MSzBobTg1ajRiZzVhSWhtbm5nSVNJVWExSDBSaEFjUUJkT1JRRDJHSDBrMU9jU2ZZN3RN?oc=5", "image": "https://hackernoon.imgix.net/images/0FC9YtxD4fbD3T7mPipOt4HSxY42-7y034nb.png" } ] > NOTE: Image URLs are retrieved directly from target article websites and may differ from those displayed on the Google News website. The scraper relies on the Google News RSS API, which does not include image links. However, using the RSS API instead of web browsing allows the scraper to efficiently gather all data through simple HTTP requests, making the process faster and more cost-effective. If you set fetchArticleDetails to false, the scraper will run significantly faster and at a lower cost, but the output will only include URLs in the RSS feed format, such as: https://news.google.com/rss/articles/CBMiiAFBVV95cUxQUXh5WVZ2RkNpNG9ndjF6V3hMRHBRTGRSVnNkelpwZDY2TWJzejBSMGZrRC1rSm5DZ1BxanpoeFFGdDRjWGpZR0tOUG9FY0kyeWFXOE9MSzBobTg1ajRiZzVhSWhtbm5nSVNJVWExSDBSaEFjUUJkT1JRRDJHSDBrMU9jU2ZZN3RN?oc=5 The output will be simplified to include RSS links only and exclude preview images: json [ { "title": "How I used Scrapy and Scrapy Playwright to take screenshot", "link": "https://news.google.com/rss/articles/CBMigAFBVV95cUxQRDJDNW1ONklDTlhJN3lsWlIwdTEzV050d284bnZ4YWo2TnQ3MU1XcUZkdHNZTTMxSTZ4NjkwRnh0UFloa0ZsZ0d5Qmx0NjRFd25qMXdBTmN6TVBmcHdKdXhGWnRCSDVMdjVJZlJkTlRTMjZSZXZwdGNYM28xbE9uTA?oc=5", "guid": "CBMigAFBVV95cUxQRDJDNW1ONklDTlhJN3lsWlIwdTEzV050d284bnZ4YWo2TnQ3MU1XcUZkdHNZTTMxSTZ4NjkwRnh0UFloa0ZsZ0d5Qmx0NjRFd25qMXdBTmN6TVBmcHdKdXhGWnRCSDVMdjVJZlJkTlRTMjZSZXZwdGNYM28xbE9uTA", "source": "substack.com", "sourceUrl": "https://substack.com", "publishedAt": "2024-06-09T07:00:00.000Z" }, { "title": "Beyond the Buzz: Making AI Work for Large-Scale Web Data Collection", "link": "https://news.google.com/rss/articles/CBMisgFBVV95cUxPVXVJeGNPTWI0bXVmOWdSc1B5MVpDRWVnX1R3QV8wdzRmb2VNU1hneU9wNEZrMGdOT2Fqc1Rremg1R0lmM1NBdjZJa3llOXc2bDVzN3RwNVppZnJ3Q1FTS1NsRUVxOTlwWl9QNlJYMUpPYUxKeFVRSWpCYXNaOVU2aFZ0RWNzd200cmRRQlJoUE8tcnU2bXNIQkZNOXUxRDBzTXAxSEMtMG5lLTRsdlVWZFRB?oc=5", "guid": "CBMisgFBVV95cUxPVXVJeGNPTWI0bXVmOWdSc1B5MVpDRWVnX1R3QV8wdzRmb2VNU1hneU9wNEZrMGdOT2Fqc1Rremg1R0lmM1NBdjZJa3llOXc2bDVzN3RwNVppZnJ3Q1FTS1NsRUVxOTlwWl9QNlJYMUpPYUxKeFVRSWpCYXNaOVU2aFZ0RWNzd200cmRRQlJoUE8tcnU2bXNIQkZNOXUxRDBzTXAxSEMtMG5lLTRsdlVWZFRB", "source": "GeekWire", "sourceUrl": "https://www.geekwire.com", "publishedAt": "2024-11-26T08:00:00.000Z" } ]