Newsweek Scraper

Overview

This actor scrapes data from Newsweek.com, providing an alternative to a free API. It extracts article details, lists articles by topic/category/author, and performs site searches.

Key Features

• Scrape individual articles – Get title, description, publish/update dates, article body, and other metadata.
• Scrape by topic – Retrieve all articles under a specific topic.
• Scrape by category – Fetch all articles within a category.
• Scrape by author – Collect every article from a specific author.
• Search – Find and retrieve articles matching any keyword.

How to Use

Provide a JSON input configuration. The actor runs on the Apify platform and stores results in a dataset for export.

Input Parameters

The input is a JSON object with the following possible fields:

• search: (Optional, String) Keyword to search for on Newsweek.
• startUrls: (Optional, Array) List of Newsweek URLs. Can be article, topic, category, or author pages.
• endPage: (Optional, Number) The final page number to scrape for paginated lists (like search results or category pages). Default is Infinite.
• maxItems: (Optional, Number) Limit the total number of items scraped.
• proxy: (Required, Object) Proxy configuration. Proxy servers are required. You can use your own or Apify Proxy.
• extendOutputFunction: (Optional, String) A custom JavaScript function to extend the output data.
• customMapFunction: (Optional, String) A custom JavaScript function to transform each scraped item.

Tip: To scrape a specific list (like page 5 of a category), set that page URL as a startUrl and set endPage to 6 to get pages 5 and 6.

Input Example

{
  "startUrls": [
    "https://www.newsweek.com/ron-desantis-abortion-six-week-bill-2024-1794367",
    "https://www.newsweek.com/topic/climate-change",
    "https://www.newsweek.com/authors/jon-jackson",
    "https://www.newsweek.com/education"
  ],
  "maxItems": 10,
  "endPage": 1,
  "search": "donald trump",
  "proxy": {
    "useApifyProxy": true
  }
}

Output

During the run, the actor outputs progress messages. Results are stored in the run's dataset. Each scraped article or list item is a separate object in the dataset. You can access and process this data via the Apify API in Python, PHP, Node.js, or other languages. See the Apify API reference for details.

Performance & Cost

The actor is optimized for speed. With minimal blocking, it can scrape roughly 100 listing items in about 2 minutes, consuming approximately 0.01-0.09 compute units.

Support & Development

This scraper is under active development. For bug reports or feature requests, please create an issue on the GitHub repository.