Playwrite-Code: Single-Page Web Scraper Template

Overview

This is an Apify Actor template for scraping data from a single web page using Node.js. It takes a URL as input, fetches the page HTML, parses it, and outputs structured data to a dataset. The template is built with Axios for HTTP requests and Cheerio for HTML parsing, providing a straightforward foundation you can modify to extract any data you need.

Key Features

• Apify SDK – Toolkit for building and running Actors.
• Input Schema – Validates input configuration, primarily expecting a url field.
• Dataset Storage – Saves scraped data in a structured format where each item has consistent attributes.
• Axios Client – Handles HTTP requests to fetch page HTML.
• Cheerio – Parses and traverses HTML/XML to extract data.
• Open Source – Full control to edit and extend the scraping logic.

How to Use

Basic Operation

1. Provide a target URL via the Actor's input.
2. The Actor uses axios.get(url) to fetch the HTML.
3. Cheerio loads the HTML (cheerio.load(response.data)) for parsing.
4. The default code extracts all heading elements (h1 through h6).
5. Extracted data is saved using Actor.pushData(headings) to an Apify dataset.

You can modify the parsing logic in step 4 to target different HTML elements. For example, the default selector is:

$("h1, h2, h3, h4, h5, h6").each((_i, element) => {...});

Running the Actor

You can run the Actor directly in the Apify Console. After building the Actor, provide the input (URL) and start a run. Results will be available in the dataset.

Local Development

To develop locally, pull the Actor using the Apify CLI:

1. Install the CLI:
   ```bash
   # Using Homebrew
   brew install apify-cli

# Using npm
npm -g install apify-cli
```

1. Pull the Actor by its unique ID or name (found in the Apify Console):
   bash apify pull <ActorId>

Input/Output

Input:
The Actor expects an input object containing:
- url (string, required): The web page URL to scrape.

Output:
The Actor stores results in an Apify dataset. By default, each dataset item represents a heading found on the page, containing its tag name and text content. The output structure can be changed by editing the parsing code.

Resources

• Web scraping in Node.js with Axios and Cheerio
• Web scraping with Cheerio in 2023
• Video tutorial on building a scraper using CheerioCrawler
• Written tutorial on building a scraper
• Apify SDK for JavaScript documentation
• Apify Platform documentation
• Join the developer community on Discord