WordPress Content Extractor

An Apify Actor that extracts structured content from WordPress websites. It combines HTML parsing with the WordPress REST API to pull posts, pages, media, and metadata.

Key Features

• Content Extraction: Pulls blog posts, static pages, custom post types, media assets (with alt text), and SEO metadata (Open Graph, meta descriptions).
• Smart Discovery: Automatically finds content via site navigation, taxonomy pages (categories/tags), and pagination. Uses the /wp-json/wp/v2/ API when available.
• Structured Data: Outputs a consistent JSON schema for all items, including content, author, dates, categories, and tags.
• Flexible Configuration: Choose which content types to extract (posts, pages, media, metadata, comments) and set page limits.
• Broad Compatibility: Works with all WordPress versions, custom themes, Gutenberg blocks, custom post types, and multisite networks.

How to Use

1. Deploy the Actor on the Apify Platform.
2. Configure the input (see below).
3. Run the Actor. It will crawl the site, extract content, and store the results in the dataset.

Input Configuration

Provide these parameters in the Actor input:

Output

The Actor outputs a dataset where each item represents a post or page with this structure:

{
  "url": "https://example.com/post-title",
  "title": "Post Title",
  "content": "Full HTML content",
  "excerpt": "Post summary",
  "metadata": {
    "description": "Meta description",
    "ogTitle": "Open Graph title",
    "ogImage": "Open Graph image URL"
  },
  "media": [
    { "src": "image-url.jpg", "alt": "Image alt text", "type": "image" }
  ],
  "comments": [
    { "author": "Commenter Name", "content": "Comment text", "date": "Comment date" }
  ],
  "publishedDate": "2024-01-01T00:00:00Z",
  "author": "Post Author",
  "categories": ["Category 1"],
  "tags": ["tag1", "tag2"],
  "type": "post"
}

Technical Details

• Built With: Apify SDK, Axios, Cheerio, Node.js.
• Performance: Uses concurrent processing, respectful crawling delays, and robust error handling.