LD+JSON Schema Scraper

Extracts structured data (LD+JSON) from web pages for automation, SEO analysis, or data integration.

Overview

This actor crawls specified web pages and parses all LD+JSON script tags (application/ld+json). It outputs the raw schema data in a structured JSON format, making it usable for analysis, monitoring, or feeding into other systems.

Key Features

• Extracts All Schema Types: Captures any LD+JSON data, including but not limited to Article, Product, Organization, BreadcrumbList, and FAQPage.
• Handles Dynamic Content: Uses a headless browser (Puppeteer) to execute JavaScript, ensuring schemas rendered client-side are captured.
• Configurable Crawling: Set maximum crawl depth and pages to control the scope of your extraction.
• Proxy Support: Built-in proxy rotation to help avoid blocks during larger scraping jobs.
• Open Source: The code is publicly available for inspection and modification.

How to Use

Run the actor on the Apify platform. You can start it via the Apify Console, using the Apify API, or integrate it into workflows with other Apify actors.

Basic Input Configuration:
Configure the actor run by providing a JSON object with the following key parameters:

{
  "startUrls": [
    { "url": "https://example.com/page-with-schema" }
  ],
  "maxDepth": 1,
  "maxPages": 10
}

Input/Output

Input (Run Configuration):
* startUrls (Required): An array of one or more URLs to start scraping from.
* maxDepth: How many links deep to follow from the start URLs (0 = only start URLs). Default is 1.
* maxPages: Maximum number of pages to scrape. Default is 1000.

Output (Dataset Items):
Each item in the output dataset represents a scraped page and contains:
* url: The source page URL.
* schemas: An array of objects, each containing the parsed JSON-LD data found on that page.
* metadata: Information like the HTTP status code and request/response details.

Example Output Item:

{
  "url": "https://example.com/product",
  "schemas": [
    {
      "@context": "https://schema.org",
      "@type": "Product",
      "name": "Example Widget",
      "description": "A great widget."
    }
  ]
}

Find the actor and its details here: https://apify.com/your-actor-page?fpr=python_automation