PromptBase Scraper

An Apify actor that scrapes AI prompts and creator data from PromptBase.com. It extracts detailed listings, pricing, and statistics, with configurable filters for categories and pricing models.

Key Features

• Scrapes comprehensive prompt data: Titles, full content, images, tags, and associated AI models (like Midjourney or ChatGPT).
• Extracts pricing and creator info: Price, currency, pricing model (free/paid), plus creator names, profiles, bios, and social links.
• Collects usage statistics: Prompt views, downloads, favorites, and ratings.
• Offers filtering: Limit results by category, pricing model (free, paid, all), and sort order (popular, newest, trending, price).
• Developer utilities: Includes HTML debugging output to the key-value store (as debug_promptbase_html) for selector analysis during development.

Input Configuration

Configure the actor using these input parameters:

Output Data

The actor outputs an array of prompt objects. Each object includes:

• Basic Info: title, detail_url, image_url (thumbnail).
• Pricing: price, currency, pricing_model.
• Creator: creator_name, creator_url.
• Categorization: category, tags.
• Statistics: rating.
• Metadata: scraped_at (timestamp), source ("promptbase.com").

If scrapeDetails is true, additional fields are added:
* full_content: The complete prompt text and instructions.
* creator_profile: An object with the creator's name, bio, avatar_url, and ai_models.
* prompt_images: An array of additional images related to the prompt (max 5).

How to Use

Basic Run

Scrape 50 prompts with full details:

{
  "maxPrompts": 50,
  "scrapeDetails": true
}

Filtered Scrapes

Scrape 200 paid writing/marketing prompts, sorted by newest:

{
  "maxPrompts": 200,
  "scrapeDetails": true,
  "categories": ["writing", "marketing"],
  "pricingModel": "paid",
  "sortBy": "newest"
}

Scrape 100 popular free prompts (without detailed pages):

{
  "maxPrompts": 100,
  "scrapeDetails": false,
  "pricingModel": "free",
  "sortBy": "popular"
}

Development & Installation

For local development:

1. Install dependencies:
   bash pip install -r requirements.txt
2. Install the Playwright browser:
   bash playwright install chromium
3. Run the scraper:
   bash python -m src

To build and run with Docker:

docker build -t promptbase-scraper .
docker run -e APIFY_TOKEN=your_token promptbase-scraper

The scraper uses Playwright for browser automation, includes error handling with retry logic, and respects the website's robots.txt rules.