X (Twitter) Scraper – Low-Cost Tweet Extraction

Overview

This Apify Actor scrapes tweets from X.com (Twitter) for search terms, user timelines, or a combination of both. It's built to be a low-cost and reliable alternative to the official API, suitable for scraping large volumes of tweets on a budget.

Common use cases include social listening, brand monitoring, trend analysis, competitor research, and academic data collection.

Key Features

• Flexible Scraping Modes: Scrape by search query, by username (user timeline), or combine both to search within a specific user's tweets.
• Granular Filters: Narrow results using a wide range of optional filters:
  • Content Type: Include or exclude replies, retweets, and quotes.
  • User & Post Attributes: Filter for verified users, posts containing links, or news content.
  • Media: Filter for posts containing any media, or specifically images, videos, native videos, or pro videos.
  • Safety: Filter for "safe" (non-NSFW) content.
  • Date Range: Use minDate and maxDate for temporal filtering.
  • Location: Bias search results using the near parameter (e.g., "New York").
• Cost-Effective: Optimized for large-scale data extraction without high costs.
• Structured Output: Results are provided in a consistent JSON schema, downloadable as JSON, CSV, or Excel.

How to Use

1. Configure Input: Provide at least a query or a username in the Actor's input. Set the maxItems to control the volume.
2. Apply Filters (Optional): Add any relevant filters from the list above to refine your dataset.
3. Run and Export: Execute the Actor. Once finished, download the resulting dataset in your preferred format.

Basic Input Examples

Global Search:

{
  "maxItems": 50,
  "query": "pizza",
  "images": "only",
  "retweets": "exclude",
  "minDate": "2024-05-01",
  "maxDate": "2024-08-31"
}

User Timeline:

{
  "maxItems": 100,
  "username": "elonmusk",
  "replies": "exclude",
  "retweets": "exclude"
}

Search Within a User's Tweets:

{
  "maxItems": 100,
  "username": "elonmusk",
  "query": "pizza",
  "images": "only"
}

Input / Output

Essential Input Parameters

* You must provide at least one of query or username.

Output Schema

Each tweet in the dataset includes the following core fields:

{
  "id": "1819469673169408242",
  "permalink": "https://x.com/elonmusk/status/1819469673169408242",
  "userUrl": "https://x.com/elonmusk",
  "username": "elonmusk",
  "fullname": "Elon Musk",
  "avatar": "https://pbs.twimg.com/profile_images/.../kqqe2iWO_bigger.jpg",
  "verified": "blue",
  "text": "I did all my coding with just Diet Coke, burgers & pizza",
  "html": "I did all my coding with just Diet Coke, burgers & pizza",
  "isRetweet": false,
  "retweetedBy": "",
  "replyingTo": []
}