X (Twitter) User profile and Timeline scraper This Apify Actor scrapes user timelines from X (Twitter) to extract tweets, user information, and engagement metrics. It works in guest mode without requiring authentication and can retrieve tweets from any public Twitter account. ## Features - Bulk scraping - Process multiple Twitter users in a single run - Timeline scraping - Extract tweets from any public X (Twitter) user timeline - Profile URL support - Accept both usernames and full profile URLs (twitter.com/x.com) - User information - Retrieve user profile data including verification status - Engagement metrics - Get retweet, like, and reply counts for each tweet - Media detection - Identify tweets with attached media (images, videos) - Guest mode operation - Works without authentication requirements - Batch processing - Process users in configurable batches with delays - Rate limiting protection - Configurable delays between requests - Structured output - Results include comprehensive tweet and user data - Configurable limits - Control the number of tweets retrieved ## Input Parameters | Field | Type | Required | Description | | ------------------------ | ------------- | -------- | ---------------------------------------------------------------- | | username | Array/String | Yes | Username(s) or profile URL(s) to scrape. Supports: elonmusk, @elonmusk, https://x.com/puma, https://twitter.com/nasa?ref=xyz | | count | Number | No | Number of tweets to retrieve per user (default: 20, max: 100) | | includeReplies | Boolean | No | Whether to include reply tweets in the results (default: false) | | includeRetweets | Boolean | No | Whether to include retweets in the results (default: true) | | batchSize | Number | No | Number of users to process in each batch (default: 5, max: 20) | | delayBetweenRequests | Number | No | Delay in seconds between processing each user (default: 2.0) | ### Example Input - Single User json { "username": "elonmusk", "count": 10, "includeReplies": false, "includeRetweets": true } ### Example Input - Bulk Scraping json { "username": [ "elonmusk", "https://x.com/puma", "@nasa", "https://twitter.com/apify" ], "count": 20, "includeReplies": false, "includeRetweets": true, "batchSize": 5, "delayBetweenRequests": 2.5 } ## How It Works 1. The actor validates and normalizes input parameters (usernames/URLs, count, etc.) 2. Extracts usernames from various input formats (plain username, @username, or full URLs) 3. Initializes a Twitter guest client (no authentication required) 4. Processes users in batches with configurable delays to avoid rate limiting 5. For each user: - Looks up the user ID from the username - Fetches the user's timeline tweets - Extracts comprehensive tweet data, user information, and engagement metrics 6. Results are pushed to the dataset with structured tweet and user information ## Output The actor outputs a structured object containing: json { "username": "elonmusk", "user_id": "44196397", "tweet_count": 20, "tweets": [ { "id": "1234567890", "text": "Tweet content here", "created_at": "2025-07-07T12:34:56", "retweet_count": 1000, "favorite_count": 5000, "reply_count": 200, "is_retweet": false, "is_reply": false, "user": { "id": "44196397", "name": "Elon Musk", "screen_name": "elonmusk", "verified": true }, "media": [ { "type": "photo", "url": "https://pbs.twimg.com/media/example.jpg" } ] } ] } ## Example Usage ### Single User - Plain Username json { "username": "elonmusk", "count": 10 } ### Single User - Profile URL json { "username": "https://x.com/puma?ref=homepage", "count": 15, "includeReplies": true, "includeRetweets": false } ### Bulk Scraping - Multiple Users json { "username": [ "elonmusk", "https://x.com/nasa", "@apify", "https://twitter.com/google" ], "count": 20, "batchSize": 2, "delayBetweenRequests": 3.0 } ## contact email : coredev.dan@gmail.com