🛠️ X / Twitter Scraper (Ultimate) Professional-grade Twitter/X data extraction tool that delivers structured JSON data instantly. Perfect for social media monitoring, lead generation, content research, and competitive analysis. --- ## What You Can Extract ### 📱 Profile Data - Complete user profiles with bio, location, website, avatar, banner - Follower/following counts and tweet statistics - Join dates and verification status - Profile metadata for lead generation ### 🐦 Tweet Content - Full tweet text with hashtags and mentions - Media URLs (images, videos with HLS preview links) - Engagement metrics (likes, retweets, comments, quotes) - Timestamps and tweet URLs - Author information with verification details ### 🔍 Search Capabilities - Global Tweet Search: Find tweets across all of Twitter/X - User Search: Discover accounts matching your criteria - Profile Search: Search within specific user profiles ### 📋 List Scraping - Extract all tweets from Twitter Lists - Track curated content collections - Monitor list updates automatically --- ## Key Features ✨ Intelligent Auto-Detection - Automatically determines the best scraping mode based on your inputs 🎯 Advanced Filtering - Filter by date range, location, media type, engagement, language, and more 🔒 Built-in Residential Proxy - Reliable data extraction with automatic proxy management 📊 Multiple Data Views - Pre-configured views for profiles, tweets, media, and engagement analysis ⚡ Streaming Results - Real-time data delivery as scraping progresses 🔄 Smart Retry Logic - Automatic retries with exponential backoff for failed requests 📈 Engagement Analytics - Filter by minimum likes, retweets, or replies --- ## Input Parameters ### Core Inputs Twitter Usernames (twitterHandles) - Enter one or more usernames (with or without @) - Example: elonmusk, apify, @naval - Automatically scrapes profile tweets List URLs (listUrls) - Twitter/X list URLs in format: https://x.com/i/lists/123456 - Scrapes all tweets from the specified lists Search Terms (searchTerms) - Search queries to find tweets or users - Examples: "web scraping", "#AI", "from:apify" - Works with global search or profile-specific search ### Search Configuration Search Type (searchType) - tweets - Search tweets globally across Twitter/X - users - Find Twitter accounts - profile_tweets - Search within specific profiles (requires usernames) ### Profile Options - Include Replies - Get reply tweets along with regular tweets - Include Retweets - Include retweeted content in results - Media Only - Extract only tweets containing images or videos - Scrape Profile Info - Extract complete profile details (default: enabled) ### Content Filters Media Filters: - Only Images - Tweets with images only - Only Videos - Tweets with videos only - Only Tweets with Links - Filter tweets containing external links Account Filters: - Only Verified Users - Limit to verified accounts - Only Quote Tweets - Extract quote tweets exclusively - Include Native Retweets - Include native retweet format (default: enabled) - Safe Search - Filter sensitive content ### Time & Location Filters - Start Date - Filter from date (YYYY-MM-DD format) - End Date - Filter until date (YYYY-MM-DD format) - Location - Filter by geographic location (e.g., "Los Angeles", "New York") ### Engagement Filters Set minimum thresholds: - Minimum Likes - Filter tweets by minimum like count - Minimum Replies - Filter tweets by minimum reply count - Minimum Retweets - Filter tweets by minimum retweet count ### Language & Limits - Tweet Language - Filter by language code (e.g., en, es, fr, de) - Max Items - Maximum items to scrape (0 = unlimited, default: 100) - Max Pages - Maximum pages per handle/query (default: 100, max: 100) - Max Retries - Retry attempts for failed requests (1-10, default: 5) --- ## Output Data Structure All scraped data is saved to the Apify dataset in structured JSON format with multiple pre-configured views: ### Available Views 1. All Results - Complete dataset with all fields 2. Profiles Only - Profile information view 3. Tweets Only - Tweet/post data view 4. Tweets with Media - Media-rich tweets 5. Top Engagement - Tweets sorted by engagement metrics ### Data Fields Profile Objects Include: - username, fullname, bio, location, website - joined date, avatar, banner URLs - stats object with tweets, followers, following counts - scrape_date timestamp Tweet Objects Include: - tweet_url, tweet_id, text, date - author object with profile details - hashtags, mentions, urls arrays - stats object with engagement metrics - media array with image/video URLs and metadata - is_retweet boolean flag - scrape_date timestamp Search Results Include: - search_query field for search context - type field indicating data type (profile, tweet, user, etc.) --- ## Usage Examples ### Example 1: Extract Profile Tweets json { "twitterHandles": ["elonmusk", "apify"], "maxItems": 100 } Scrapes the latest 100 tweets from each profile. ### Example 2: Media-Only Scraping json { "twitterHandles": ["apify"], "mediaOnly": true, "maxItems": 50 } Extracts only tweets containing images or videos. ### Example 3: Search with Filters json { "searchTerms": ["LLM", "AI"], "searchType": "tweets", "startDate": "2024-01-01", "endDate": "2024-12-31", "onlyVerified": true, "minLikes": 10, "tweetLanguage": "en", "maxItems": 200 } Searches for verified English tweets about LLM/AI from 2024 with at least 10 likes. ### Example 4: Profile Search json { "twitterHandles": ["apify"], "searchTerms": ["scraping", "automation"], "searchType": "profile_tweets", "maxItems": 100 } Searches for specific keywords within a profile's tweets. ### Example 5: User Discovery json { "searchTerms": ["data scientist", "ML engineer"], "searchType": "users", "maxItems": 50 } Finds Twitter accounts matching the search terms. ### Example 6: List Scraping json { "listUrls": ["https://x.com/i/lists/78783491"], "maxItems": 200 } Scrapes tweets from a Twitter List. ### Example 7: Engagement-Based Filtering json { "searchTerms": ["web scraping"], "searchType": "tweets", "minLikes": 50, "minRetweets": 10, "onlyImages": true, "maxItems": 100 } Finds high-engagement image tweets about web scraping. ### Example 8: Complete Profile with Replies json { "twitterHandles": ["elonmusk"], "includeReplies": true, "includeRetweets": true, "scrapeProfileInfo": true, "maxItems": 500 } Comprehensive profile extraction including replies and retweets. --- ## Output Views Explained The dataset includes 5 pre-configured views for easy data analysis: ### 📊 All Results View Displays complete dataset with all available fields. Perfect for comprehensive data export. ### 👤 Profiles Only View Filtered view showing only profile information. Ideal for lead generation and user research. ### 🐦 Tweets Only View Tweet-focused view sorted by date (newest first). Best for content analysis and trend monitoring. ### 🎬 Tweets with Media View Media-rich tweets only. Perfect for visual content analysis and media asset collection. ### 📈 Top Engagement View Tweets sorted by engagement metrics (likes + retweets). Great for identifying viral content. --- ## Technical Specifications ### Supported Operations - Profile timeline scraping (normal, with replies, media-only modes) - Twitter List extraction - Global tweet search - User account search - Profile-specific search - Advanced filtering and pagination ### Data Extraction - Automatic pagination handling - Real-time streaming results - Structured JSON output - Multiple data views for different use cases ### Reliability Features - Built-in residential proxy support (automatic) - Configurable retry logic (1-10 attempts) - Error handling and graceful degradation - Rate limiting protection --- ## Use Cases ### 🎯 Lead Generation Extract verified users, filter by engagement, and collect profile data for outreach campaigns. ### 📊 Social Media Monitoring Track mentions, hashtags, and trending topics with date and location filters. ### 🔍 Competitive Analysis Monitor competitor profiles, analyze engagement patterns, and track content strategies. ### 📈 Content Research Find high-performing tweets, analyze media content, and identify trending topics. ### 🎬 Media Collection Extract images and videos from profiles or search results with direct URLs. ### 👥 User Discovery Search for users by keywords, industry, or interests for targeted outreach. --- ## Best Practices 1. Start Small - Test with maxItems: 50 before scaling up 2. Use Filters - Apply engagement filters to get high-quality results 3. Date Ranges - Use date filters to focus on specific time periods 4. Residential Proxy - Enabled by default for reliable extraction 5. Monitor Limits - Set reasonable maxPages and maxItems to control costs 6. Language Filtering - Use tweetLanguage to target specific regions 7. Profile Info - Enable scrapeProfileInfo for complete user data --- ## Data Privacy & Compliance This tool is designed for public data extraction. Users are responsible for: - Compliance with Twitter/X Terms of Service - Adherence to data protection regulations (GDPR, CCPA, etc.) - Ethical use of scraped data - Respect for user privacy and consent requirements --- ## Support & Resources For technical support, feature requests, or questions: - Check the Actor logs for detailed error messages - Review input parameters in the Actor interface - Consult Apify documentation for platform-specific questions --- ## Version Current Version: 1.0 Categories: Social Media, Lead Generation, Automation Platform: Apify --- Ready to extract Twitter/X data? Configure your inputs, set your filters, and start scraping. The Actor handles everything automatically! 🚀