Twitter X-Plorer 🦜

# 💫 Welcome to Twitter X-Plorer ## 🦜 About Twitter.com Twitter is an online social media and social networking service owned and operated by American company X Corp., the legal successor of Twitter, Inc. Twitter users outside the United States are legally served by the Ireland-based Twitter International Unlimited Company, which makes these users subject to Irish and European Union data protection laws[9][10] ## 🦜 About This Actor 💫 All-In-One Twitter Data Extractor - ⭐Scrape User timeline, followers, following, media, likes, lists, topics, highlights, etc. - ⭐Scrape List timeline, members, followers - ⭐Scrape Specific Topic - ⭐Scrape Status & Threads - ⭐Scrape Media: Photo / Video / Gif, etc. - ⭐Advanced Search - ⭐Content Formatting: HTML or Plain TEXT - ⭐Low Memory Cost - ⭐A.S.A.P DISCLAIMER: > - This Twitter scraper only collects data that’s publicly available. This means data that’s accessible without logging in to Twitter and without accepting Twitter’s terms of use. Please note that if you accepted Twitter’s terms of use, your ability to scrape Twitter data may be limited. If that is the case, please review the terms and make an informed decision yourself. > - By providing your personal auth_token, you agree to obey twitter TOS, especially regarding rate-limit policy 50 request per 15 minutes enforced by elonmusk ## 🦜 Tutorial ### 👉 Parameters | Parameter | Type | Options | Description | |-|-|-|-| | query | string | | List of Query, Keywords or URL | filters | object | | Advanced search filters | limit | integer | numeric | Number of results (per-query) | content | string | text, html | Format of results content | auth_token | string | cookie | Value of auth_token cookie 💫 NOTES : If a multiline Query is provided (Queries separated by line breaks), each line will be processed separately. Example : ### 👉 Basic Usage Scrape user profiles yaml { "query": "@elonmusk @apify" } Scrape user posts yaml { "query": "@elonmusk/posts" } Advanced Search (Cookies required) yaml { "query": "what is happening", "limit": 10 } Scrape an URL yaml { "query": "https://x.com/elonmusk/media", "limit": 10 } ### 👉 Advanced Usage Twitter Query Language (TQL) html [ <COMMAND:> | # | @ ] [ <NAME> | <ID> | <KEYWORD> | <URL> ] [ /<SECTION> ] #### ❓ QUERIES Overview | Format | Examples | Description | |-|-|-| | <KEYWORD> | web scraping | Search Anything | | jobs:<KEYWORD> | jobs:python engineer | Search Jobs | | #<HASHTAG> | #AI | Search Hashtag | | @<USER>
/info /about /replies /media /articles
/likes /affiliates /followers
/following /lists /topics
/highlights /subscriptions /jobs | @elonmusk
@elonmusk/info
@elonmusk/about
@elonmusk/replies
@elonmusk/followers | Scrape User Data | | <STATUS_ID>
/info /quotes /reposts /likes | 1562015197543497728 | Scrape Posting (Tweet) | | topic:<TOPIC_ID> | topic:1280550787207147521 | Scrape Topic | | list:<LIST_ID>
/info /posts
/members /followers | list:1334803406523953152
list:1334803406523953152/posts
list:1334803406523953152/members | Scrape List | | comm:<COMM_ID> | | Scrape Community | | https:<URL> | https://x.com/... | Start URL | #### ❓ Possible QUERY Values : | Format | Example | Description | |-|-|-| | <KEYWORD> | web scraping | Search Anything | | #<HASHTAG> | #AI | Search Hashtag | | @<USER> | @elonmusk | Scrape User Data | | /info | @elonmusk/info | User info | | /about | @elonmusk/about | User about | | /replies | @elonmusk/replies | User replies | | /highlights | @elonmusk/highlights | | | /media | @elonmusk/media | | | /jobs | @elonmusk/jobs | Jobs opportunity | | /articles | @elonmusk/articles | | | /likes | @elonmusk/likes | | | /lists | @elonmusk/lists | | | /topics | @elonmusk/topics | | | /affiliates | @elonmusk/affiliates | | | /followers | @elonmusk/followers | | | /verified_followers | @elonmusk/verified_followers | | | /followers_you_know | @elonmusk/followers_you_know | | | /following | @elonmusk/following | | | /subscriptions | @elonmusk/subscriptions | | | <STATUS_ID> | 1562015197543497728 | Scrape Posting (Tweet) | | /info | 1562015197543497728/info | Post content | | /replies | 1562015197543497728/replies | | | /quotes | 1562015197543497728/quotes | | | /reposts | 1562015197543497728/reposts | | | /likes | 1562015197543497728/likes | | |/analytics | 1562015197543497728/analytics | | | topic:<TOPIC_ID> | topic:1280550787207147521 | Scrape Topic Timeline | | list:<KEYWORD> | list:java script | Search List| | list:<LIST_ID> | list:1334803406523953152 | List Data| | /info | list:1334803406523953152/info | List info | | /posts | list:1334803406523953152/posts | List posts | | /members | list:1334803406523953152/members | List members | | /followers | list:1334803406523953152/followers | List followers| | com:<KEYWORD> | com:python | Search Community| | com:<COM_ID> | com:1672458762852921344 | Community Data| | /info | com:1672458762852921344/info | Community info | | /top | com:1672458762852921344/top | Top posts | | /latest | com:1672458762852921344/latest | Latest posts | | /media| com:1672458762852921344/media | Media posts | | /members| com:1672458762852921344/members | Members list | | /moderators| com:1672458762852921344/moderators | Moderators list | | job:<KEYWORD> | job:python engineer | Search Jobs | | job:<JOB_ID> | job:1723106649395671391 | Job Data | | https:<URL> | https://x.com/... | Start URL| ### 👉 TheAUTH_TOKEN parameter (auth_token cookie) Some function require auth_token cookie to work properly (required sign-in to Twitter.com). When you receive log error something like below, then probably you need to supply parameters with auth_token value. json ❌ Authorization: Denied by access control: unspecified reason ❌ HTTP error 404: Not Found Important Notes : > - This is NOT your APIFY Token, instead a value from your browser cookie, named auth_token. > - Use this only if necessary, as it have risk your account getting blocked by @elonmusk. > - Your cookies is your SECRET. Please don't share it with someone else. > - The auth_token value will always valid until you logged out from Twitter.com To get auth_token cookie value : 1. Login to Twitter.com 2. Open Chrome Developer Tools (Ctrl + Shift + I) 3. Open Application Tab 4. On left panel, go to: Storage -> Cookies -> https://twitter.com 5. Find cookie named auth_token (40 characters string value). 6. Copy & Paste Here : yaml { "query": "@elonmusk/followers", "auth_token": "YOUR_TWITTER_AUTH_TOKEN" } ## 🦜 Input Examples ### 👉 Example #1 : Searching Search for Posts by Keyword yaml { "query": "Web Scraping" } Search for Posts by #Hashtag yaml { "query": "#AI" } ### 👉 Example #2 : User Info User profile @USERNAME yaml { "query": "@apify" } User media @USERNAME/media yaml { "query": "@apify/media" } User following Lists: @USERNAME/lists yaml { "query": "@elonmusk/lists" } ### 👉 Example #3 : Topic Topic latest Posts : topic/<TOPIC_ID> yaml { "query": "topic:1280550787207147521/posts" } # Arts & Culture List lates Posts :list/<LIST_ID> yaml { "query": "list:1334803406523953152/posts" } # Web scraping + automation ### 👉 Example #4 : Advanced Searchfilters yaml { "filters.from" : "@apify", # (@ symbol is optional) "filters.hashtag" : "#scraper", # (# symbol is optional) "filters.type" : "videos", "filters.phrase" : "Space The Final Frontier", "filters.replies" : 250, "filters.since" : "2022-01-20" } ### 👉 Example #5 : Content Formatting In case you want content format in HTML use content parameter : yaml { "query": "@apify/posts", "limit": 10, "content": "html" } ## 🦜 Advanced Search Filters | Parameter | Type | Example | Summary | Description | |-|-|-|-|-| | filters.raw | string | Animals +cat -dog lang:en | | Searching raw query | filters.type | string | One of: latest, top, photos, videos, people | |Post type | filters.word | string | what’s happening |both “what’s” and “happening” | All of these words | filters.phrase | string | happy hour | the exact phrase “happy hour” | This exact phrase | filters.any | string | cats dogs | either “cats” or “dogs” (or both) | Any of these words | filters.exclude | string | cats dogs | does not contain “cats” and does not contain “dogs” | None of these words | filters.hashtag | string | #ThrowbackThursday | the hashtag #ThrowbackThursday | These hashtags | filters.from | string | @Twitter | sent from @Twitter | From these accounts | filters.to | string | @Twitter | sent in reply to @Twitter | To these accounts | filters.mention | string | @SFBART @Caltrain | mentions @SFBART or mentions @Caltrain | Mentioning these accounts | filters.replies | integer | 250 | | Minimum replies | filters.faves | integer | 200 | | Minimum likes | filters.retweets | integer | 100 | | Minimum retweets | filters.since | date | 2022-01-20 | | Since date YYYY-MM-DD | filters.until | date | 2022-02-30 | | Until date YYYY-MM-DD ## 🦜 Did You Know ? Twitter internally detect faces on images. yaml ... "media": [ { "id": "1695968940185534464", "image": "https://pbs.twimg.com/media/F4lKw_XawAAT-xn.jpg", "url": "https://t.co/Lk3VtAaKQ7", "features": { "orig": { "faces": [{"x": 315, "y": 130, "h": 232,"w": 232}] } }, "__type": "photo", "key": "3_1695968940185534464" } ] ## 🦜 To-Do List - Constructing status URL: https://twitter.com/_/status/<status_id> or https://twitter.com/<screen-name>/status/<status-id> - Resize and format images URL: https://pbs.twimg.com/media/xxxxxxxxxx.jpg?format=[jpg|png|webp]&name=[orig|normal|large|medium|thumb] ## 🦜 Support ⚡️ Feel free to reach out to the developer for any issues or suggestions for improvement.