Bluesky Feed Scraper for Apify This is an Apify actor that scrapes data from a specified Bluesky feed URL and outputs detailed information about the posts, including metadata, authors, embedded media, and statistics such as likes, replies, and reposts. ## Features - Scrapes Bluesky feed posts from a given feed URL. - Extracts detailed post data, including: - Author details (DID, handle, display name, avatar URL, etc.). - Post text, tags, and languages. - Embedded images, with metadata (alt text, aspect ratio, URLs). - Engagement statistics (likes, replies, reposts, quotes). - Thread and reply information. - Record metadata, including creation and indexing timestamps. ## Input The actor requires the following input: | Field | Type | Description | |---------------|--------|-----------------------------------------------| | url | String | The URL of the Bluesky feed you want to scrape. Example: https://bsky.app/profile/username/feed. | ### Example Input json { "url": "https://bsky.app/profile/c3rmen.bsky.social/feed" } ## Output The actor produces a JSON array where each object represents a post from the feed. The structure includes: - uri and cid: Unique identifiers for the post. - author: Details about the author (DID, handle, avatar, etc.). - record: Post text, tags, languages, and embedded media. - embed: View-ready image metadata (e.g., thumbnails, full-size URLs). - Engagement metrics (replyCount, repostCount, likeCount, quoteCount). - Thread and reply-related data. - Timestamps (createdAt, indexedAt). ### Example Output json [ { "uri": "at://did:plc:z72i7hdynmk6r22z27h6tvur/app.bsky.feed.post/3lbsizxfxa22r", "cid": "bafyreifohcetdw6e5mudaz6anigzsm5ssjpm3oreyxu4a2l665k7hpxo4q", "author": { "did": "did:plc:z72i7hdynmk6r22z27h6tvur", "handle": "bsky.app", "displayName": "Bluesky", "avatar": "https://cdn.bsky.app/img/avatar/plain/did:plc:z72i7hdynmk6r22z27h6tvur/bafkreihagr2cmvl2jt4mgx3sppwe2it3fwolkrbtjrhcnwjk4jdijhsoze@jpeg", "associated": { "chat": { "allowIncoming": "none" } }, "labels": [], "createdAt": "2023-04-12T04:53:57.057Z" }, "record": { "createdAt": "2024-11-25T21:52:30.840Z", "embed": { "external": { "description": "Bluesky is social media as it should be. Find your community among millions of users, unleash your creativity, and have some fun again. https://bsky.app", "thumb": { "ref": { "$link": "bafkreihh7dthuxfqel6zwcmxapcu47tr34rat7thjtxlfmrwidvxfsmqne" }, "mimeType": "image/jpeg", "size": 384236, "$type": "blob" }, "title": "BlueskySocial - Twitch", "uri": "https://www.twitch.tv/blueskysocial" }, "$type": "app.bsky.embed.external" }, "facets": [ { "features": [ { "did": "did:plc:qjeavhlw222ppsre4rscd3n2", "$type": "app.bsky.richtext.facet#mention" } ], "index": { "byteEnd": 55, "byteStart": 40 }, "$type": "app.bsky.richtext.facet" }, { "features": [ { "did": "did:plc:ragtjsm2j2vknwkz3zp4oxrd", "$type": "app.bsky.richtext.facet#mention" } ], "index": { "byteEnd": 76, "byteStart": 64 }, "$type": "app.bsky.richtext.facet" }, { "features": [ { "did": "did:plc:4ewnpnebeh7zuk5pbardaxqz", "$type": "app.bsky.richtext.facet#mention" } ], "index": { "byteEnd": 226, "byteStart": 203 }, "$type": "app.bsky.richtext.facet" } ], "langs": [ "en" ], "text": "Join us for another livestream with COO @rose.bsky.team and CTO @pfrazee.com, where they'll share team updates, the story of how Bluesky began, and what’s next. \n\nPlus, a special guest appearance from @flavorflav.bsky.social! 🎉\n\nToday 11/25 @ 5 pm PT / 8 pm ET / 1 am GMT / 10am JST", "$type": "app.bsky.feed.post" }, "embed": { "external": { "uri": "https://www.twitch.tv/blueskysocial", "title": "BlueskySocial - Twitch", "description": "Bluesky is social media as it should be. Find your community among millions of users, unleash your creativity, and have some fun again. https://bsky.app", "thumb": "https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:z72i7hdynmk6r22z27h6tvur/bafkreihh7dthuxfqel6zwcmxapcu47tr34rat7thjtxlfmrwidvxfsmqne@jpeg" }, "$type": "app.bsky.embed.external#view" }, "replyCount": 324, "repostCount": 1041, "likeCount": 9147, "quoteCount": 84, "indexedAt": "2024-11-25T21:52:35.058Z", "labels": [] }, // ...more posts ] ## Usage 1. Deploy the Actor: Use the Apify console to set up and deploy this actor. 2. Provide Input: Supply the url in the input configuration. 3. Run the Actor: Start the actor, and it will scrape the feed URL and return the posts as JSON. ## Notes - Ensure the url is publicly accessible. - The actor fetches only visible posts; private or restricted feeds will not be included. Feel free to suggest additional features or report any issues! 🚀