Discord Message Scraper An Apify actor designed to scrape messages and channel data from Discord. This tool allows you to extract message history, guild channel lists, and other structured data from Discord servers and channels using a user or bot token. --- ## Features - Scrape Messages: Fetch up to a specified limit of messages from a Discord channel with support for cursor-based pagination. - Guild Channel List: Retrieve all channels within a guild, including metadata. - Guild Message Scraping: Collect messages from all text channels in a guild, with customizable limits. - Cursor Management: Automatically saves and reloads cursors to ensure continuity between scrapes. - Error Handling: Handles permission errors gracefully and provides detailed error logs. --- ## Use Cases - Data Analysis: Gather historical data for text analysis, sentiment tracking, or other insights. - Community Management: Monitor conversations or archive messages for compliance. - Content Backup: Maintain a backup of important conversations and shared media. - Guild Mapping: Extract the structure of a Discord server, including its channels and metadata. --- ## How It Works 1. Input Configuration: The actor takes input parameters like channel ID, Discord token, action type, and optional pagination cursor. 2. Action Selection: Based on the action parameter, the scraper performs one of the following: - Fetch channel messages. - Retrieve all guild channels. - Collect messages from all guild channels. 3. Cursor Management: Saves and retrieves cursors for seamless pagination. 4. Data Processing: Formats raw Discord data into structured JSON with metadata, attachments, and timestamps. 5. Output: Pushes the processed data to the Apify dataset for further use. --- ## Input Schema The scraper accepts a JSON object with the following parameters: | Field | Type | Required | Default | Description | | ------------ | -------- | -------- | --------- | --------------------------------------------------------------------------------------------------- | | channelId | string | Yes | "" | The ID of the Discord channel to scrape. | | token | string | Yes | "" | Discord user or bot token. Use carefully (may violate TOS). | | limit | number | No | 100 | Maximum number of messages to fetch (per channel in guild mode). | | action | string | Yes | "" | Action to perform: "Scrape Messages", "Scrape Channel List for Guild", "Scrape Messages for Guild". | | cursorType | string | No | "after" | Cursor direction: "before" or "after". | | cursor | string | No | null | Optional user-specified cursor for pagination. | --- ## Example Input json { "channelId": "123456789012345678", "token": "YOUR_DISCORD_TOKEN", "limit": 50, "action": "Scrape Messages", "cursorType": "after", "cursor": null } --- ## Output The actor outputs structured JSON data to the Apify dataset, formatted as: ### Scrape Messages json [ { "type": 0, "content": "Give me a fun picture", "mentions": [], "mention_roles": [], "attachments": [], "embeds": [], "timestamp": "2023-10-29T02:03:46.247000+00:00", "edited_timestamp": null, "flags": 0, "components": [], "resolved": { "users": {}, "members": {}, "channels": {}, "roles": {} }, "id": "1168007238766559311", "channel_id": "1143396487057899611", "author": { "id": "936929561302675456", "username": "Midjourney Bot", "avatar": "f6ce562a6b4979c4b1cbc5b436d3be76", "discriminator": "9282", "public_flags": 589824, "flags": 589824, "bot": true, "banner": null, "accent_color": null, "global_name": null, "avatar_decoration_data": null, "banner_color": null, "clan": null, "primary_guild": null }, "pinned": false, "mention_everyone": false, "tts": false } ] ### Scrape Channel List for Guild json [ { "id": "1143396487057899613", "type": 2, "last_message_id": null, "flags": 0, "guild_id": "1143396486411984942", "name": "Best Channel Ever", "parent_id": "1143396487057899611", "rate_limit_per_user": 0, "bitrate": 64000, "user_limit": 0, "rtc_region": null, "position": 0, "permission_overwrites": [], "nsfw": false }, { "id": "1143396487057899613", "type": 2, "last_message_id": null, "flags": 0, "guild_id": "1143396486411984943", "name": "Even Better Channel", "parent_id": "1143396487057899613", "rate_limit_per_user": 0, "bitrate": 64000, "user_limit": 0, "rtc_region": null, "position": 0, "permission_overwrites": [], "nsfw": false } ] --- ## Example Output - Messages: List of all scraped messages with metadata and attachments. - Channels: Array of guild channels with IDs, names, and types. --- ## Limitations - Discord TOS: Using a user token may violate Discord’s Terms of Service. Use responsibly. - Rate Limits: The scraper is subject to Discord API rate limits and permissions. - Data Size: Large guilds or channels may require multiple runs due to API limitations. --- ## Notes - Customization: The scraper can be extended to support additional data processing or Discord endpoints. Please reach out. --- ## Disclaimer This project is not affiliated with or endorsed by Discord. Use responsibly and adhere to Discord’s API Terms of Service.