Webhook Event Store & Replay System > The ultimate debugging tool for webhook integrations. Capture, inspect, and replay webhook events from any service. ## 🎯 What It Does Webhook Event Store provides a complete lifecycle management system for webhooks. Capture incoming webhooks from services like Stripe, GitHub, Shopify, or custom APIs, store them with full metadata, inspect payloads, validate signatures, and replay events to any endpoint for testing and debugging. Perfect for: - Backend Developers: Debug webhook integrations without waiting for real events - QA Teams: Test webhook handling with real production payloads - DevOps Engineers: Audit webhook activity and troubleshoot delivery issues - Integration Engineers: Validate webhook signatures and inspect headers ## ✨ Key Features ### 🎣 Capture Mode - Receive & Store Webhooks - HTTP Server: Spin up endpoint to receive webhooks from any service - Automatic Storage: All webhooks saved to Apify dataset with full metadata - Signature Validation: Built-in support for GitHub, Stripe, and generic HMAC validation - Event Type Detection: Automatically identifies event types from headers/payload - Smart Auto-Stop: Stop capturing after inactivity or max duration - Health Endpoint: Monitor capture status in real-time ### 🔄 Replay Mode - Resend Events - Targeted Replay: Send captured events to any endpoint for testing - Flexible Filtering: Replay all events, specific IDs, or filter by event type - Replay Headers: Mark replayed events with special headers - Rate Limiting: Configure delays between replays to respect limits - Response Tracking: Record response status and headers for each replay - Bulk Operations: Replay multiple events in sequence ### 🔍 Query Mode - Search & Inspect - Advanced Filtering: Search by event type, source IP, date range - Payload Inspection: View complete headers, query params, and body - Chronological View: Events sorted by timestamp - Result Limiting: Control output size for large datasets - Age Indicators: Human-readable timestamps ("2 hours ago") ## 🚀 Use Cases ### Use Case 1: Debugging Stripe Webhooks Problem: Your Stripe integration isn't working, but you can't see the webhook payload json { "mode": "capture", "captureConfig": { "port": 3000, "captureEndpoint": "/stripe-webhook", "validateSignatures": true, "signatureHeader": "Stripe-Signature", "signatureSecret": "whsec_your_stripe_secret", "maxCaptureMinutes": 30 } } Result: Point Stripe at your Actor's public URL, trigger test events, inspect full payload ### Use Case 2: Testing GitHub Webhook Handlers Problem: Need to test your GitHub webhook handler without creating real PRs/issues json { "mode": "capture", "captureConfig": { "port": 3000, "captureEndpoint": "/github", "validateSignatures": true, "signatureHeader": "X-Hub-Signature-256", "signatureSecret": "your_github_webhook_secret", "maxCaptureMinutes": 60 } } After capturing real events, replay them to your dev environment: json { "mode": "replay", "replayConfig": { "targetUrl": "https://dev.myapp.com/webhooks/github", "replayAll": false, "filterByEventType": "github.pull_request", "addReplayHeaders": true, "delayBetweenRequests": 500 }, "storageDatasetId": "your_capture_dataset_id" } ### Use Case 3: Audit Webhook Activity Problem: Need to investigate webhook delivery issues from last week json { "mode": "query", "queryConfig": { "dateFrom": "2025-11-17T00:00:00Z", "dateTo": "2025-11-24T23:59:59Z", "filterByEventType": "shopify.orders/create", "limit": 100, "includePayload": true }, "storageDatasetId": "your_capture_dataset_id" } ## 📥 Input Configuration ### Mode Selection (Required) - mode: "capture" | "replay" | "query" ### Capture Mode Configuration json { "captureConfig": { "port": 3000, // Server port "captureEndpoint": "/webhook", // URL path for webhooks "validateSignatures": false, // Enable signature validation "signatureHeader": "X-Signature", // Header containing signature "signatureSecret": "", // Secret for validation "maxCaptureMinutes": 60, // Max capture duration "autoStop": true // Stop after 5min inactivity } } Signature Validation Support: - GitHub: X-Hub-Signature-256 (HMAC SHA256) - Stripe: Stripe-Signature (HMAC SHA256 with timestamp) - Generic: Any HMAC SHA256 signature header ### Replay Mode Configuration json { "replayConfig": { "targetUrl": "https://example.com/webhook", // Required "replayAll": true, // Replay all events "eventIds": [], // Specific event IDs (if not replayAll) "filterByEventType": "", // Filter by event type "addReplayHeaders": true, // Add X-Webhook-Replay header "delayBetweenRequests": 100 // Delay in milliseconds }, "storageDatasetId": "abc123" // Dataset from capture mode } ### Query Mode Configuration json { "queryConfig": { "filterByEventType": "", // Optional: Filter by event type "filterBySource": "", // Optional: Filter by source IP "dateFrom": "", // Optional: ISO 8601 start date "dateTo": "", // Optional: ISO 8601 end date "limit": 100, // Max results "includePayload": true // Include full payload }, "storageDatasetId": "abc123" } ## 📤 Output Data ### Capture Mode Output Each captured webhook is stored with complete metadata: json { "id": "webhook_1700000000000_abc123", "timestamp": "2025-11-24T15:30:00.000Z", "method": "POST", "path": "/webhook", "headers": { "content-type": "application/json", "x-github-event": "pull_request", "x-hub-signature-256": "sha256=..." }, "query": {}, "body": { "action": "opened", "pull_request": { ... } }, "sourceIp": "140.82.115.0", "eventType": "github.pull_request", "signatureValid": true } ### Replay Mode Output Each replay generates a result record: json { "originalEventId": "webhook_1700000000000_abc123", "replayedAt": "2025-11-24T16:00:00.000Z", "responseStatus": 200, "responseHeaders": { "content-type": "application/json" }, "success": true } ### Query Mode Output Filtered events with age indicators: json { "id": "webhook_1700000000000_abc123", "timestamp": "2025-11-24T15:30:00.000Z", "eventType": "github.pull_request", "method": "POST", "path": "/webhook", "sourceIp": "140.82.115.0", "signatureValid": true, "age": "30 minutes ago", "headers": { ... }, // If includePayload: true "query": { ... }, // If includePayload: true "body": { ... } // If includePayload: true } ## 🔐 Security Best Practices ### Signature Validation Always enable signature validation for production webhooks: json { "validateSignatures": true, "signatureHeader": "X-Hub-Signature-256", "signatureSecret": "your_webhook_secret" } ### Network Access - Use Apify's standby mode to keep capture running - Configure firewall rules to limit webhook sources - Use HTTPS for replay targets to protect sensitive data ### Data Privacy - Be mindful of PII in webhook payloads - Set appropriate dataset retention periods - Don't share dataset IDs with untrusted parties ## 💡 Advanced Workflows ### Workflow 1: Continuous Webhook Monitoring 1. Run Actor in capture mode on standby 2. Configure webhook provider to send to Actor URL 3. Schedule periodic queries to analyze patterns 4. Set up alerts for failed signature validations ### Workflow 2: Integration Testing Pipeline 1. Capture production webhooks once 2. Store dataset ID in your test config 3. Replay to staging environment before deployments 4. Validate handler behavior against real payloads ### Workflow 3: Debugging Production Issues 1. Enable capture mode during incident 2. Collect 10-20 minutes of webhook traffic 3. Query for suspicious patterns or errors 4. Replay problematic events to local dev environment 5. Fix issue and verify with replay ## 🛠 Technical Details ### Capture Mode Architecture - HTTP Server: Express.js with body-parser - Raw Body Preservation: For accurate signature validation - Event Type Detection: Heuristic-based header/payload analysis - Auto-stop Logic: Configurable inactivity timeout ### Signature Validation Supports multiple signature schemes: - GitHub: SHA256 HMAC with sha256= prefix - Stripe: SHA256 HMAC with timestamp (t=,v1=) - Generic: Plain SHA256 HMAC Uses crypto.timingSafeEqual() to prevent timing attacks. ### Replay Mechanism - Header Sanitization: Removes hop-by-hop headers (host, connection) - Replay Markers: Adds custom headers to identify replays - Rate Limiting: Configurable delays between requests - Error Handling: Continues on failures, logs all results ### Query Performance - In-Memory Filtering: Fast for datasets up to 10k events - Chronological Sorting: Most recent events first - Efficient Output: Optional payload exclusion for large datasets ## 📊 Typical Performance | Operation | Duration | Notes | |-----------|----------|-------| | Capture (1 event) | <50ms | Per webhook received | | Replay (1 event) | 100-500ms | Depends on target latency | | Query (1000 events) | 1-3s | With filtering and sorting | | Signature validation | <10ms | Per webhook | ## 🔄 Integration Examples ### Stripe Setup bash # In Stripe Dashboard: # Webhook URL: https://api.apify.com/v2/acts/YOUR_ACTOR/runs/last/webhook # Events: Select all or specific events # Secret: Copy to signatureSecret input ### GitHub Setup bash # In GitHub Repo Settings → Webhooks: # Payload URL: https://api.apify.com/v2/acts/YOUR_ACTOR/runs/last/webhook # Content type: application/json # Secret: Generate and copy to signatureSecret input # Events: Select triggers (Push, PR, Issues, etc.) ### Custom Webhook Testing bash # Send test webhook with curl curl -X POST https://api.apify.com/v2/acts/YOUR_ACTOR/runs/last/webhook \ -H "Content-Type: application/json" \ -H "X-Custom-Signature: your_signature" \ -d '{"event": "test", "data": {"key": "value"}}' ## 🚨 Troubleshooting ### Webhooks Not Captured - Check Actor is running in standby mode - Verify webhook URL matches captureEndpoint - Check webhook provider for delivery failures - Review Actor logs for errors ### Signature Validation Failing - Verify signatureSecret matches provider - Check signatureHeader name (case-insensitive) - Ensure webhook provider is using supported algorithm - Review Actor logs for validation details ### Replay Not Working - Verify targetUrl is accessible - Check target endpoint accepts replayed content-type - Review replay output for response status codes - Ensure delayBetweenRequests doesn't cause timeouts ## 📈 Best Practices ### For Development - Start with capture mode to collect real events - Use query mode to inspect payload structure - Replay to localhost with ngrok/tunnels for debugging - Keep signatureSecret in environment variables ### For Production - Always enable signature validation - Set reasonable maxCaptureMinutes - Use autoStop to conserve resources - Monitor dataset size and clean up periodically ### For Testing - Capture production events once, replay many times - Filter by event type to test specific handlers - Use addReplayHeaders to identify test traffic - Adjust delayBetweenRequests for rate limits ## 🤝 Contributing Found a bug? Need a feature? Please open an issue! Suggested improvements: - Additional signature validation schemes (HMAC SHA1, RSA) - Webhook transformation/filtering before storage - Built-in replay scheduling - Real-time webhook forwarding mode ## 📄 License MIT License - use freely in your projects! ## 🏆 Apify $1M Challenge Built for the Apify $1M Challenge to solve real webhook debugging pain points. Help us improve: - Share feedback on signature validation - Suggest new webhook providers to support - Report edge cases or bugs - Star and share if you find it useful! --- Debug webhooks like a pro 🎯