Never miss API changes
Rob Galanakis on May 17, 2023
When changes to 3rd party systems absolutely cannot to missed, how do you make sure you never, ever make a mistake?
Maybe you are using an API to send lots of money, and you need to check when payments clear or fail. While it's not hard for a few thousand payments, how do you handle millions of payments?
You can quickly see some solutions and related problems:
- Polling is a huge resource sink, causing millions of useless API requests.
- Webhooks can be unreliable if you're experiencing downtime or bugs.
- Your task processing system may not be 100% durable.
- Event logs can be difficult to process properly, if your API provider has them at all.
These are all significant problems to using 3rd party APIs at scale. But they're problems WebhookDB is designed to solve:
- Instead of polling an API, you query a database. You can scale this database as you need (or use something serverless). Queries are orders of magnitude faster than HTTP or gRPC requests.
- Webhooks can be used to get rapid updates, but absolute reliability is achieved through other means.
- Designed to be 100% durable and resilient, including against process termination, disk failures, and datastore availability.
- In addition to webhooks and event logs, sync with incremental polling.
While your application could do all this, it probably shouldn't. The requirements of your application are different from the requirements for perfect synchronization of 3rd party APIs. The same way you don't build a database, message broker, or HTTP server into your application, there's no reason to build your own API syncing capability. Run WebhookDB, and then interface with it via database queries, HTTP Sync, or DB Sync.