Add Change Data Capture to Any API
Rob Galanakis on March 20, 2023
Change Data Capture is the process of capturing changes to data and delivering those changes in real-time to downstream systems.
Most databases support some form of CDC. Webhooks, available in many APIs, provide a form of CDC. Many APIs, however, don't support webhooks at all, or their support is incomplete (not available for certain types of resources, or their delivery can be unreliable so you can miss key CDC events).
WebhookDB allows you to set up Change Data Capture for any API, no matter what its webhooks look like.
As an extreme case, let's take an example of an API that doesn't support webhooks at all. WebhookDB will paginate through resources and conditionally upsert only modified rows, using the most efficient algorithm possible. For example, many APIs have some sort of 'last modified' timestamp, but many do not. In these cases, we add our own timestamp, and will efficiently compare the payload and mark the row as modified only if the resource data has changed.
These modified rows are then synced to a database or HTTP endpoint as part of
WebhookDB Change Data Capture implementation (see more on
webhookdb dbsync. You can also set up your own webhooks
to be sent from WebhookDB using
WebhookDB's CDC is so simple and flexible, you can even use it for CDC with your own application databases. Database CDC is usually trigger, replication, or query based. WebhookDB fits somewhere in the "query" CDC implementation. It's dead simple to set up, with no database schema or infrastructure dependencies, so it can be an excellent first step.
Relying on flexible, easy, and inexpensive commodity software to solve interesting problems for the 99% of software developers is WebhookDB's bread and butter. If you're interested in setting up Change Data Capture with WebhookDB, get set up with WebhookDB or get in touch.
Recent Blog Posts
May 11, 2023
Learn about securing API keys when using React or building any client software.Read More →
May 4, 2023
Ensure a smooth launch of your integrations with 3rd party APIs using our checklist.Read More →
April 17, 2023
Learn about our new transparent and affordable pricing tiers.Read More →
April 10, 2023
Most APIs we use every day are not your Stripes or Shopifys. These are the API 99%.Read More →