Add Change Data Capture to Any API

Rob Galanakis on March 20, 2023

boxes of documents stacked in a warehouse

Change Data Capture is the process of capturing changes to data and delivering those changes in real-time to downstream systems.

Most databases support some form of CDC. Webhooks, available in many APIs, provide a form of CDC. Many APIs, however, don't support webhooks at all, or their support is incomplete (not available for certain types of resources, or their delivery can be unreliable so you can miss key CDC events).

WebhookDB allows you to set up Change Data Capture for any API, no matter what its webhooks look like.

As an extreme case, let's take an example of an API that doesn't support webhooks at all. WebhookDB will paginate through resources and conditionally upsert only modified rows, using the most efficient algorithm possible. For example, many APIs have some sort of 'last modified' timestamp, but many do not. In these cases, we add our own timestamp, and will efficiently compare the payload and mark the row as modified only if the resource data has changed.

These modified rows are then synced to a database or HTTP endpoint as part of WebhookDB Change Data Capture implementation (see more on webhookdb httpsync and webhookdb dbsync. You can also set up your own webhooks to be sent from WebhookDB using webhookdb notification.

WebhookDB's CDC is so simple and flexible, you can even use it for CDC with your own application databases. Database CDC is usually trigger, replication, or query based. WebhookDB fits somewhere in the "query" CDC implementation. It's dead simple to set up, with no database schema or infrastructure dependencies, so it can be an excellent first step.

Relying on flexible, easy, and inexpensive commodity software to solve interesting problems for the 99% of software developers is WebhookDB's bread and butter. If you're interested in setting up Change Data Capture with WebhookDB, get set up with WebhookDB or get in touch.

Recent Blog Posts

AI-generated image of balloons and a computer
WebhookDB is Open Source

March 11, 2024

We're aligning our business with our values and community and going Open Source,

Read More →
zoomed in artificial snowflakes, each unique
Every API is Unique

June 8, 2023

Just like people, every API is unique in its own special way.

Read More →
webhookdb hook logo wearing angel wings
WebhookDB Gives You Wings!

June 1, 2023

Answer any question instantaneously, instead of drowning in documentation and tools.

Read More →
programmers reading code behind two doors, one with more cursing than the other, correlating code quality and cursing
Why would they do that!

May 24, 2023

Or, how to stop worrying and learn to love every API.

Read More →