Add Change Data Capture to Any API

Rob Galanakis on March 20, 2023

boxes of documents stacked in a warehouse

Change Data Capture is the process of capturing changes to data and delivering those changes in real-time to downstream systems.

Most databases support some form of CDC. Webhooks, available in many APIs, provide a form of CDC. Many APIs, however, don't support webhooks at all, or their support is incomplete (not available for certain types of resources, or their delivery can be unreliable so you can miss key CDC events).

WebhookDB allows you to set up Change Data Capture for any API, no matter what its webhooks look like.

As an extreme case, let's take an example of an API that doesn't support webhooks at all. WebhookDB will paginate through resources and conditionally upsert only modified rows, using the most efficient algorithm possible. For example, many APIs have some sort of 'last modified' timestamp, but many do not. In these cases, we add our own timestamp, and will efficiently compare the payload and mark the row as modified only if the resource data has changed.

These modified rows are then synced to a database or HTTP endpoint as part of WebhookDB Change Data Capture implementation (see more on webhookdb httpsync and webhookdb dbsync. You can also set up your own webhooks to be sent from WebhookDB using webhookdb webhook.

WebhookDB's CDC is so simple and flexible, you can even use it for CDC with your own application databases. Database CDC is usually trigger, replication, or query based. WebhookDB fits somewhere in the "query" CDC implementation. It's dead simple to set up, with no database schema or infrastructure dependencies, so it can be an excellent first step.

Relying on flexible, easy, and inexpensive commodity software to solve interesting problems for the 99% of software developers is WebhookDB's bread and butter. If you're interested in setting up Change Data Capture with WebhookDB, get set up with WebhookDB or get in touch.

Recent Blog Posts

open lock hanging from a door
Securing API keys

May 11, 2023

Learn about securing API keys when using React or building any client software.

Read More →
checklist written in a notebook
API Integration Checklist

May 4, 2023

Ensure a smooth launch of your integrations with 3rd party APIs using our checklist.

Read More →
water falling into tiers of pools
New Pricing Structure

April 17, 2023

Learn about our new transparent and affordable pricing tiers.

Read More →
protesters in the occupy wall street movement
Building for the API 99%

April 10, 2023

Most APIs we use every day are not your Stripes or Shopifys. These are the API 99%.

Read More →