All About Custom Integrations

In our last post, we looked at what it means when we say WebhookDB is designed to be self-hosted. The reason were:

• WebhookDB will end up on the critical path of your application and ETL, and you should have full ownership over its operations, including if and when you take downtime, how your data is secured, and what compute and storage resources should be available.
• There's no way to write integrations for every single API, and any 'custom integration' system would have too many constraints to be useful. Self-hosting makes running custom integrations easy.

In this post we focus on the second, around custom integrations.

The fact is, even after having developed integrations for dozens of APIs, we still find new things thrown at us. This is inevitable with APIs in verticals like healthcare, education, and travel, that cater to enterprise clients. Designing a system to dynamically run user-supplied code, needed for custom integrations, is a dead-end of complexity, security, and developer experience.

Instead, when you write a custom integration for WebhookDB, you have access to the same environment we do as first-party developers. That means:

• You write code that fulfills an interface, such as declaring how to transform and index data from the API, and what the unique external ID is.
• You can unit test your interface against our shared behaviors that assert it works correctly.
• You have no restrictions on the behavior of your integration. While you should rarely if ever have to override the way WebhookDB integrations work, you probably can't control the API you're integrating. Sometimes you will need to do things we all recognize are not ideal, like multiple HTTP calls when upserting. This is a decision you can make.

Custom integrations go into a plugin folder that is loaded a runtime, so they will never conflict with bundled plugins.

WebhookDB is delivered as a Git repo, as explained in our last article. We'll push changes to this repo when there are relevant updates according to an agreed upon process, like using particular branch names.

Your custom integration/plugin development can go on in this repo, so you'll keep history and can use your normal development process. It should feel pretty similar to however you currently develop software. Except we've taken out a lot of the complexity, so you know you end up with custom integrations that 'just work.'

Hopefully that provides some insight into how WebhookDB handles custom integrations for self-hosted customers. If you're not yet a customer, you can get started with our hosted version, and migrate seamlessly when you're ready. Give WebhookDB a try or get in touch.