Designed for Self-Hosting
Rob Galanakis on March 27, 2023
WebhookDB is designed to be self hosted. There are two reasons we designed it this way:
- WebhookDB will end up on the critical path of your application and ETL, and you should have full ownership over its operations, including if and when you take downtime, how your data is secured, and what compute and storage resources should be available.
- There's no way to write integrations for every single API, and any 'custom integration' system would have too many constraints to be useful. Self-hosting makes running custom integrations easy.
In this post we focus on the first reason, around actual infrastructure and hosting concerns.
WebhookDB is designed to be as self-contained as possible. It requires:
- A PostgreSQL database for its application data. The needs here are modest and relatively constant, so even a small database can handle significant workloads.
- A Redis database for its job queue. The needs here scale with your usage. But a very high throughput is generally possible with modest resources.
- A way to run the container/application. We provide a Dockerfile, and WebhookDB can run natively in Render and Heroku. Ideally you'd have two process types (web and worker) running, but it's also possible to run with just a web process, and run the job system in the web process. You can even run WebhookDB in function-as-a-service systems like AWS Lambda.
- Somewhere to write the replicated API data. This is outside the scope of WebhookDB itself — all WebhookDB needs is a connection string that can manage tables in some database. Load on this database will scale with usage. As of this writing, WebhookDB can use Postgres or DynamoDB. Other databases are on our roadmap.
There are also a couple optional dependencies:
- Some way to send email. This can be via Postmark, Amazon SES, or SMTP.
- Error reporting. We currently support Sentry, but if you need another error reporting integration let us know.
WebhookDB is delivered to you as Git repository. This should make it simple to hook up to your existing build and deploy tooling, the same way you'd handle a first-party service. Delivering as a Git repository also makes it easy to write custom integrations and to support continuity as part of our friendly licensing.
You also get extensive documentation and runbooks to ease configuration and operation.
Our goal, and experience, is that it doesn't take more than a few minutes to get WebhookDB running in production.
Of course, if you're not ready to self-host, our hosted WebhookDB is pretty great, and can migrate seamlessly to a self-hosted version. In any case, we'd love it if you gave WebhookDB a try or got in touch.
Recent Blog Posts
March 11, 2024
We're aligning our business with our values and community and going Open Source,
Read More →June 8, 2023
Just like people, every API is unique in its own special way.
Read More →June 1, 2023
Answer any question instantaneously, instead of drowning in documentation and tools.
Read More →