Continuous Parquet Sync to Amazon S3

Rob Galanakis on March 27, 2023

apache parquet logo

WebhookDB can currently sync data from its central storage (PostgreSQL or DynamoDB) to another Postgres database, SnowflakeDB, or an arbitrary HTTP endpoint.

Today we are opening a waitlist for syncing data to Apache Parquet files stored in Amazon S3, which is probably the most common setup for data analytics tooling.

This uses WebhookDB's Change Data Capture concept and the the webhookdb dbsync command to automatically write changes to Parquet files stored in S3.

This setup, especially combined with serverless central storage like Amazon Aurora Serverless or DynamoDB, creates exceptionally low operational cost and complexity for WebhookDB.

If you're interested in syncing your data to Parquet files in Amazon S3, please let us know so we can get you on the waitlist.

Recent Blog Posts

zoomed in artificial snowflakes, each unique
Every API is Unique

June 8, 2023

Just like people, every API is unique in its own special way.

Read More →
webhookdb hook logo wearing angel wings
WebhookDB Gives You Wings!

June 1, 2023

Answer any question instantaneously, instead of drowning in documentation and tools.

Read More →
programmers reading code behind two doors, one with more cursing than the other, correlating code quality and cursing
Why would they do that!

May 24, 2023

Or, how to stop worrying and learn to love every API.

Read More →
tired hippo sleeping
Never miss API changes

May 17, 2023

Ensuring changes to external systems are never missed is a difficult problem best left to dedicated services.

Read More →