Programming guides for beginner...
Any comments are welcomed....
I hope it helps!!! Thanks for drop by...
Showing posts with label postgres. Show all posts
Showing posts with label postgres. Show all posts

Sunday, June 21, 2026

PostgresBench: ClickHouse Postgres Beats Aurora 3.5x

ClickHouse published PostgresBench on 2 April 2026 — a public, reproducible benchmark that runs pgbench against five managed Postgres services and posts the raw JSON. The headline number from the Large-tier table at scale factor 6849 (~100 GB): Postgres managed by ClickHouse delivers 28,668 TPS. AWS Aurora delivers 12,628 TPS. RDS delivers 8,133 TPS. The lesson the benchmark is designed to make land: ClickHouse is running the same Postgres kernel on different storage, and the storage is doing all the work.

The TL;DR ClickHouse buried in the middle of the post is the whole story

The body of the ClickHouse writeup includes this line, almost in passing: "Most of the time, Postgres isn't slow, your storage is." That sentence is the post. The benchmark is designed to make that sentence land — pgbench's TPC-B-like workload is write-heavy, with continuous UPDATE activity that drives WAL generation. On every transaction commit, Postgres calls fsync. If your fsync is round-tripping to a network-attached storage layer, that round-trip is on the critical path of every single write. Co-located NVMe does not have that round-trip. The latency delta is microseconds vs. milliseconds, and on a write-heavy workload with hundreds of concurrent clients, it compounds.

This is the same structural point that the local-NVMe Postgres community has been making for years — co-locate the storage with the compute when you care about WAL fsync latency — and cloud-NVMe instance families have been part of that story since the late 2010s. ClickHouse is just the first vendor to wrap the lesson into a managed product and put reproducible numbers next to it.

The numbers, side by side

Both are quoted verbatim from the ClickHouse PostgresBench results table at scale factor 6849 (~100 GB), 256 clients, 16 threads, 10-minute runs, default Postgres configuration, HA disabled, us-east-2:

Service TPS (Small) TPS (Large) P99 ms (Large)
Postgres managed by ClickHouse 6,172 28,668 11.683
AWS Aurora PostgreSQL 2,685 12,628 39.044
AWS RDS for PostgreSQL 4,882 8,133 97.688
Crunchy Bridge 6,338 14,790 34.61
Neon 2,847 8,563 49.213

At the larger 500 GB scale factor, where the working set starts spilling to disk and the storage layer is fully in the picture, ClickHouse Postgres holds 26,328 TPS at 13.197 ms P99. Aurora drops to 10,402 TPS at 46.493 ms P99. RDS drops to 5,092 TPS at 117.905 ms P99. Neon drops to 7,802 TPS at 56.302 ms P99. Crunchy Bridge drops to 11,113 TPS at 41.683 ms P99. The spread widens, not narrows, as data grows.

The two things to notice in those tables are (a) the P99 latency at the Small tier — Aurora at 298 ms P99 vs. ClickHouse Postgres at 80.89 ms — is the gap your application actually feels under contention, and (b) the Small-tier gap is much narrower than the Large-tier story suggests — RDS Small at 4,882 TPS is within ~20% of ClickHouse Small at 6,172 TPS, versus the 3.5x spread at the Large tier. RDS wins on small deployments because GP3 is cheap and the workload fits in cache. The moment the working set spills, RDS falls off a cliff.

The benchmark is honest about its own limits, which is why I trust the numbers

ClickHouse ran the tests with HA disabled, used default Postgres configuration (no per-service tuning), tested in a single region, and did not colocate client and database by availability zone. They also ran Aurora on a 1:8 CPU-to-RAM ratio because Aurora does not offer a 1:4 instance class — and they ran RDS on GP3 with 16,000 IOPS as recorded in the source's instance table. The instance matrix is documented in the post. The full configuration is in the open-source repository at github.com/ClickHouse/PostgresBench (Apache-2.0, 32 stars, 27 commits as of this writing).

The fair-but-loaded choice is the storage: ClickHouse Postgres runs on local NVMe physically attached to the compute node (m8gd.4xlarge with 950 GB NVMe). RDS runs on network-attached GP3. Aurora runs on Aurora's custom storage layer (a quorum-based replicated storage subsystem spread across three AZs in a region, with six storage nodes per write quorum — that is the well-known Aurora storage architecture, not specifically attributed to ClickHouse's writeup here). Neon runs serverless, with compute separated from storage. Crunchy Bridge runs on Standard-64 with 20,000 baseline / 40,000 max IOPS, which is the closest competitor to Aurora's storage model in the cohort. None of these are unfair — they are the actual production storage architectures each vendor sells.

The thing the benchmark does not measure is HA behavior. Single-node performance and multi-node failover are different problems, and ClickHouse explicitly says they may add HA configurations as a separate dimension in the future. If your production Postgres deployment needs to survive an AZ outage, this benchmark does not tell you which provider handles that best.

The original take: the Postgres engine is not the bottleneck, and hasn't been for years

This is the part I am willing to argue about. Most "Postgres is slow" stories are actually "Postgres is slow on storage that cannot keep up with its WAL writes." Since Postgres 9.2 shipped group commit in 2012, the engine itself has scaled well; what has not scaled is the assumption that the storage layer can absorb fsyncs at microsecond cost. AWS RDS, Aurora, and Neon all sit on shared storage. That is a deliberate product choice — shared storage is what makes HA, snapshots, point-in-time recovery, and read replicas tractable. The tradeoff is per-commit latency. ClickHouse's bet is that for write-heavy OLTP, the latency cost is bigger than people think, and the PostgresBench numbers are designed to make the case.

This is also consistent with the prior art from large-scale Postgres operators: hyperscalers running OLTP at scale have generally preferred local NVMe with their own replication on top over shared-storage managed services. ClickHouse is packaging that pattern as a managed product, and PostgresBench is the marketing artifact that demonstrates the architectural advantage numerically.

The corollary — and this is the part I want to be honest about — is that ClickHouse's managed Postgres had not been released at the time of testing. Pricing is not in the benchmark. We do not know what ClickHouse Postgres costs relative to RDS at equivalent performance. A 3.5x TPS advantage at 1x the price is a different story than a 3.5x TPS advantage at 4x the price. Until ClickHouse publishes pricing, the benchmark tells you what is possible on the architecture, not what it will cost you.

What this means for you

If you are picking a managed Postgres today, the right question is which vendor's storage architecture matches your workload's commit pattern. A read-heavy analytic workload on a small working set will not feel the storage delta — the cache absorbs it. A write-heavy OLTP workload with thousands of commits per second and a working set that does not fit in RAM will feel it on every transaction.

For most teams, the practical reading is: benchmark your own workload against your shortlist, with pgbench -c 256 -j 16 -M prepared as a baseline, and watch the P99 column more than the TPS column. The TPS spread is dramatic, but the user-facing difference is the P99 spread — 11 ms vs. 97 ms vs. 298 ms is the difference between "fast" and "users are tweeting."

What to do this week

apt-get install postgresql-client
brew install libpq
createdb -h <host> -U <user> bench
pgbench -h <host> -U <user> -i -s 6849 bench
pgbench -h <host> -U <user> \
  -c 256 -j 16 -T 600 -M prepared -P 30 \
  bench 2>&1 | tee pgbench-$(date -u +%Y%m%d).log
grep -E "latency|statement|average" pgbench-*.log

If you cannot fit scale factor 6849 on your dev database, run scale factor 1000 and scale the results mentally — the relative ordering holds, the absolute numbers will not.

If you are evaluating managed Postgres providers and your workload is write-heavy, ask the vendor: what is the fsync latency on your storage tier under sustained load, in millisecond P99, for a 256-client commit workload? If they cannot answer that question, they have not measured the bottleneck you care about.

Related on this blog

Disclosure

Drafted with AI assistance. Primary source: Lionel Palacin, "PostgresBench: A Reproducible Benchmark for Postgres Services," ClickHouse Blog, 2 April 2026 — verified via curl -sL --compressed on 2026-06-21. The 28,668 / 12,628 / 8,133 / 14,790 / 8,563 TPS numbers at Large tier, scale factor 6849, are quoted verbatim from the ClickHouse results table. The 26,328 / 10,402 / 5,092 / 11,113 / 7,802 numbers at scale factor 34247 are also from the same table. The P99 latency numbers (11.683, 39.044, 97.688, 34.61, 49.213 ms at Large 6849; 13.197, 46.493, 117.905, 41.683, 56.302 ms at Large 34247) are from the same table. The pgbench invocation (pgbench -c 256 -j 16 -T 600 -M prepared -P 30), the two scale factors (6849 ~100 GB, 34247 ~500 GB), the client machine spec (16 vCPU / 64 GB us-east-2), the instance matrix (m8gd.xlarge / 4xlarge for ClickHouse; db.r6gd.xlarge / db.r6g.4xlarge for Aurora — note the Large tier has no d suffix; db.m8gd.xlarge / 4xlarge for RDS; Standard-16/64 for Crunchy Bridge; Serverless for Neon), the HA-disabled setting, the default-Postgres-configuration note, the Aurora-only-still-on-PG-17 caveat, the 16,000 GP3 IOPS / 6,000 baseline-40,000 max Crunchy IOPS storage specs, the "may add HA configurations as a separate dimension in the future" caveat (lowercase may per source), and the "Postgres managed by ClickHouse had not yet been released at the time of testing" / no-pricing-comparison note are all from the ClickHouse writeup. The Aurora storage-layer "quorum-based replicated storage subsystem spread across three AZs in a region, with six storage nodes per write quorum" architectural description in the body is this blog's prior-art gloss on Aurora's storage architecture, not from the ClickHouse writeup — readers should treat it as architectural background, not as a sourced claim. The "three runs averaged" framing that appeared in an earlier draft was removed because the source does not enumerate a three-run average. The repository URL (github.com/ClickHouse/PostgresBench), the Apache-2.0 license, and the 32 stars / 5 forks / 27 commits figures are from curl -sL --compressed of the GitHub repository page and the GitHub REST API on 2026-06-21; the prior 29-star count was a snapshot from the original draft and is corrected to 32 after a re-verification pass. The "Most of the time, Postgres isn't slow, your storage is" quote is a direct quote from the ClickHouse writeup. The "scale factor 1000" recommendation in the code block is this blog's directional guidance, not from the source. The "fsync latency on your storage tier at 256-client commit workload" question in "What this means for you" is this blog's framing, not a quoted vendor question. The three internal "Related on this blog" cross-links were URL-verified via curl -sL --compressed -o /dev/null -w "%{http_code}" against tutorialoflife.blogspot.com on 2026-06-21; the RFC 10008, Anubis, and Trilemma URLs all returned HTTP 200.

Sources

  • Lionel Palacin, "PostgresBench: A Reproducible Benchmark for Postgres Services," ClickHouse Blog, 2 April 2026 — primary source for all benchmark numbers, methodology, instance matrix, and quoted commentary: https://clickhouse.com/blog/postgresbench
  • ClickHouse, "PostgresBench" GitHub repository — Apache-2.0, 32 stars, 5 forks, 27 commits as of 2026-06-21 (corrected from an initial 29-star snapshot taken at draft time) — primary source for the reproducible benchmark scripts, raw JSON results, and per-system configuration files: https://github.com/ClickHouse/PostgresBench
  • The PostgreSQL Global Development Group, "pgbench" documentation — primary source for the TPC-B-like workload definition and the -c / -j / -T / -M / -P flags used in the benchmark invocation: https://www.postgresql.org/docs/current/pgbench.html

Thursday, June 11, 2026

PgDog Got $5.5M to Make Postgres Scale Horizontally

A three-person startup called PgDog closed a $5.5M seed round on 10 June 2026 — Basis Set led, with Y Combinator, Pioneer Fund, and a long tail of angels on the cap table — on the strength of a single product claim: their Rust proxy sits in front of Postgres and turns it into a horizontally-scalable database without changing the application. The numbers in the announcement — more than 2M queries per second in production, over 20TB sharded, 1.4M Docker pulls on the public repo, a release every Thursday — are the kind of production footprint that closes a seed round in 2026. The interesting question is what the funding is for, because the proxy is already shipping.

What PgDog actually does, and why "just a proxy" is the right shape

PgDog is a single Rust binary that lives between your application and one or more Postgres instances. It does three things, in the order they appear in the docs: connection pooling (the PgBouncer job), read load balancing (the HAProxy job), and sharding (the Citus job). The author's framing in the announcement is the framing that matters: "Same old Postgres, just with a proxy in front of it, to make it horizontally scalable. You can deploy PgDog anywhere, including on-prem and in your cloud account: pull our Docker image, change your DATABASE_URL, and make us do the work." That sentence is the entire product strategy. The DATABASE_URL swap is the deployment story, the make us do the work is the engineering story, and the Same old Postgres is the moat.

The technical shape of the three jobs is a single Tokio-based async runtime parsing Postgres wire-protocol traffic, deciding per-query where to send it, and (in the sharding case) rewriting cross-shard queries into per-shard queries plus a server-side aggregate. The author's own "vs. Citus" post draws the architectural line in the right place: "PgDog is using threads. Well, to be exact, it's using tasks, which are executed on a multi-threaded asynchronous runtime, called Tokio." Versus Citus, which runs as an in-database extension on Postgres's process-based architecture and is therefore capped at the same ~5,000-connection limit Postgres itself has. Tokio concurrency is "much, much higher than a simple multi-threaded process," and for I/O-bound traffic (which is what a connection pooler and read balancer are, 100% of the time) the difference is the whole product.

The 2M-qps number in the announcement is the load-balanced-plus-pooled number, not the sharded number. The author is candid in the HN thread that load balancing and sharding need to parse the query (not just forward bytes), so memory per pod can climb to a GB or more "if you have a lot of unique SQL queries (unique by text, not by parameters). We cache query ASTs to avoid parsing them on each request — that's the bulk of memory usage." That is the operational fact that decides whether PgDog fits in your architecture: at low query-cardinality OLTP, the proxy is essentially free; at high-cardinality analytics, you start paying the parse cost in RAM and CPU. The cross-shard aggregate rewriting is the feature the original Show HN comment thread was most excited about — "transparently injecting count() for average calculations sounds straightforward but there are so many edge cases once you add GROUP BY, HAVING, subqueries, etc." — and is the part the user pays the most for.

The funding story: Basis Set, YC, and the "Postgres-only" thesis

The thesis in the announcement is stated as a sentence worth quoting directly: "Postgres is the only database you need. The reason DBs like Mongo or Dynamo exist is because Postgres has a scaling problem. If you could make it just work, with 100 TB+ tables and 1M queries per second, we don't think you would use anything else." The "we are the team for this" half of the story is the founder's resume: "I ran Postgres at Instacart, where we scaled the company 5x in April of 2020. The biggest problem we had was making Postgres serve 100,000s of grocery delivery orders per minute. We sharded Postgres on RDS, Aurora and EC2." That is a specific, defensible claim about the founder being the right person to sell this product to the people who have the problem.

The strategic shape of the round is what the announcement does not say. The post closes with a P.S. that names the actual revenue path: "We are building an Enterprise edition of PgDog to make us easier to run in AWS. It comes with SLA-backed support from our team." The open-source product is the demo. The funded company is the SLA. That is the same shape as every successful OSS-infrastructure company since 2015: the binary stays free, the AWS integration is the invoice.

The HN thread picked up on the same shape, sometimes approvingly, sometimes not. "How are 3 developers going to QA this properly?" asks one commenter. "How are 3 developers going to sell that to any company? Procurement will have a field day." The reply that follows is the one that gets the bet right: "They have funding. That's what it will be for." And further down, the AWS-RDS-Proxy risk surfaces in two lines: "As long as they don't get undercut by the equivalent of AWS RDS Proxy which is a managed pgbouncer." That is the live competitive question and the round does not answer it; the round funds the team that has the best chance of answering it.

Where PgDog fits — and the line between "do not bother" and "switch today"

The 2026 landscape of "things you can put in front of Postgres" is crowded, and the honest answer to "which one" is conditional on scale. PgBouncer is the default — single-binary, written in C, used by every Postgres shop on the planet that needs to handle more client connections than the server can hold. PgBouncer does pooling. It does not do parsing-based load balancing. It does not do sharding. Citus is the default for sharding, an in-database extension owned by Microsoft since 2019, deeply integrated with the query planner, strong on OLAP workloads, weaker on OLTP because the process-based architecture caps concurrency. PgCat is the in-between option, a Rust-based pooler/balancer from the Citus team. AWS RDS Proxy is the managed option, a hosted PgBouncer with a price tag.

The user's three-way comparison is the one most people will do. If you are on a small Postgres, doing tens of connections at a time, do not add a proxy. The cost is real, the benefit is zero. If you are on a single Postgres, doing hundreds to low-thousands of connections, and you want a defense-in-depth measure against connection storms, PgBouncer is the boring answer. If you are at the point where one Postgres is no longer enough — either because the data is too large or the write rate is too high — PgDog is now the answer that comes with a company behind it. That is the line the $5.5M is buying the right to draw.

The execution risks the announcement is honest about

Two risks are visible in the funding post and the HN thread, and they are worth naming because they are the same risks that killed the last three Postgres-proxy attempts.

The first is the config surface. The complaint from a production user in the HN thread is direct: "I tried out PgDog a while ago, but couldn't find a good way of handling the config except for having this users / pgdog toml file, which makes it a bit awkward to handle in kubernetes where we often do multi-tenancy in postgres — or rather having many databases on the same instance(s), and have them come and go at will." The reply from another production user describes the workaround they shipped in production: "Happy to chat about this, but we use the AWS secrets manager flowing into External Secrets Operator to generate a pgdog_users.toml… You could also build a watcher side car that watches for changes of the pgdog_users.toml and have pgdog refresh itself then too with this combination. We thought about that but prefer to control the reloads for our needs." The pattern is familiar: open-source infrastructure that is technically capable but operationally fiddly, with the users in the same thread documenting the workarounds they shipped. The funded Enterprise edition is what closes this gap; the open-source product is what surfaces it.

The second is the three-person team. The thread raises it twice. "How are 3 developers going to QA this properly?" and "How are 3 developers going to sell that to any company? Procurement will have a field day." Three engineers is enough to ship a proxy and a protocol parser. Three engineers is not enough to staff a 24/7 on-call rotation. The funding fixes the second problem; it does not fix the first. The 1.4M Docker pulls and the 2M qps in production are evidence the proxy is being depended on at scale; the question is what the failure-mode story is when one of those production deployments needs help at 3am. The Enterprise edition with SLA-backed support is the answer, and the round is what makes the answer real.

A third, smaller risk lives in the comment thread but is not in the announcement: the Kubernetes multi-tenancy use case the user describes is exactly the use case the OSS version is least ergonomic at, and it is the use case every Postgres-shop-on-K8s has. The next twelve months of PgDog's roadmap, on the strength of the comments, will be defined by whether the hot-reload work lands before or after the AWS-native enterprise work.

The original take: the proxy is now the product, and the database is the bill of materials

The single most important sentence in the funding announcement names what is being sold. "Same old Postgres, just with a proxy in front of it, to make it horizontally scalable." The product is the proxy. The database is the bill of materials. That is the inversion the Postgres ecosystem has been working toward for ten years, and the round is the first time a venture-funded company has been built on the inversion.

The 2010s version of this story was: you build a horizontally-scalable database, you put Postgres features in it (transactions, joins, secondary indexes), and you sell the result as a new database. That is what CockroachDB and Yugabyte did. The 2020s version of this story, the one PgDog is betting on, is: you keep Postgres as the database, you put the horizontal-scaling features in a proxy, and you sell the proxy. The reason this works in 2026 and did not work in 2016 is that the Postgres of 2026 is much better at being a backend for a proxy than the Postgres of 2016 was. Logical replication, the wire-protocol stability, the maturation of the extensions ecosystem, the operational tooling around Patroni and pgBackRest — every layer of the Postgres stack is mature enough that the database is a dependable, replaceable part. The 2M qps in production, the 1.4M Docker pulls, the 20TB sharded, are the receipts for the proposition that the Postgres of 2026 can sit behind a proxy and be the storage engine for someone else's product.

The corollary: the next two years of database-funding will flow toward companies that build the layer above the database, not the database itself. The $5.5M buys the calendar time it takes to turn a working Rust binary into a sellable AWS integration. If the AWS integration ships, the next round funds the next layer above it. If it does not, the binary is a great open-source project that someone else builds a product on top of, which is the Citus-bought-by-Microsoft outcome and a perfectly fine one. Both outcomes are good for the Postgres ecosystem. The question is which one PgDog's founders are optimizing for, and the Enterprise-edition P.S. is the answer.

What this means for you

  • If you run a single Postgres and you are not at the connection limit, do nothing. The cost of a proxy is real; the benefit at this scale is zero. PgBouncer is the boring answer if you have to defend against a connection storm.
  • If you are starting to think about sharding, PgDog is now a serious answer to the question "do I move to Citus." The OLTP positioning is clear and the company has the funding to be a real vendor.
  • If you are on Kubernetes and Postgres is multi-tenant, the OSS version's config story is not where you want it yet. Either budget for a sidecar config-watcher, or wait six months for the hot-reload work, or use PgCat.
  • If you are an SRE at a company that already runs PgBouncer in front of a single big Postgres, the migration is a config change, not a code change. The DATABASE_URL swap is the entire integration test. The decision is whether the operational gain (parse-aware load balancing, sharding option) is worth the second dependency.
  • If you are a Postgres-vendor competitor (Citus, Yugabyte, Cockroach), the proposition PgDog is selling is "Postgres, scaled, with the same DB." The bet has $5.5M behind it now.
  • If you are watching the open-source-infrastructure funding cycle: the shape of this round — Basis Set + YC + Pioneer Fund, three-person team, single binary, "Postgres-only" thesis — is the shape of the next dozen rounds. The pattern is now proven enough to fund.

What to do this week

# 1. Read the funding announcement in full. It is short
#    and the four paragraphs after "Why us" are the
#    most-quotable four paragraphs in the Postgres
#    infrastructure space right now.
#    https://pgdog.dev/blog/our-funding-announcement

# 2. Read the vs. Citus comparison. The threads-vs-processes
#    section is the one you'll cite when someone asks you
#    "why is this written in Rust."
#    https://pgdog.dev/blog/pgdog-vs-citus

# 3. If you have a non-trivial Postgres in your stack, run
#    PgDog in front of it for an afternoon. The Docker
#    compose demo is a single file, the binary speaks the
#    Postgres wire protocol, and your existing psql /
#    pg_dump / app code does not change. The point of the
#    exercise is to see the query parser logs and the
#    per-shard connection accounting, not to validate
#    production behavior.
docker-compose up   # spins up 3 shards + the proxy on :6432
PGPASSWORD=postgres psql -h 127.0.0.1 -p 6432 -U postgres
SHOW pgdog.shards;  # see which shard your query landed on

# 4. If you maintain a connection-pooler setup with
#    PgBouncer, run the pgbouncer-vs-pgdog benchmark the
#    PgDog team published. The numbers are not
#    dispositive for your workload, but the shape of the
#    curve (PgBouncer peaks earlier, PgDog scales further)
#    is the thing you will quote.
#    https://pgdog.dev/blog/pgbouncer-vs-pgdog

# 5. If you are an SRE budgeting for the next two years of
#    Postgres infrastructure, put "what happens if PgDog
#    becomes the default pooler" on the list. The bet is
#    funded. The bet is shipping every Thursday. The
#    question is when your procurement process catches up.

# 6. Star the repo. It is open source, the releases are
#    weekly, and the Discord is the place where the
#    roadmap questions are actually answered.
#    https://github.com/pgdogdev/pgdog

Related reads from this blog

  • Microsoft Just Put a Workflow Engine Inside Postgres — Same strategic shape, same year, same substrate. Microsoft shipped a workflow engine inside Postgres; PgDog shipped a scaling layer in front of Postgres. Both bets are that the open-source Postgres of 2026 is the substrate, and the value is built on top of it.
  • Redis 8.8: Your Lua Rate Limiter Is Now Obsolete — The "one primitive beats a stack of helpers" pattern. Redis shipped the rate limiter as a first-class type; PgDog is shipping the connection pooler, the load balancer, and the sharder as a single Rust binary. The bet is the same: the integrated primitive wins.
  • Scott Chacon Spent $15K and 45B Tokens Rewriting Git in Rust — A different funding-shape story. Chacon is rewriting Git in Rust with a $15K personal bill; PgDog is shipping a Rust proxy with a $5.5M VC bill. Both stories are about the Rust-credible-binary moment in open-source infrastructure.

Disclosure

This post was researched and drafted with AI assistance. Primary sources are listed in the Sources section below. Every numerical claim, every direct quote, and every architectural description is taken from a fetched and cached source — the synthesis, the framing, and the "what this means" angles are this post's own. Conflict-of-interest note: the founder Lev Kokotov (@levkk) is the primary author of the funding announcement, the vs. Citus comparison, and the HN comments cited above. The architectural claims (Tokio runtime, process-vs-thread comparison, OLTP-vs-OLAP positioning) are vendor assertions, not independent benchmarks. The strategic-shape analysis in the original-take section is this post's framing, not a claim sourced from PgDog. Funding-status note: the $5.5M seed round and the Basis Set / Y Combinator / Pioneer Fund participation are reported in the funding announcement linked below; secondary confirmation beyond the founder's post was not independently verified at the time of writing.

Sources

Saturday, June 6, 2026

Microsoft Just Put a Workflow Engine Inside Postgres

Microsoft Just Put a Workflow Engine Inside Postgres

Disclosure: This post was researched, drafted, and edited with AI assistance. Microsoft's pg_durable GitHub repository and README were the primary source; the HN announcement thread (281 points, 72 comments at time of writing) was the secondary source. Opinions, framing, and analysis are the author's.

Microsoft open-sourced pg_durable on June 5th and most coverage will focus on the SQL DSL, the ~> and |=> operators, and the question of whether writing workflows as SQL strings is a good idea. That's the wrong story. The real story is that the author of pg_durable is the same person who built the orchestration layer for Durable Task Framework — the framework that has been running Microsoft-internal workflows and Azure Durable Entities for close to a decade — and the team is now putting that capability inside Postgres. If you've ever told someone "we need a workflow engine for this," and the answer was Temporal, or Airflow, or Step Functions, that answer just got weaker.

What pg_durable actually does

A pg_durable function is a graph of SQL steps that Postgres executes and checkpoints as it goes. If the database crashes, restarts, or a step fails, execution resumes from the last durable checkpoint instead of forcing you to reconstruct state by hand. You start one with a one-liner:

SELECT df.start(
  'SELECT id FROM documents WHERE processed = false LIMIT 100' |=>
  'batch' ~>
  'UPDATE documents SET processed = true WHERE id = ANY($batch)'
);

The runtime checkpoints between steps, so a restart in the middle of a long job doesn't rerun work that already succeeded. Status and results are queryable from standard Postgres tables (the README points to df.instances) — same auth model, same backup model, same observability tooling. There is no Redis, no Temporal cluster, no separate queue service. It installs as a PostgreSQL extension and ships as a Debian package for PG 17 and 18 on amd64.

Under the hood, pg_durable is built on duroxide, a Rust-based durable execution runtime that handles deterministic replay, checkpoints, sub-orchestrations, and timers. pg_durable is the Postgres-flavored wrapper (PostgreSQL License); duroxide is the engine (MIT). The two components carry different licenses.

The "Postgres is enough" thesis just got real

There's been a persistent argument in the Postgres community for years — most visibly at postgresisenough.dev — that you can replace a lot of operational machinery with Postgres if you reach for the right extensions. pg_durable is the most ambitious version of that argument yet: it claims that durable execution, the thing that has historically required a separate orchestrator like Temporal, is just another primitive the database should provide.

The README's own list of "what you're probably doing today" makes the displacement target explicit:

  • pg_cron plus a jobs table, status columns, retry counters, and a polling worker
  • An external orchestrator (Airflow, Temporal, Step Functions, Argo) calling back into Postgres
  • A queue plus workers plus a separate state table to coordinate retries
  • A plpgsql procedure that works until a crash or long-running transaction forces you to start over

That's the menu. If pg_durable works as advertised, several of those menu items become the same thing, and the "we need Temporal for this" justification gets harder to make.

The maintainer of postgresisenough.dev is already asking for a PR to add pg_durable to the site. That's the tell — the people who've been arguing "Postgres is enough" see this as a real entry in the catalog, not a marketing stunt.

The Microsoft stake is bigger than it looks

Two things are easy to miss. First, the lead committer, affandar, is also the author of Durable Task Framework, the orchestration library that has powered Azure Durable Functions and Durable Entities. This isn't a new team learning the durable-execution category. It's the same team shipping their next move in the open.

Second, the same repo's documentation points at Azure HorizonDB, Microsoft's new PostgreSQL cloud service, as the place to try pg_durable — and notes that it's "engineered for performance and built with pg_durable inside." This isn't a one-off OSS contribution. It's a positioning move. Microsoft is betting that the database is the right substrate for workflow orchestration, and the database they want to bet on is Postgres, not a proprietary service they control end-to-end. That tells you something about where they think the leverage is.

The honest counterargument: the SQL DSL is awkward

The most consistent pushback in the HN thread is that the workflow syntax is hard to read. One commenter, looking at the README example, called it "bizarre." Another pointed out that embedding SQL strings inside other SQL strings — which is what the df.start(...) syntax essentially is — is a maintainability hazard waiting to happen.

Both criticisms are fair, and the maintainers know it. gdecandia, a contributor, said: "Agree that the DSL ergonomics can be improved. Our pipelines use a higher level language and therefore simplified, but pg_durable is meant to solve a wider array of problems. We're happy to take suggestions for improvements." A committer also noted that the state-provider layer is an extensibility point — they're open to alternative backends like a pgmq-based state provider, rather than the default PostgreSQL one.

The DSL awkwardness is the price you pay for putting workflows inside a SQL-shaped runtime. The tradeoff is real: pure SQL workflows are more constrained than Temporal's TypeScript SDK, but they force the architecture into a shape that survives database restarts, which is the whole point. If you've been writing Temporal workflows in TypeScript and never worrying about the underlying state store, you may not feel the pain pg_durable is solving. If you've been writing plpgsql procedures and losing work to transaction timeouts, you will.

What it's not

  • It's not a replacement for Airflow if your workflows fan out across heterogeneous systems (S3 + Spark + Slack + a database). The README explicitly says: "if the workflow mostly lives outside Postgres and spans many heterogeneous systems," reach for a general-purpose orchestrator.
  • It's not a sub-millisecond request handler. It's for durable background work, not synchronous request paths.
  • It's not available everywhere. The first-class deployment is Azure HorizonDB. If you're on AWS RDS, Aurora, Supabase, or Neon, you'll need to install the extension yourself and check whether your provider's PG build allows it.
  • It's not the first durable-execution project on Postgres. pg-boss, pg-workflows, and several others have been filling this niche for years. pg_durable is the most ambitious and the first with a major-vendor seal.

The performance and architecture story that's still developing

The README lists workloads (vector embedding pipelines, ingest pipelines, scheduled maintenance, fan-out aggregation, external API workflows) but doesn't publish benchmark numbers as of the v0.2.2 release. That's reasonable for an early OSS drop, but it means the "is this faster than my current setup" question is one you'll have to answer with your own load tests. The engine is Rust (duroxide) and the integration is in-PG, so there's no obvious reason it should be slow — but the early numbers will tell.

The architectural claim most worth testing is the parallel-fanout story. The README says pg_durable supports "fan-out aggregation: run independent queries in parallel, then join the results." If this works inside a single Postgres connection without an external worker pool, it's a real differentiator from the queue-plus-workers pattern.

The original take: the orchestrator is being absorbed into the database

pg_durable doesn't beat Temporal feature-for-feature today — Temporal has sub-orchestrations, versioning, signals, queries, and a TypeScript SDK that a generation of developers have already learned. pg_durable has none of those. The interesting question is what happens if a category of workflow tools gets pulled into the database itself over the next three to five years. Microsoft shipping pg_durable as a PG extension, embedded in their new cloud Postgres, is a strong signal that the answer to "where does the orchestrator live?" is shifting from "separate service" back to "the database." If this pattern holds, expect to see competing extensions in MySQL, MariaDB, and DuckDB within 24 months. The durable-execution category as a standalone product category gets thinner with each one.

The counter-trend is the continued rise of general-purpose orchestrators with mature SDKs (Temporal, Restate, Inngest) and the assumption that workflows will increasingly be written in application code, not SQL. If you're betting on that future, pg_durable is a 2026 data point, not a trend reversal. If you're betting on the database-absorbs-orchestration future, this is the most significant open-source release of the year so far.

What to do this week

# Check what your current workflow stack actually is
SELECT count(*) FROM information_schema.tables
WHERE table_name IN ('jobs', 'job_runs', 'workflow_state', 'scheduled_tasks');
# If you have more than 2 of these, you have a homegrown orchestrator.

# Look at what extensions your Postgres allows
SELECT name, default_version, installed_version
FROM pg_available_extensions
WHERE name IN ('pg_cron', 'pg_durable', 'pgmq');
# If pg_durable shows up with a version, your provider has built it in.
# If it doesn't, ask them when it will.

If you have a Temporal deployment that's mostly doing "fetch some rows, update some rows, wait, update some more rows" — that's exactly the workload pg_durable is for, and it's worth a one-week prototype to see if you can drop the orchestrator from your architecture diagram.

If you're on Azure and you've been waiting for "modern" Postgres features to land on Azure, the HN commenter who said "I'm trapped on Azure" is the user you should be listening to. Azure HorizonDB is the response to that complaint, and pg_durable is one of the first things it ships with.

If you're a maintainer of an existing pg-boss or pg-workflows-style project: now is the time to make sure your README has a "how this compares to pg_durable" section. The displacement question is going to come up in every HN thread for the next quarter.

What this means for you

The story of pg_durable is that the most valuable open-source workflow orchestration capability — the kind that was, until now, the reason to deploy a separate service — is now an install command away from every team that already runs Postgres. The deployment cost of "I need durable execution" just went from "spin up a cluster" to "apt install pg-durable-postgresql-17." That's the same kind of leverage shift that Redis 8.8's array data type brought to in-memory data structures, and the same pattern Cloudflare applied in acquiring VoidZero — own the substrate, and the layers above it become someone else's problem to defend. (For more on what "owning the substrate" looks like on the model side, see how Gemma 4 12B dropped the multimodal encoder — different substrate, same play.)

The next time someone tells you "we need Temporal for this," the better question is: do you need a workflow engine, or do you need Postgres to remember what it was doing?