A few years ago, picking a data platform was like picking a country to move to. You didn't just choose where your data lived — you adopted its language, its pricing model, its tooling, its roadmap. Leaving later meant more than a migration. It meant re-learning how to work.
That's quietly stopped being true, and most of the industry commentary is still describing it wrong.
The Real Change Isn't That Engines Got Better
Every major engine has spent this year shipping the same underlying idea in different packaging: your data shouldn't belong to us anymore. CDC support, Iceberg-native writes, dialect compatibility, catalog interoperability — strip away the branding and it's the same concession, made independently, by companies that used to compete on lock-in.
That's the headline most people are missing. It's not that any single engine got dramatically better this year. It's that engines independently agreed to stop being the whole platform. They're becoming components.
Components are interesting for a reason nobody likes to say out loud: components are replaceable. And a piece of infrastructure that's replaceable stops being a strategic bet and starts being a commodity.
Open Formats Don't Give You Freedom. They Give You the Option to Take It.
Here's where most of the "open lakehouse" narrative stalls out. Apache Iceberg means five different engines can technically read the same table. It does not mean your organization is actually able to use that fact.
In practice, teams that adopt an open table format still end up hardcoding a single engine into every application, every dashboard, every pipeline — because nothing is deciding, at query time, which engine a given workload should actually hit.
And the table itself doesn't stay usable for free. A streaming job checkpointing once a minute across a few hundred partitions can leave a table with well over 100,000 small files inside a quarter — each one a separate open and read that every downstream query pays for, on every engine, forever. Nobody's expiring snapshots, so a catalog that should hold a few weeks of history quietly holds a year, and every time-travel query wades through all of it to answer a question about last Tuesday. Manifests fragment until query planning — before a single row gets scanned — takes longer than the query itself used to take end to end. None of this trips an alert. It shows up as a dashboard that used to load in two seconds silently costing more and running slower every week, until someone finally asks why.
So the open format sits there, technically interoperable, practically unused — because interoperability isn't a property of storage. It's a property of what's operating the storage: continuous table maintenance, not a one-time migration to Iceberg.
A Scenario That Plays Out More Than People Admit
Picture a mid-size data team running Trino for scheduled jobs and starting to feel real pressure on interactive dashboard latency. The obvious fix is a faster engine for the interactive path — something like StarRocks. The less obvious problem is what that migration actually costs: every BI connection string gets touched, every downstream tool needs re-pointing, and someone has to stage a "flag day" where the old engine is retired and the new one takes over, hoping nothing breaks in between.
Now picture the same migration behind a routing layer. Traffic shifts from 10% to 100% on the new engine gradually, client connections never change, and if something regresses, traffic shifts back with no rollback plan required because nothing was ever cut over in the first place. Same destination. Completely different risk profile.
That gap — between "we could theoretically move workloads between engines" and "we actually do it, routinely, without a project plan" — is the whole ballgame. Most teams have the first. Almost none have the second.
Governance Doesn't Get to Take the Year Off
There's a version of this story that only talks about performance and cost, and it's incomplete. The minute you stop routing every query through a single vendor's walls, you also lose that vendor's built-in governance — the access controls, audit trails, and compliance guarantees that used to come bundled in whether you asked for them or not.
An interoperable stack has to replace that, not quietly drop it. Policies about who can touch what, and audit trails proving it, need to travel with the data itself rather than living inside whichever engine happens to be handling a given query this week. If governance doesn't survive the move to multiple engines, "open" just becomes a nicer word for "ungoverned," and that's not a trade any serious data team should be willing to make.
The Same Question Is Coming for AI Agents
There's a newer version of the engine-sprawl problem showing up right now: AI agents that want to query production data directly, in natural language, without an engineer in the loop writing the SQL. That's a genuinely useful capability, and it's also a genuinely uncomfortable one if "give an agent access to the lakehouse" means giving it the same unrestricted access a human engineer has.
The same control plane that decides which engine handles which query and which tables need maintenance is the natural place to answer that question too — scoped, guardrailed access for agents that behaves like a real permission boundary instead of an honor system. If autonomous agents are going to be a normal part of how data gets queried a year from now, that access needs to be governed from day one, not retrofitted after something goes wrong.
This Is the Gap We Built Two Things to Close
Strip away the vendor names and there are exactly two decisions a lakehouse has to make continuously, not once at migration time: which engine should handle this query right now, and what does this table need right now to stay fast. Almost nobody has software making either decision — they have a human who made it once, a year ago, and nobody's revisited it since. We built one tool for each.
The first decision is where should this query run right now. That's QueryFlux, a Rust proxy that speaks Trino, PostgreSQL, MySQL, and Arrow Flight on the client side, and dispatches to Trino, DuckDB, StarRocks, or whatever else sits behind it, based on rules you control. A dashboard doesn't need to know it's hitting StarRocks. A batch job doesn't need to know it's landing on Trino. The rules aren't a coin flip, either: route by workload shape — CPU-heavy joins to whichever pool is priced for compute, scan-heavy point lookups to something cheap and columnar like DuckDB, ad-hoc analyst queries to a group with headroom instead of the one dashboards depend on — and you encode the cost model once instead of re-deciding it every time someone opens a ticket about a slow chart. You get to change which engine handles which workload without anyone downstream noticing — because in our own testing, routing by workload shape rather than by default habit cut total query spend by up to 56%, with some individual queries costing 90% less simply by landing on the right engine instead of the usual one. It also absorbs pressure gracefully: when a cluster saturates, queries queue at the proxy instead of hammering the backend, with queue depth exposed as a real metric instead of a mystery.
The second decision is what does this table need right now to stay fast. That's LakeOps, which watches every table's file-size distribution, snapshot count, delete-file ratio, and the query patterns actually hitting it, then runs the maintenance — compaction, snapshot expiry, orphan cleanup, manifest rewrites — the moment a table crosses the threshold that would make queries slower or more expensive, not on a nightly cron that wastes compute on tables that are already healthy and misses the one that degraded at 2 a.m. It doesn't matter which engine wrote the last snapshot. LakeOps keeps the table ready for whichever engine reads the next one, and it does it with declarative policies and audit trails baked in rather than bolted on — so governance travels with the table across every engine, and controlled, guardrailed access is available for the AI agents that increasingly want to query it too. Teams running it have seen up to 80% cost reduction and 12x query acceleration — without moving a single byte of underlying data.
Neither tool asks you to pick a side. That's not an accident — it's the entire point.
"Isn't the Router Just a New Thing to Get Locked Into?"
It's a fair question, and it deserves a real answer rather than a marketing dodge. The honest answer is: a routing and maintenance layer only earns the right to sit above your engines if leaving it is easier than leaving an engine ever was. QueryFlux is open source under Apache-2.0 — the routing logic is config, not code, and it's yours to inspect, fork, or run yourself. LakeOps operates on your existing catalogs and storage without copying or migrating your data anywhere, which means turning it off doesn't strand anything — your tables are exactly as portable the day you stop using it as the day you started.
A control plane that can't say that isn't actually solving the lock-in problem. It's just moving it one layer up.
The Only Lock-In Left Is the One You Choose to Keep
Engines will keep turning over. Something will out-compete StarRocks. Something will out-compete Trino. That's not a threat to plan defensively against — it's just what a market looks like when it's actually competitive instead of walled off.
The mistake isn't picking the wrong engine today. The mistake is building your platform as though today's pick has to be permanent. Once routing, table maintenance, and governance are handled by something that sits above the engines rather than inside one of them, "which engine" stops being an architectural decision and becomes a Tuesday afternoon config change.
We're not trying to win the engine war. We think the engine war stops mattering once nobody has to fight it.
What's still locking your team into a single engine today — and is it actually a technical constraint, or just a decision nobody's revisited?
