Engine Management

LakeOps provides a unified view of all query engines connected to your Iceberg lake. Monitor health, compare performance, and add new engines — without engine-specific scripts or duplicate tooling.

Why unified engine management?

Modern data lakes connect multiple query engines for different workloads: Trino for interactive analytics, AWS Athena for serverless queries, Snowflake for BI dashboards, DuckDB for local development. Without a unified view, teams end up with fragmented monitoring, inconsistent configurations, and no way to compare engines objectively.

LakeOps solves this by providing a single control plane where you can see every engine's status, usage, cost, and health — and take action from one place.

Supported engines

LakeOps supports the following query engines out of the box:

AWS Athena
Serverless query service
Trino
Distributed SQL query engine
Snowflake
Cloud data platform
DuckDB
In-process analytical database
StarRocks
Real-time analytics engine
ClickHouse
Column-oriented DBMS
Apache Sparkbeta
Unified analytics engine
Apache Flinkbeta
Stream processing framework
Databricksbeta
Lakehouse platform
Dremiobeta
Lakehouse engine

Additional engines can be connected via the Add Engine wizard with custom connection parameters.

Query engines screen

Navigate to Engines > Overview in the sidebar. The screen provides a complete view of your engine fleet.

Summary cards

Four stat cards at the top show fleet-level metrics:

Registered
Total engine connections
Active
Ready for queries
Maintenance
Not available
Queries (24h)
Routed / custom

Quick links

Below the stat cards, three action tiles provide shortcuts:

  • Compare engines — side-by-side cost, latency, and volume comparison
  • Engine health — uptime, incidents, and resource signals per engine
  • Add engine — guided wizard for credentials and connection tests

Engine directory

The main area shows engine cards in a grid layout. Each card displays:

FieldDescription
Engine nameName and type with brand icon
StatusActive (green), Inactive (gray), or Maintenance (amber)
QueriesTotal queries routed to this engine
Avg runtimeAverage query execution time
Cost / queryAverage cost per query execution
Last usedHow recently the engine processed a query

Use the search bar and status filter to find specific engines quickly.

Engines Compare

Navigate to Engines > Compare to benchmark engines side by side. The compare view shows:

  • Query success rate — percentage of queries completed without errors
  • Average runtime — mean query execution time per engine
  • Cost per query — average execution cost for direct comparison
  • Total queries — query volume processed per engine
  • Data scanned — total data read during query execution

Toggle engines from the engine tiles to populate the comparison table. A cost vs. performance visualization helps identify the most efficient engine for each workload.

Engine Health

Navigate to Engines > Health for real-time health monitoring. The health view shows each engine with:

  • Status indicator — Healthy, Inactive, or Maintenance
  • Uptime percentage — availability over the monitoring period
  • Response time — current average latency per engine
  • Resource utilization — CPU, Memory, Disk I/O, and Network usage bars per engine
  • CHECK NOW button — trigger an immediate health check

Adding a new engine

Navigate to Engines > Add Engine in the sidebar. The guided wizard walks you through four steps:

1Select Engine — choose from the supported engine grid (AWS Athena, Trino, DuckDB, StarRocks, Snowflake, ClickHouse, and more).
2Configure — provide host, port, authentication credentials, and any engine-specific connection parameters.
3Test Connection — LakeOps validates the connection parameters and verifies the engine is reachable.
4Finish — review the configuration and complete setup. LakeOps begins collecting telemetry immediately and the engine appears in the directory.

Engine statuses

StatusMeaningRouting behavior
ActiveEngine is connected, healthy, and accepting queriesEligible for query routing
InactiveEngine is registered but not currently connectedExcluded from routing until reactivated
MaintenanceEngine is temporarily unavailable (planned downtime)Excluded from routing; failover triggers for in-flight queries

Engines & routing

Engines are the foundation of query routing. Each routing group references one or more engines. When you add or remove an engine, routing groups that reference it are updated automatically. If an engine enters Maintenance or Inactive status, the routing layer excludes it and fails over to remaining engines in the group.