Governance
Declarative maintenance policies, GDPR-compliant retention, and per-operation audit trails — across every catalog and engine in your Iceberg lakehouse.
LakeOps| Operation | Table | Duration | Impact | Time | Status |
|---|---|---|---|---|---|
| Compact Data Files | customer_orders orders | 4s | 1.24 TB, 16 → 1 files | 57 minutes ago | SUCCESS |
| Expire Snapshots | payment_transactions payments | 27s | 8.2 TB | 4 hours ago | SUCCESS |
| Expire Snapshots | inventory_snapshots_20250702 warehouse | 3s | 2.1 TB | 4 hours ago | SUCCESS |
| Rewrite Manifests | raw_clickstream analytics | 1.9s | 3 → 1 manifests | 5 hours ago | SUCCESS |
| Compact Data Files | product_catalog products | 6m 11.3s | 3,008 → 1,256 files | 6 hours ago | SUCCESS |
Open and runs on your stack
How it works
Define governance at the organization level. Drill into each table's policy state, execution history, and retention status.
Organization-wide policies
The Policies dashboard lists every active and inactive policy with its type, scope, schedule, last run, and status toggle. Create, edit, enable, or disable any policy from one place — across every catalog and namespace in your lake.
Six policy types
Expire snapshots, remove orphan files, compact data files, rewrite manifests, rewrite position delete files, and rewrite equality delete files — each configurable with retention windows, age thresholds, target sizes, and merge parameters.
Audit trail
Every compaction, snapshot expiration, orphan removal, and manifest rewrite is logged with full context: duration, files before and after, bytes reclaimed, status, and the policy that triggered it. Filter by catalog, operation type, or time range.
Baseline retention, orphan cleanup
Daily compaction, 7-day retention
Sort compaction, hourly expiry
GDPR 30-day, 6h compaction
Table-level overrides take precedence. New tables inherit from their namespace automatically.
Policy inheritance
Set baseline policies at the organization level. Override at the catalog for specific environments. Fine-tune per namespace or individual table. New tables inherit governance rules automatically — no manual setup, no gaps.
Snapshot retention
Count max: 50
GDPR deletion
Full audit trail
Orphan cleanup
Safety window active
Pipeline: DELETE → compact → expire → orphan cleanup
Retention & compliance
Set snapshot retention periods, orphan cleanup schedules, and data deletion policies. LakeOps enforces the full GDPR pipeline — DELETE → compact → expire snapshots → remove orphan files — and logs every step for compliance evidence.
AWS Glue
1,240 tables
Polaris
860 tables
Nessie
430 tables
Gravitino
215 tables
One policy, all catalogs
Zero drift between governance and table state
Cross-catalog enforcement
Whether your tables live in AWS Glue, Apache Polaris, Project Nessie, Apache Gravitino, Lakekeeper, or S3 Tables — governance policies apply uniformly. No per-catalog scripts, no engine-specific maintenance logic.
Two governance layers
Catalogs like Polaris and Glue control who accesses data. LakeOps controls how data is maintained, retained, and audited — the layer the catalog doesn't cover.
Polaris, Glue IAM, and Nessie enforce role-based access at the catalog layer. LakeOps connects to all of them — governance policies apply alongside your existing access controls.
Short-lived, table-scoped storage tokens replace long-lived cloud keys. Catalogs issue credentials per request — LakeOps operates within the same security boundary.
The new REST catalog spec standardizes row filters and column masks at the catalog level. LakeOps complements access governance with operational governance — the layer the catalog doesn't cover.
Lakehouse Control Plane
Policies sit next to compaction, snapshot expiry, query routing, observability, and MCP access — same catalogs, same tables.
| Table | NS | Size | Status |
|---|---|---|---|
| customer_orders | orders | 1.24 TB | HEALTHY |
| payment_transactions | payments | 860 GB | WARNING |
| raw_clickstream | analytics | 4.6 TB | CRITICAL |
| product_catalog | products | 42 GB | HEALTHY |
| user_sessions | analytics | 1.9 TB | WARNING |
| inventory_levels | operations | 320 GB | HEALTHY |
| shipping_events | logistics | 580 GB | HEALTHY |
| search_query_logs | analytics | 3.2 TB | CRITICAL |
Learns from query patterns and table signals to decide what to optimize, when, and how — no schedules, no manual tuning.
The gap
Without declarative policies and continuous enforcement, table maintenance depends on scripts that drift, tribal knowledge that doesn't scale, and compliance evidence assembled after the fact.
Cron jobs and Spark scripts break on cluster upgrades, skip tables after schema changes, and nobody notices until storage costs spike or queries time out. Tribal knowledge replaces auditable control.
Catalog RBAC controls who reads and writes tables. It says nothing about when snapshots expire, how often compaction runs, or which orphan files get cleaned up. Operational governance is a separate layer entirely.
Auditors need proof that GDPR deletion requests were fully resolved — data deleted, compacted, expired, orphan files removed. Without per-operation audit trails, teams spend days assembling evidence from scattered logs.
Glue tables get one set of maintenance rules. Polaris tables get another. REST catalog tables get nothing. Governance fragments across catalogs instead of applying uniformly from one place.
Resources
LakeOps reads Iceberg metadata from every connected catalog, discovers tables, and applies governance rules automatically — no scripts, no pipeline changes.
Only metadata is processed — never retained or stored.
Telemetry reveals table health and actions needed.
Autopilot, manual approval, or policy-driven.
