Adaptive Maintenance
Adaptive Maintenance is one policy that keeps an Iceberg table healthy. LakeOps measures how each table changes and runs the operations it needs, when it needs them, instead of on a fixed schedule per operation. It is the recommended way to run maintenance.
What it runs
Five operations, all on by default. When more than one is due, they run in this order:
| Operation | What it does | API flag |
|---|---|---|
| 1. Compaction | Merge small files toward the target file size. | compactionEnabled |
| 2. Expire snapshots | Remove snapshots older than the table's history.expire.max-snapshot-age-ms (default 5 days), keeping at least history.expire.min-snapshots-to-keep (default 1). | expireSnapshotsEnabled |
| 3. Rewrite position deletes | Rewrite position delete files to drop dead entries. | rewriteDeleteFilesEnabled |
| 4. Remove dangling deletes | Remove delete files that no longer apply to any live data file. | removeDanglingDeletesEnabled |
| 5. Rewrite manifests | Consolidate manifest files. | rewriteManifestsEnabled |
Orphan file cleanup is not part of Adaptive Maintenance. Run it with a Remove Orphan Files policy or on demand; see Orphan File Cleanup.
Automatic or Scheduled
Automatic (default)
An operation runs when the table's signals cross its threshold. There is no cron. A follow-on operation, such as a manifest rewrite after compaction, can run as a separate job once LakeOps re-measures the table.
Scheduled
A cron expression caps how often maintenance can run. On each run, only the operations that are due execute, in the order above. If none is due, the run is skipped. Nothing runs between scheduled times.
Use Scheduled to keep maintenance inside a window, for example 0 2 * * * for 2:00 every day.
The signals behind each decision
LakeOps samples every table from its metadata: the share of small files, bytes that could be compacted, snapshot and manifest counts, position, equality, and dangling delete files, and how fast compactable bytes and snapshots grow per hour. For each operation it keeps:
- •Needed or not, and a score for how close it is to its threshold.
- •Projected trigger time, from the current accumulation rate.
- •In progress, and a cooling-down time after a run, so the same operation does not repeat back to back.
See them in a table's Maintenance tab, from GET /api/tables/{catalog}/{namespace}/{table}/maintenance, or with the MCP tools get_maintenance_signals and get_table_profile.
Turn it on in the console
For one table
- Open the table (Data > Explore or Data > Tables) and go to the Optimization tab.
- In the Adaptive Maintenance section, pick Automatic or Scheduled (with a cron).
- Click Save. If the table already inherits a policy, the button reads Save table override, and Disable for this table excludes it instead.
While it is on, the Snapshot Retention, File Compaction, Orphan Files Cleanup, and Rewrite Manifests sections cannot be saved or toggled. Execute still runs any of them once.
For a catalog, namespaces, or many tables
- Go to Manage > Policies, click Create Policy, and choose Adaptive Maintenance.
- Set a name, the scope, and a priority.
- Pick Automatic or Scheduled (with a cron), then create the policy.
Tables in scope inherit the policy unless they have their own. See how policies apply.
With the REST API or MCP
The console always enables all five operations. Through the API you can turn individual ones off and choose how scheduled runs behave. This needs a key with policies:write.
curl -X POST https://api.lakeops.dev/api/policies/ADAPTIVE_MAINTENANCE \
-H "Authorization: Bearer $LAKEOPS_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Adaptive maintenance for sales",
"catalogName": "prod_glue",
"namespaces": ["sales"],
"executionType": "MANUAL",
"config": { "rewriteManifestsEnabled": false }
}'- •
executionType: "MANUAL"is Automatic mode (LakeOps decides when to run). Use"SCHEDULED"with acronExpressionfor Scheduled. - •
config: {}enables all five operations. Set a flag from the table above tofalseto skip one. - •For Scheduled,
scheduledBundleModeisAS_NEEDED(default, only due operations run) orFULL_WINDOW(every enabled operation runs on each tick). - •Agents can do the same with the MCP
create_policytool. See Manage the Lakehouse for example prompts.
Check that it is working
- •The table's Maintenance tab shows what is due, cooling down, or running.
- •The Events tab, or Data > Events for every table, lists each run with its status.
- •File distribution, excessive snapshot, and excessive manifest insights on Data > Insights clear as the table recovers.
