Organization-Wide Policies

Why use policies?

Configuring optimization settings table-by-table works for a handful of tables, but doesn't scale. Policies let you define optimization rules once and apply them across hundreds or thousands of tables automatically.

• •Consistency — every table gets the same optimization standards without manual setup
• •Scale — onboard new tables automatically as they inherit catalog or namespace-wide policies
• •Governance — every policy change is versioned and auditable with full history
• •Control — enable or disable any policy instantly with a single toggle

How policies work

A policy is a named rule that applies a specific optimization operation to a set of tables on a schedule. Each policy has:

• •A name and optional description
• •A type (Adaptive Maintenance, Expire Snapshots, Remove Orphan Files, Rewrite Manifests, or Configuration)
• •A scope (which tables it applies to)
• •An execution schedule — cron expression, fixed interval, or manual-only
• •An enable/disable toggle for instant control
• •Type-specific settings (retention period, minimum snapshots, age threshold, etc.)

When enabled, LakeOps executes the policy on the configured cron schedule. You can also trigger any policy manually at any time.

Policy types

LakeOps supports the following policy types. When creating a new maintenance policy, you choose the operation type from the selection screen:

Policy scope

Policies can be scoped at different levels of your data hierarchy:

Precedence rules

More specific policies override broader ones. A per-table policy always takes precedence over a namespace or catalog-wide policy for the same operation type. This lets you set sensible defaults at the catalog level and override only where needed.

Global Policies screen

Navigate to Manage > Policies in the sidebar to access the central policy management screen. This is where you create, search, filter, and manage all policies across your organization.

Screen layout

Policy table columns

Creating a policy

Per-table policy assignment

You can also view and manage policies from the perspective of a single table. Navigate to Data > Explore, select a table, then open the Policies tab.

What you see

The per-table Policies tab shows all policies currently assigned to the selected table, including inherited policies from namespace, catalog, or organization scope. Each row shows:

Assigning a policy

Click + Assign Policy to link an existing policy to this table. You can assign multiple policies of different types to the same table (e.g. one compaction policy plus one snapshot expiration policy).

Example: typical policy set

A production table typically has multiple policies covering different optimization operations:

Adaptive Maintenance handles compaction and other operations data-driven. Manifest rewrites run daily for file layout. Snapshot expiration runs hourly to prevent metadata bloat. Orphan cleanup runs daily at the catalog level to catch stragglers.

Policies vs. per-table Optimization tab

Both policies and the per-table Optimization tab configure the same underlying operations. The difference is scope and management:

A common pattern: set catalog-wide policies for baseline hygiene, then use the per-table Optimization tab to override settings on tables that need special treatment.

Scheduling best practices

Policies use cron expressions to control execution timing. Consider these guidelines:

• •Stagger schedules — avoid running compaction, snapshot expiration, and manifest rewrites at the same time. Spread them across different hours.
• •Run during low-traffic windows — schedule heavy operations like compaction during off-peak hours to minimize impact on query workloads.
• •Order matters — run snapshot expiration before orphan cleanup so that expired data files become detectable as orphans.
• •Start conservative — begin with longer intervals and tighten as you observe results.

Recommended schedule

Policy inheritance

When a policy is scoped at the catalog or namespace level, tables within that scope automatically inherit the policy. The per-table Optimization tab shows inherited policies with an Inherited badge.

• •Inherited policies — shown with an amber badge indicating the source policy name and scope type. Clicking the policy name navigates to the governance policy.
• •Table-level overrides — saving a table-level configuration creates a local policy that takes precedence over the inherited one. Removing the table policy reverts to the governance policy.
• •Excluding a table — toggle off an inherited policy to exclude the specific table from the governance policy without affecting other tables.

When Adaptive Maintenance is inherited, it locks all individual operation sections in the Optimization tab for that table. The notice indicates which governance policy manages the table.

Auditing & versioning

Every policy change is tracked with full audit history:

• •Version history — see when a policy was created, modified, enabled, or disabled, and by whom
• •Execution log — every policy run is recorded in the Events tab for each affected table, showing the operation performed, duration, and impact
• •Updated timestamp — the global Policies table shows when each policy was last modified

Monitoring policy execution

Track policy health and impact through:

• •Global Policies screen — Next Run and Last Run columns show scheduling health at a glance
• •Events tab (per-table) — detailed log of every operation executed by the policy, including before/after metrics
• •Insights tab (per-table) — warnings that should resolve after policy-driven optimizations take effect
• •Dashboard — aggregated operations count and cost savings reflect the cumulative impact of your policies