Connect a Catalog
LakeOps works on the Iceberg catalogs you already run. You register a catalog once, LakeOps discovers its namespaces and tables, and health analysis starts. Your tables stay where they are.
Before you start
- •You are signed in to the console in the data region where you want this catalog to live.
- •You have an AWS identity for LakeOps, access keys or an IAM role, with the permissions in Credentials and permissions.
- •For REST catalogs, the catalog URI is reachable from LakeOps over the network. The template's
localhostvalues are placeholders.
Add a catalog in the console
- Go to Data > Catalogs and click Add Catalog.
- Pick a template. It sets the catalog type and pre-fills the properties.
- Fill in the form (below) and replace every placeholder property value with your own.
- Click Create Catalog. LakeOps builds the Iceberg catalog from your settings first; if that fails, nothing is saved and the error explains why.
- On success, LakeOps runs a first discovery scan. Tables then appear under Data > Explore and Data > Tables.
Form fields
| Field | Notes |
|---|---|
| Catalog name | Required. No spaces. This is the name you use in the REST API and MCP tools. |
| Description | Optional. |
| Catalog type | Set by the template. Read-only. |
| URI | The catalog endpoint. Not used for Glue. |
| Warehouse | Where table data lives: an S3 path, or the table bucket ARN for S3 Tables. |
| Properties | Key/value Iceberg catalog properties, passed to the catalog as-is. Add rows with Add property. |
Catalog templates
Templates pre-fill the most common setups, and you can edit, remove, or add any property before you save. Each template except Custom also sets type and Iceberg's S3 file IO (io-impl = org.apache.iceberg.aws.s3.S3FileIO). The cards list the other properties it pre-fills.
Glue + S3
type: glueThe AWS Glue Data Catalog, with tables on Amazon S3.
- URI
- Leave empty
- Warehouse
- s3://your-bucket/path
- Properties
- client.region, client.access-key-id, client.secret-access-key
AlphaAccess keys in the client.access-key-id and client.secret-access-key properties aren't applied to Glue yet. For now, our team connects Glue catalogs with an IAM role that LakeOps assumes.
REST + S3
type: restAny catalog that serves the Apache Iceberg REST API, such as Apache Polaris or Project Nessie, with tables on Amazon S3 or S3-compatible storage.
- URI
- Your REST catalog endpoint
- Warehouse
- s3://your-bucket/path
- Properties
- s3.endpoint, s3.access-key-id, s3.secret-access-key, s3.region, s3.path-style-access
S3 TablesAlpha
type: s3Amazon S3 Tables, through its Iceberg REST endpoint.
- URI
- https://s3tables.<region>.amazonaws.com/iceberg
- Warehouse
- arn:aws:s3tables:<region>:<account-id>:bucket/<table-bucket>
- Properties
- s3.access-key-id, s3.secret-access-key, s3.region, s3.path-style-access, rest.sigv4-enabled = true, rest.signing-name = s3tables, rest.signing-region, rest.access-key-id, rest.secret-access-key, client.region
Not supported end to end yet. Talk to our team before you rely on it.
DynamoDB + S3Alpha
type: dynamodbIceberg's DynamoDB catalog, with tables on Amazon S3.
- URI
- dynamodb://<table-name>
- Warehouse
- s3://your-bucket/path
- Properties
- table-name, s3.endpoint, s3.access-key-id, s3.secret-access-key, s3.region, dynamodb.access-key-id, dynamodb.secret-access-key, dynamodb.region
Not supported end to end yet. Talk to our team before you rely on it.
CustomAlpha
type: customStart from an empty form and set every property yourself.
- URI
- If your catalog needs one
- Warehouse
- Your warehouse location
- Properties
- The properties you add
Not supported end to end yet. Talk to our team before you rely on it.
Credentials and permissions
LakeOps reaches your catalog and storage with an AWS identity you control: access keys for a dedicated IAM user, or an IAM role in your account that LakeOps assumes. Whichever you use, scope it to the catalog and the bucket where your tables live. For Glue, access keys are in alpha Alpha, so use an IAM role. Our team can help you pick and set it up.
How credentials are handled
LakeOps stores the access keys with the catalog configuration and uses them only to run maintenance. Anyone in your organization who can read catalogs, in the console or with an API key that has catalog read access, can view them. Use a dedicated, scoped identity and rotate its keys when a member leaves.
What each permission is for
| LakeOps needs to | AWS actions (Glue + S3) |
|---|---|
| Discover and analyze | glue:GetDatabases, glue:GetDatabase, glue:GetTables, glue:GetTable, s3:ListBucket, s3:GetBucketLocation, s3:GetObject |
| Commit maintenance results | glue:UpdateTable, s3:PutObject, s3:AbortMultipartUpload |
| Expire snapshots and remove orphan files | s3:DeleteObject |
| Rename or delete a table from LakeOps | glue:CreateTable, glue:DeleteTable |
Sample IAM policy for Glue + S3
Replace the region (us-east-1), account ID (123456789012), and bucket (my-warehouse-bucket) with your own.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ReadAndCommitGlueTables",
"Effect": "Allow",
"Action": [
"glue:GetDatabases",
"glue:GetDatabase",
"glue:GetTables",
"glue:GetTable",
"glue:UpdateTable"
],
"Resource": [
"arn:aws:glue:us-east-1:123456789012:catalog",
"arn:aws:glue:us-east-1:123456789012:database/*",
"arn:aws:glue:us-east-1:123456789012:table/*/*"
]
},
{
"Sid": "RenameAndDeleteGlueTables",
"Effect": "Allow",
"Action": ["glue:CreateTable", "glue:DeleteTable"],
"Resource": [
"arn:aws:glue:us-east-1:123456789012:catalog",
"arn:aws:glue:us-east-1:123456789012:database/*",
"arn:aws:glue:us-east-1:123456789012:table/*/*"
]
},
{
"Sid": "ListWarehouseBucket",
"Effect": "Allow",
"Action": ["s3:ListBucket", "s3:GetBucketLocation"],
"Resource": "arn:aws:s3:::my-warehouse-bucket"
},
{
"Sid": "ReadWriteDeleteWarehouseFiles",
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:AbortMultipartUpload"],
"Resource": "arn:aws:s3:::my-warehouse-bucket/*"
}
]
}- •Narrow it. Replace
database/*andtable/*/*with your database and table names, andmy-warehouse-bucket/*with your table location prefix. Add a bucket for each location your tables use. - •Start read-only. Keep only the Get and List actions and
s3:GetObject. Discovery and health analysis work; maintenance fails until you add the write and delete actions. - •Optional statement. Drop
RenameAndDeleteGlueTablesif you will not rename or delete tables from LakeOps. - •Encrypted buckets. With SSE-KMS, also allow
kms:Decryptandkms:GenerateDataKeyon the key. - •Lake Formation. If Lake Formation governs the catalog, also allow
lakeformation:GetDataAccessand grant the identity access to the databases and tables in Lake Formation. - •Other catalogs. The S3 statements apply to any catalog with tables on S3. Add what your catalog needs to read tables and commit changes. S3 Tables uses its own
s3tables:permissions instead of bucket access.
Register a catalog with the REST API
With an org API key that has the catalogs:write scope, you can register a catalog without the console. The body takes the same fields as the form; name and type are required.
curl -X POST https://api.lakeops.dev/api/catalogs \
-H "Authorization: Bearer $LAKEOPS_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "prod_rest",
"type": "rest",
"uri": "https://catalog.example.com/api/catalog",
"warehouse": "<warehouse>",
"description": "Production Iceberg REST catalog",
"properties": {
"credential": "<CLIENT_ID>:<CLIENT_SECRET>",
"io-impl": "org.apache.iceberg.aws.s3.S3FileIO",
"s3.region": "us-east-1",
"s3.access-key-id": "<ACCESS_KEY_ID>",
"s3.secret-access-key": "<SECRET_ACCESS_KEY>"
}
}'- •
201returns the new catalog and starts discovery. - •
400with"Invalid catalog configuration"means the catalog could not be built from your settings. Themessageanddetailsfields carry the underlying error.
credential is the OAuth client ID and secret your REST catalog issues; use token instead if it gives you a bearer token. Set warehouse to the value your catalog expects.
Use the API host for your region; Connect Your Agent shows it. The REST API reference covers the other catalog routes.
After you connect
- •Review health on the Dashboard and in Data > Tables (Quick Start, step 3).
- •Turn on Adaptive Maintenance for a table or the whole catalog.
