Connect a Catalog

LakeOps works on the Iceberg catalogs you already run. You register a catalog once, LakeOps discovers its namespaces and tables, and health analysis starts. Your tables stay where they are.

Google Cloud and Azure are in alpha. Amazon S3 and S3-compatible storage are supported today. Support for Iceberg tables on Google Cloud Storage and Azure (Blob Storage and ADLS) is in alpha. Contact us to connect a catalog there.

Before you start

  • •You are signed in to the console in the data region where you want this catalog to live.
  • •You have an AWS identity for LakeOps, access keys or an IAM role, with the permissions in Credentials and permissions.
  • •For REST catalogs, the catalog URI is reachable from LakeOps over the network. The template's localhost values are placeholders.

Add a catalog in the console

  1. Go to Data > Catalogs and click Add Catalog.
  2. Pick a template. It sets the catalog type and pre-fills the properties.
  3. Fill in the form (below) and replace every placeholder property value with your own.
  4. Click Create Catalog. LakeOps builds the Iceberg catalog from your settings first; if that fails, nothing is saved and the error explains why.
  5. On success, LakeOps runs a first discovery scan. Tables then appear under Data > Explore and Data > Tables.

Form fields

FieldNotes
Catalog nameRequired. No spaces. This is the name you use in the REST API and MCP tools.
DescriptionOptional.
Catalog typeSet by the template. Read-only.
URIThe catalog endpoint. Not used for Glue.
WarehouseWhere table data lives: an S3 path, or the table bucket ARN for S3 Tables.
PropertiesKey/value Iceberg catalog properties, passed to the catalog as-is. Add rows with Add property.

Catalog templates

Templates pre-fill the most common setups, and you can edit, remove, or add any property before you save. Each template except Custom also sets type and Iceberg's S3 file IO (io-impl = org.apache.iceberg.aws.s3.S3FileIO). The cards list the other properties it pre-fills.

Glue + S3

type: glue

The AWS Glue Data Catalog, with tables on Amazon S3.

URI
Leave empty
Warehouse
s3://your-bucket/path
Properties
client.region, client.access-key-id, client.secret-access-key

AlphaAccess keys in the client.access-key-id and client.secret-access-key properties aren't applied to Glue yet. For now, our team connects Glue catalogs with an IAM role that LakeOps assumes.

REST + S3

type: rest

Any catalog that serves the Apache Iceberg REST API, such as Apache Polaris or Project Nessie, with tables on Amazon S3 or S3-compatible storage.

URI
Your REST catalog endpoint
Warehouse
s3://your-bucket/path
Properties
s3.endpoint, s3.access-key-id, s3.secret-access-key, s3.region, s3.path-style-access

S3 TablesAlpha

type: s3

Amazon S3 Tables, through its Iceberg REST endpoint.

URI
https://s3tables.<region>.amazonaws.com/iceberg
Warehouse
arn:aws:s3tables:<region>:<account-id>:bucket/<table-bucket>
Properties
s3.access-key-id, s3.secret-access-key, s3.region, s3.path-style-access, rest.sigv4-enabled = true, rest.signing-name = s3tables, rest.signing-region, rest.access-key-id, rest.secret-access-key, client.region

Not supported end to end yet. Talk to our team before you rely on it.

DynamoDB + S3Alpha

type: dynamodb

Iceberg's DynamoDB catalog, with tables on Amazon S3.

URI
dynamodb://<table-name>
Warehouse
s3://your-bucket/path
Properties
table-name, s3.endpoint, s3.access-key-id, s3.secret-access-key, s3.region, dynamodb.access-key-id, dynamodb.secret-access-key, dynamodb.region

Not supported end to end yet. Talk to our team before you rely on it.

CustomAlpha

type: custom

Start from an empty form and set every property yourself.

URI
If your catalog needs one
Warehouse
Your warehouse location
Properties
The properties you add

Not supported end to end yet. Talk to our team before you rely on it.

Credentials and permissions

LakeOps reaches your catalog and storage with an AWS identity you control: access keys for a dedicated IAM user, or an IAM role in your account that LakeOps assumes. Whichever you use, scope it to the catalog and the bucket where your tables live. For Glue, access keys are in alpha Alpha, so use an IAM role. Our team can help you pick and set it up.

How credentials are handled

LakeOps stores the access keys with the catalog configuration and uses them only to run maintenance. Anyone in your organization who can read catalogs, in the console or with an API key that has catalog read access, can view them. Use a dedicated, scoped identity and rotate its keys when a member leaves.

What each permission is for

LakeOps needs toAWS actions (Glue + S3)
Discover and analyzeglue:GetDatabases, glue:GetDatabase, glue:GetTables, glue:GetTable, s3:ListBucket, s3:GetBucketLocation, s3:GetObject
Commit maintenance resultsglue:UpdateTable, s3:PutObject, s3:AbortMultipartUpload
Expire snapshots and remove orphan filess3:DeleteObject
Rename or delete a table from LakeOpsglue:CreateTable, glue:DeleteTable

Sample IAM policy for Glue + S3

Replace the region (us-east-1), account ID (123456789012), and bucket (my-warehouse-bucket) with your own.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadAndCommitGlueTables",
      "Effect": "Allow",
      "Action": [
        "glue:GetDatabases",
        "glue:GetDatabase",
        "glue:GetTables",
        "glue:GetTable",
        "glue:UpdateTable"
      ],
      "Resource": [
        "arn:aws:glue:us-east-1:123456789012:catalog",
        "arn:aws:glue:us-east-1:123456789012:database/*",
        "arn:aws:glue:us-east-1:123456789012:table/*/*"
      ]
    },
    {
      "Sid": "RenameAndDeleteGlueTables",
      "Effect": "Allow",
      "Action": ["glue:CreateTable", "glue:DeleteTable"],
      "Resource": [
        "arn:aws:glue:us-east-1:123456789012:catalog",
        "arn:aws:glue:us-east-1:123456789012:database/*",
        "arn:aws:glue:us-east-1:123456789012:table/*/*"
      ]
    },
    {
      "Sid": "ListWarehouseBucket",
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetBucketLocation"],
      "Resource": "arn:aws:s3:::my-warehouse-bucket"
    },
    {
      "Sid": "ReadWriteDeleteWarehouseFiles",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:AbortMultipartUpload"],
      "Resource": "arn:aws:s3:::my-warehouse-bucket/*"
    }
  ]
}
  • •Narrow it. Replace database/* and table/*/* with your database and table names, and my-warehouse-bucket/* with your table location prefix. Add a bucket for each location your tables use.
  • •Start read-only. Keep only the Get and List actions and s3:GetObject. Discovery and health analysis work; maintenance fails until you add the write and delete actions.
  • •Optional statement. Drop RenameAndDeleteGlueTables if you will not rename or delete tables from LakeOps.
  • •Encrypted buckets. With SSE-KMS, also allow kms:Decrypt and kms:GenerateDataKey on the key.
  • •Lake Formation. If Lake Formation governs the catalog, also allow lakeformation:GetDataAccess and grant the identity access to the databases and tables in Lake Formation.
  • •Other catalogs. The S3 statements apply to any catalog with tables on S3. Add what your catalog needs to read tables and commit changes. S3 Tables uses its own s3tables: permissions instead of bucket access.

Register a catalog with the REST API

With an org API key that has the catalogs:write scope, you can register a catalog without the console. The body takes the same fields as the form; name and type are required.

curl -X POST https://api.lakeops.dev/api/catalogs \
  -H "Authorization: Bearer $LAKEOPS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "prod_rest",
    "type": "rest",
    "uri": "https://catalog.example.com/api/catalog",
    "warehouse": "<warehouse>",
    "description": "Production Iceberg REST catalog",
    "properties": {
      "credential": "<CLIENT_ID>:<CLIENT_SECRET>",
      "io-impl": "org.apache.iceberg.aws.s3.S3FileIO",
      "s3.region": "us-east-1",
      "s3.access-key-id": "<ACCESS_KEY_ID>",
      "s3.secret-access-key": "<SECRET_ACCESS_KEY>"
    }
  }'
  • •201 returns the new catalog and starts discovery.
  • •400 with "Invalid catalog configuration" means the catalog could not be built from your settings. The message and details fields carry the underlying error.

credential is the OAuth client ID and secret your REST catalog issues; use token instead if it gives you a bearer token. Set warehouse to the value your catalog expects.

Use the API host for your region; Connect Your Agent shows it. The REST API reference covers the other catalog routes.

After you connect