AI workloads are the fastest-growing consumers of lakehouse data. Training pipelines scan petabytes of feature tables. RAG systems query embedding indexes at sub-second latency. ML inference services pull model artifacts and reference data on every request. These workloads are computationally intense, operationally critical, and — in most deployments — running with far more storage access than they need.
The default security model for most Iceberg-on-S3 deployments is a pair of long-lived AWS access keys, distributed to every Spark job, Trino cluster, and notebook environment that needs to read the lake. The keys grant broad S3 permissions — often s3:* on the entire warehouse bucket — because narrowing the scope for each workload is operationally painful. The result is an architecture where every compute engine is one compromised credential away from full access to every table in the lake.
This article walks through a zero-trust data architecture for AI workloads on Apache Iceberg and S3. The architecture replaces static credentials with short-lived, table-scoped tokens — enforced at the storage layer, orchestrated through Kubernetes, and designed for the access patterns that AI/ML workloads actually produce.
Why AI/ML workloads demand a different security model
Traditional analytics workloads — BI dashboards, scheduled reports, ad-hoc SQL queries — have predictable access patterns. A Trino cluster runs SELECT queries against a known set of tables. An Athena user scans a specific partition range. The blast radius of a compromised credential is bounded by what the user was going to access anyway.
AI workloads are different in three ways that make static credentials dangerous.
Breadth of access. A feature engineering pipeline touches dozens of tables across multiple namespaces — user events, transaction histories, product catalogs, behavioral signals. A single training job might scan 50 tables to construct a feature matrix. Granting static keys scoped to a single table is impractical when the workload legitimately needs cross-table access. So teams grant broad permissions and accept the risk.
Frequency of access. An ML inference service running at 10,000 requests per second pulls reference data continuously. A RAG pipeline re-indexes embeddings every 15 minutes. A real-time feature store serves lookups at microsecond latency. These are not batch jobs that run once and terminate — they are long-running services with persistent storage connections. A compromised credential in an inference service remains exploitable for weeks or months until someone rotates the key.
Autonomy of execution. AI agents and automated pipelines run without human oversight. An agent exploring a lakehouse to discover training features will access tables no one anticipated. A hyperparameter search might spawn 200 parallel training jobs, each scanning different subsets of the data. Unlike a human analyst who opens a known dashboard, AI workloads generate unpredictable access patterns that static permission boundaries cannot anticipate.
The combination of broad access, persistent connections, and autonomous execution means that static credentials in AI/ML infrastructure are not just a security gap — they are a liability that scales with the workload.
The problem with static credentials
Static credentials fail at three levels: security, operations, and compliance.
Configuration drift. When a data engineering team provisions a new Spark cluster, they copy the S3 credentials from an existing cluster. The credentials work, the job runs, and the team moves on. Six months later, the original IAM user has been granted additional permissions for unrelated workloads — and every cluster using those copied credentials silently inherits the expanded access. No one audits which clusters have which keys. No one knows that the staging cluster credentials now have write access to production tables.
Over-permissioned access. Scoping IAM policies to individual S3 prefixes per workload requires creating a separate IAM user (or role) for each compute job, maintaining the mapping between jobs and permissions, and updating both sides when table locations change. In practice, teams create one IAM user per environment (dev, staging, prod) with s3:GetObject and s3:PutObject on s3://warehouse-bucket/*. Every job in that environment can read and write every table. An inference service that only needs read access to two tables has write access to all of them.
Credential sprawl. In a typical multi-engine lakehouse, the same S3 credentials are embedded in Spark spark-defaults.conf, Trino catalog/iceberg.properties, Jupyter notebook environment variables, Airflow connection strings, Kubernetes secrets, CI/CD pipelines, and application configuration files. The credentials exist in 15–30 locations across the infrastructure. Rotating a key means touching every one of those locations, testing each integration, and coordinating the cutover across teams. Most organizations do not rotate lakehouse credentials. Some have credentials that have not been rotated in over two years.
No attribution. When all Spark jobs use the same IAM user, CloudTrail shows that iam-user/spark-production accessed s3://warehouse/events/data/part-00001.parquet at 14:32 UTC. It does not show which Spark job, which user submitted it, which table it was querying, or whether the access was authorized by the catalog. If the access was malicious — an exfiltration attempt, an unauthorized data export — the audit trail provides no useful forensic detail.
Compliance failure. SOC 2, HIPAA, and GDPR all require demonstrable least-privilege access controls and audit trails that trace data access to specific principals and purposes. An architecture where every compute engine shares a small set of over-permissioned, long-lived credentials fails both requirements. The auditor asks "who accessed patient records in Q3?" and the answer is "any Spark job in production could have."
Solving these problems requires more than better credentials — it requires operational visibility across the entire lakehouse. LakeOps connects to existing catalogs (Glue, Polaris, REST, S3 Tables) and engines as a dedicated control plane, providing centralized monitoring of table health, access patterns, and structural integrity without moving data. When zero-trust credentials scope access per-table, having a unified view of which tables are healthy, which are degraded, and which need immediate attention becomes a security prerequisite — not just an operational nice-to-have.
Zero-trust principles applied to lakehouse data
Zero-trust is not a product — it is a set of design constraints. Applied to lakehouse data on S3, these constraints reshape how compute engines interact with storage.
Never trust, always verify. Every data access request must be authenticated and authorized at the time of access, regardless of network position or prior access history. A Spark executor that successfully read a table five seconds ago must re-authenticate to read it again. No cached trust. No ambient credentials.
Least-privilege access. Credentials are scoped to the minimum set of S3 objects required for the specific operation — not the bucket, not the prefix, but the exact table path. A training job that reads s3://warehouse/features/user_behavior/data/*.parquet gets credentials valid only for that path. It cannot read s3://warehouse/features/transaction_history/ or write to any location.
Assume breach. The architecture assumes that any compute node can be compromised at any time. Short-lived credentials limit the window of exploitation. Table-scoped access limits the blast radius. Storage-layer enforcement ensures that a compromised engine cannot bypass catalog-level access controls by hitting S3 directly.
Continuous verification. Authorization decisions are not made once at connection time. Each table load triggers a fresh credential vend with current permissions. Role changes, table relocations, and policy updates take effect immediately — not at the next credential rotation cycle.
Storage-layer enforcement. Access controls are enforced by the S3-compatible object store itself — not by a proxy, sidecar, or compute engine plugin. This is the critical architectural distinction. Compute engines are untrusted. The catalog is the policy authority. The storage layer is the enforcement point. No middleware, no performance penalty, no bypass path.
Vended credentials: the core mechanism
Vended credentials replace static keys with short-lived, table-scoped tokens issued on every data access request. The mechanism is built into the Iceberg REST catalog specification and supported by every major catalog implementation — Apache Polaris, Snowflake Horizon, Databricks Unity Catalog, AWS Glue, and Dremio.
The flow works in four steps.
Step 1: Engine requests table access. A Spark executor, Trino worker, or PyIceberg client calls the catalog's loadTable endpoint, including the HTTP header X-Iceberg-Access-Delegation: vended-credentials. This header tells the catalog that the client wants temporary storage credentials rather than relying on its own ambient identity.
Step 2: Catalog authenticates and authorizes. The catalog verifies the client's identity (via OAuth2 token, Kubernetes service account, or other OIDC provider), resolves the requested table, and checks RBAC policies. Does this principal have read access to this table? Write access? No access at all?
Step 3: Catalog vends scoped credentials. If authorized, the catalog calls the storage provider's token service — AWS STS AssumeRole for S3, SAS token generation for ADLS, signed blob tokens for GCS — and requests temporary credentials scoped to the table's storage path. For S3, this means an IAM session policy that restricts s3:GetObject to s3://warehouse/namespace/table/data/*. The session has a configurable TTL, typically 15–60 minutes depending on the sensitivity of the data.
Step 4: Engine accesses storage directly. The catalog returns the table metadata (schema, partition spec, manifest list location) along with the temporary credentials. The engine uses these credentials to read data files directly from S3. When the credentials expire, the engine calls loadTable again and receives a fresh set.
The result is that no engine ever holds credentials broader than what it needs for the current table, and no credentials persist longer than the configured TTL. A compromised Spark executor has, at worst, read access to one table for 15 minutes.
What vended credentials look like in practice
On AWS, the catalog calls sts:AssumeRole with a session policy that restricts S3 operations to the table's prefix. The resulting temporary credentials — an access key ID, secret access key, and session token — are valid for the configured TTL and carry an inline policy like this:
1{2 "Version": "2012-10-17",3 "Statement": [4 {5 "Effect": "Allow",6 "Action": ["s3:GetObject"],7 "Resource": "arn:aws:s3:::warehouse/analytics/user_events/data/*"8 }9 ]10}The session token is the enforcement mechanism. Even if the underlying IAM role has s3:* on the entire bucket, the session policy restricts the temporary credentials to the specific table prefix. S3 evaluates both the role policy and the session policy — the effective permissions are the intersection. This is AWS's native least-privilege enforcement, requiring no custom infrastructure.
Multi-cloud credential vending
Apache Polaris v1.4 (released April 2026) can issue credentials for tables stored in different cloud providers from a single Polaris instance. A Polaris deployment serving a multi-cloud lakehouse vends AWS STS tokens for S3 tables, Azure SAS tokens for ADLS tables, and GCS signed tokens for Google Cloud Storage tables — all through the same REST catalog API. This eliminates the need for separate catalog deployments per cloud, simplifying lakehouse governance for organizations with multi-cloud data strategies.
The control plane: Kubernetes-native orchestration
Credential vending is the security mechanism. The control plane is what makes it operationally manageable at scale — deploying catalogs, configuring storage integrations, managing IAM roles, and orchestrating the lifecycle of lakehouse components.
The architecture uses a Kubernetes-native control plane built on Custom Resource Definitions (CRDs) and operators. This is the Glue-Operator pattern — not to be confused with AWS Glue. The Kubernetes Glue Operator is a meta-operator from the Java Operator SDK that lets you define operators declaratively by applying custom resources rather than writing controller code.
How the control plane works
CRDs define lakehouse components. Each component of the zero-trust lakehouse — the catalog server, storage integrations, IAM role bindings, compute engine configurations — is represented as a Kubernetes custom resource. A PolarisCatalog CRD defines the catalog deployment. A StorageIntegration CRD defines the S3 bucket, IAM role ARN, and STS configuration. A CatalogRole CRD maps principals to table-level permissions.
Operators reconcile desired state. The Glue-Operator pattern uses Kubernetes reconciliation to ensure that the declared configuration matches the running infrastructure. When a platform engineer applies a new StorageIntegration resource, the operator provisions the IAM role, configures the trust policy, creates the S3 bucket policy, and registers the storage integration with Polaris — all automatically.
Terraform automates prerequisites. The underlying AWS resources — IAM roles, trust policies, S3 bucket policies, KMS keys — are provisioned by Terraform. The Kubernetes operator references these resources by ARN. This separation of concerns keeps infrastructure provisioning (Terraform) separate from application orchestration (Kubernetes), while ensuring that both layers are declarative and version-controlled.
Rook manages the storage layer. For organizations running S3-compatible object storage on Ceph, Rook is the Kubernetes operator that manages the Ceph cluster lifecycle — deployment, scaling, upgrades, and health monitoring. Rook provisions the RADOS Gateway (RGW) that serves the S3 API, configures STS endpoints for credential vending, and manages the IAM integration that Polaris depends on.
The Kubernetes-native approach has a specific advantage for AI workloads: compute engines are already running in Kubernetes. Spark on Kubernetes, Trino on Kubernetes, Ray on Kubernetes, Jupyter notebooks on Kubernetes. The zero-trust control plane lives in the same orchestration layer as the workloads it governs. Service account tokens used for pod-level authentication flow directly into catalog OAuth2 without additional identity bridging.
Example: declaring a storage integration
1apiVersion: polaris.apache.org/v12kind: StorageIntegration3metadata:4 name: production-warehouse5 namespace: lakehouse6spec:7 storageType: s38 bucket: warehouse-production9 region: us-east-110 roleArn: arn:aws:iam::123456789012:role/polaris-catalog-role11 stsEndpoint: https://sts.us-east-1.amazonaws.com12 credentialTtlSeconds: 90013 allowedLocations:14 - s3://warehouse-production/analytics/*15 - s3://warehouse-production/ml-features/*This CRD declares that the production-warehouse integration covers two S3 prefixes — analytics and ml-features. Vended credentials are scoped to these prefixes with a 15-minute TTL. The operator ensures the IAM role's trust policy allows sts:AssumeRole and sts:TagSession from the Polaris service identity.
The data plane: Polaris + S3 + compute engines
The data plane is where queries actually execute. It consists of three layers — the REST catalog (Apache Polaris), the object storage (S3 or S3-compatible), and the compute engines (Spark, Trino, Flink, PyIceberg).
Apache Polaris as the governance hub
Apache Polaris is an open-source, Iceberg-native REST catalog that graduated from the Apache Incubator in February 2026. It implements the full Iceberg REST catalog specification — credential vending, server-side commit deconflicting, multi-table commits, OAuth2 authentication, and namespace management.
For zero-trust architectures, Polaris provides three critical capabilities.
Centralized RBAC. Polaris uses a three-level permission model — principals, principal roles, and catalog roles. Principals (users or service accounts) are assigned to principal roles. Principal roles are granted catalog roles. Catalog roles carry the actual permissions — read access to a specific table, write access to a namespace, admin access to a catalog. This indirection decouples identity from permissions, allowing role assignments to change without touching catalog grants.
Credential vending with STS session tags. Starting with Polaris 1.4, vended credentials carry AWS STS session tags that propagate through CloudTrail. Five tags are attached to every assumed-role session: polaris:principal, polaris:catalog, polaris:namespace, polaris:table, and polaris:request-id. When a Spark job reads a Parquet file from S3, the CloudTrail GetObject event carries the session tags — linking the S3 access back to the exact principal, catalog, and table that authorized it.
Catalog federation. A single Polaris instance can route requests to multiple backend catalogs — Hive Metastore, AWS Glue, other Iceberg REST catalogs. This enables incremental adoption: organizations can register existing Glue-managed tables in Polaris without migrating metadata. The zero-trust credential vending layer applies uniformly regardless of which backend catalog manages the table.
S3 as the enforcement point
The security guarantee of vended credentials depends on S3 evaluating session policies on every request. When a compute engine presents temporary credentials with an inline session policy, S3 evaluates the intersection of the IAM role policy and the session policy. The effective permissions are the most restrictive combination.
This is storage-layer enforcement. The compute engine is untrusted — it could be compromised, misconfigured, or running malicious code. It does not matter. S3 enforces the access boundary on every GetObject, PutObject, and ListBucket call. No proxy, sidecar, or engine plugin is in the path.
For S3-compatible object stores like Ceph RGW, the same enforcement model applies. Ceph RGW supports STS-issued credentials and evaluates session policies natively. The Polaris-vended tokens are passed in each S3 call and enforced by RGW — the same zero-trust guarantee as AWS S3, running on-premises.
S3 object tagging adds a second enforcement layer. Polaris can apply S3 tags to data files that map them to specific tables and namespaces — iceberg:table=user_events, iceberg:namespace=analytics. Session policies can include tag-based conditions, restricting access to objects tagged with specific table identifiers. This provides defense-in-depth: even if a session policy has a broader-than-intended prefix, the tag condition prevents access to objects belonging to other tables in the same prefix.
Compute engine integration
Every major Iceberg-compatible engine supports vended credentials through the REST catalog protocol.
Apache Spark is the most common engine for AI/ML workloads — feature engineering, model training data preparation, and large-scale ETL. Spark's Iceberg integration requests vended credentials automatically when configured with a REST catalog and the X-Iceberg-Access-Delegation: vended-credentials header. Credential refresh is handled transparently — when a long-running training job's credentials approach expiration, the Spark Iceberg runtime calls loadTable again to obtain fresh tokens.
1spark.sql.catalog.lakehouse = org.apache.iceberg.spark.SparkCatalog2spark.sql.catalog.lakehouse.type = rest3spark.sql.catalog.lakehouse.uri = https://polaris.internal:8181/api/catalog4spark.sql.catalog.lakehouse.credential = ${OAUTH2_CLIENT_ID}:${OAUTH2_CLIENT_SECRET}5spark.sql.catalog.lakehouse.scope = PRINCIPAL_ROLE:ML_ENGINEER6spark.sql.catalog.lakehouse.warehouse = ml-warehouse7spark.sql.catalog.lakehouse.header.X-Iceberg-Access-Delegation = vended-credentialsTrino provides interactive SQL over Iceberg tables — critical for exploratory data analysis, feature validation, and ad-hoc queries during model development. Trino's Iceberg connector supports the REST catalog with credential vending natively. Configuring Trino involves setting the Iceberg connector to use the REST catalog type, pointing it at the Polaris endpoint, and enabling the native S3 filesystem to accept vended credentials from the catalog rather than using static configuration.
PyIceberg is the Python-native client for Iceberg — used in Jupyter notebooks, data science workflows, and custom ML pipelines. PyIceberg supports REST catalogs with credential vending, making it straightforward to build Python applications that access Iceberg tables without embedding static S3 credentials.
Flink handles streaming ingestion into Iceberg tables. Flink's Iceberg sink supports REST catalogs, meaning that streaming writes also benefit from vended credentials — each checkpoint cycle obtains fresh tokens scoped to the write path.
Integration with Spark and Trino for AI/ML querying
Zero-trust credential vending changes the security model, but it should not change the query performance characteristics that AI/ML workloads depend on. Two integration patterns are critical.
Spark for training pipelines
A typical ML training pipeline reads feature tables with billions of rows, joining multiple tables to construct the training dataset. The Spark-Iceberg integration with vended credentials supports this pattern with minimal overhead.
The credential vend occurs at table load time — once per table, not once per partition or file. A training job that reads five tables makes five credential vend calls. Each call adds approximately 50–100ms of latency (the STS AssumeRole call). On a training pipeline that runs for hours, this is negligible.
For long-running training jobs, credential refresh is critical. Spark's Iceberg runtime tracks credential expiration and refreshes tokens before they expire. A training job reading a 2 TB table over 45 minutes will refresh credentials 2–3 times (with a 15-minute TTL) — each refresh is transparent to the Spark task.
Partition pruning, predicate pushdown, and data skipping work identically with vended credentials. The Iceberg metadata — partition specs, column statistics, manifest-level min/max bounds — is returned alongside the credentials in the loadTable response. Spark's query planner uses this metadata to skip irrelevant partitions and files before issuing any S3 reads with the vended tokens.
Trino for feature validation and exploration
Data scientists exploring feature tables during model development need interactive query latency — sub-5-second responses on filtered aggregations over tables with hundreds of millions of rows. Trino's Iceberg connector with vended credentials delivers this without compromising security.
Trino caches table metadata (including credential expiration) and refreshes tokens proactively. Credential vending adds no perceptible latency to interactive queries because the STS call is pipelined with metadata loading — by the time the query planner finishes generating the execution plan, the credentials are already available.
For multi-table joins — common in feature engineering where features from different source tables are joined on entity keys — each table in the join gets its own set of vended credentials. The engine holds N sets of temporary credentials for an N-way join, each scoped to its respective table prefix. Trino's query execution engine handles this transparently.
Production patterns
Deploying zero-trust credential vending in production requires more than configuring the catalog. These operational patterns address the challenges teams encounter at scale.
Credential rotation and TTL tuning
TTL is a trade-off between security and performance. Shorter TTLs reduce the window of exploitation for compromised credentials but increase STS API calls and introduce latency from more frequent credential refreshes.
For PII and regulated data (HIPAA, SOC 2): 15-minute TTL. The increased STS call volume is a worthwhile trade-off for tables containing personal health information, financial records, or other regulated data. Monitor STS ThrottlingException errors in CloudTrail — AWS imposes account-level STS rate limits that can be hit under heavy credential vending loads.
For general analytics tables: 30-minute TTL. Balances security with credential cache reuse in the catalog. Polaris caches vended credentials keyed by (principal, table, session context) — a shorter TTL invalidates the cache more frequently, increasing STS calls.
For high-throughput ML inference: 60-minute TTL. An inference service processing 10,000 requests per second cannot afford per-request credential vends. The 60-minute TTL allows credential reuse across many requests while maintaining the scoped access guarantee. Ensure the inference service requests credential refresh before expiration — not after.
Audit trails with STS session tags
Polaris 1.4's STS session tags transform CloudTrail from a raw access log into a queryable audit trail for lakehouse access. The five session tags — polaris:principal, polaris:catalog, polaris:namespace, polaris:table, and polaris:request-id — appear on every S3 data event generated with vended credentials.
Query pattern: "Which principals accessed the pii.user_profiles table in the last 30 days?" Filter CloudTrail S3 data events where sessionContext.sessionIssuer.sessionPolicy contains the session tags. Aggregate by polaris:principal to get a list of every identity that touched the table — with timestamps, operation types, and request IDs for correlation.
Compliance pattern: For GDPR data subject access requests, filter CloudTrail by polaris:table matching the relevant table and time window. The audit trail shows exactly which principals accessed the data, through which catalog, with full request-level granularity.
Enabling session tags requires a configuration change in Polaris and an IAM trust policy update:
1polaris:2 features:3 INCLUDE_SESSION_TAGS_IN_SUBSCOPED_CREDENTIAL: "true"4 INCLUDE_TRACE_ID_IN_SESSION_TAGS: "true"The IAM role's trust policy must explicitly allow sts:TagSession:
1{2 "Effect": "Allow",3 "Principal": {4 "AWS": "arn:aws:iam::123456789012:role/polaris-service-role"5 },6 "Action": ["sts:AssumeRole", "sts:TagSession"]7}The optional INCLUDE_TRACE_ID_IN_SESSION_TAGS flag adds an OpenTelemetry trace ID to the session tags, enabling end-to-end correlation from the Polaris request trace through STS to S3 data events. This is powerful for debugging — a slow query in Trino can be traced through Polaris credential vending to the specific S3 GetObject calls — but be aware that trace IDs are high-cardinality and effectively disable credential caching. Enable this for debugging or compliance-critical tables, not globally.
Cross-account access
Multi-account AWS architectures are standard for AI/ML workloads — training runs in a compute account, data in a data lake account, model artifacts in a model registry account. Vended credentials support cross-account access through IAM role chaining.
The Polaris service role in the catalog account assumes a cross-account role in the data lake account. The session policy scoping works identically — the vended credentials are restricted to the specific table prefix regardless of account boundary. STS session tags are marked as transitive, meaning they persist through the role chain. A CloudTrail event in the data lake account shows the polaris:principal tag from the original catalog request, even though the access crossed account boundaries.
The IAM trust policy on the data lake account role specifies the Polaris service role as a trusted principal:
1{2 "Effect": "Allow",3 "Principal": {4 "AWS": "arn:aws:iam::111111111111:role/polaris-service-role"5 },6 "Action": ["sts:AssumeRole", "sts:TagSession"],7 "Condition": {8 "StringEquals": {9 "aws:PrincipalOrgID": "o-organization-id"10 }11 }12}The aws:PrincipalOrgID condition restricts the trust relationship to roles within the same AWS Organization, preventing accidental cross-organization access.
Rate limiting and STS throttling
Heavy credential vending generates significant STS API traffic. A lakehouse with 500 concurrent Spark executors, each accessing 10 tables with 15-minute TTL credentials, generates approximately 20,000 STS AssumeRole calls per hour. AWS's default STS rate limit is 500 calls per second per account — well above this volume — but burst patterns during cluster startup can trigger ThrottlingException errors.
Polaris mitigates this through credential caching. Credentials for the same (principal, table, session context) tuple are cached and reused until they approach expiration. This reduces the effective STS call volume by 80–90% in production deployments. The cache key includes session tags, so credentials for different principals accessing the same table are never cross-contaminated.
For high-volume deployments, use regional STS endpoints (sts.us-east-1.amazonaws.com) instead of the global endpoint (sts.amazonaws.com). Regional endpoints have separate rate limits and lower latency for same-region calls.
Multi-tenant lakehouse environments
Zero-trust credential vending is particularly well-suited for multi-tenant lakehouse architectures where multiple teams, business units, or customers share the same physical infrastructure while requiring strict data isolation.
Tenant isolation through catalog hierarchy
Polaris supports multiple isolated catalogs within a single deployment. Each tenant gets its own catalog with its own storage integration, IAM roles, and permission grants. The catalog boundary is the isolation boundary — a principal with access to catalog-a cannot access any table in catalog-b, regardless of S3 permissions.
Per-tenant IAM roles. Each tenant's catalog is backed by a separate IAM role with permissions scoped to the tenant's S3 prefix. The IAM role for tenant-alpha has s3:GetObject on s3://warehouse/tenant-alpha/* and nothing else. Even if a bug in Polaris returned the wrong catalog's credentials (which the RBAC layer prevents), the IAM role's policy would block cross-tenant access at the storage layer.
Per-tenant credential TTLs. Tenants with different compliance requirements can have different credential TTLs. A healthcare tenant processes PHI with a 15-minute TTL. A marketing analytics tenant uses a 60-minute TTL for performance. The configuration is per storage integration, not global.
Per-tenant audit trails. STS session tags include polaris:catalog, providing tenant-level audit granularity. Filter CloudTrail by catalog name to produce per-tenant access reports — critical for B2B SaaS platforms that need to demonstrate data isolation to enterprise customers.
Namespace-level isolation within tenants
Within a single catalog, namespaces provide a second isolation layer. A data science team gets access to ml-features.* tables. A business intelligence team gets access to analytics.*. Vended credentials are scoped to the specific table's S3 prefix — even if both namespaces share the same S3 bucket, the IAM session policies prevent cross-namespace access.
This namespace-level isolation is particularly valuable for AI workloads where different teams train models on different subsets of data. The feature engineering team has write access to feature tables. The model training team has read access to the same tables. The production inference team has read access to a subset of feature tables and read access to model metadata tables. Each team's vended credentials reflect exactly these permissions — no more, no less.
Shared compute with isolated data
Multi-tenant environments often share compute infrastructure — a single Trino cluster or Spark deployment serves multiple tenants. Vended credentials enable this safely. Each query authenticates as a specific principal, receives credentials scoped to the authorized tables, and executes with those scoped credentials. Two tenants running queries on the same Trino worker process cannot access each other's data because S3 enforces the session policies independently.
This eliminates the need for per-tenant compute clusters — a significant cost reduction for platforms serving many small tenants. The security guarantee comes from the storage layer, not from compute isolation.
Secure data must also be performant
Zero-trust credential vending secures access to lakehouse data at the storage layer. But secure data must also be performant — an AI training job scanning 50 million files across poorly compacted tables fails regardless of how elegant the credential vending is. The 15-minute credential TTL expires before the query finishes planning because the engine is still listing manifests. The scoped S3 permissions are correct, but the query reads 10x more data than necessary because the table lacks sort order optimization for the training job's access pattern.
LakeOps ensures the tables AI workloads query are optimized for the access patterns ML jobs actually use. Continuous compaction merges small files into optimal sizes — reducing the 50 million files from streaming ingestion to a few thousand properly-sized Parquet files. Query-aware sort ordering rearranges data so that predicate pushdown and min/max statistics eliminate irrelevant row groups before any S3 read. Manifest consolidation keeps metadata files compact so that query planning completes in milliseconds, well within the credential TTL window.
The combination is what makes zero-trust viable for AI at scale. Credential vending provides the security layer. Table optimization provides the performance layer. Without both, you get either secure but slow (credential vending on uncompacted tables), or fast but insecure (optimized tables with static credentials).
For organizations building zero-trust lakehouses, LakeOps managed Iceberg provides autonomous table maintenance that keeps every table in the lake optimized for its actual access patterns. Compaction, manifest rewrites, snapshot expiration, and orphan cleanup run continuously as a coordinated pipeline — ensuring that the tables your AI workloads query are always ready for the next credential-scoped read.
Architecture reference
The complete zero-trust data architecture for AI workloads on Iceberg and S3 consists of five layers:
1. Identity layer. OAuth2/OIDC providers (Keycloak, Okta, AWS IAM Identity Center) authenticate principals. Kubernetes service accounts map pod-level identity to catalog principals. Every request carries a verifiable identity token.
2. Catalog layer. Apache Polaris serves as the governance hub — RBAC, credential vending, federation, and audit. Deployed on Kubernetes, managed by operators, configured through CRDs.
3. Credential layer. AWS STS (or equivalent) issues short-lived, scoped tokens. Session tags carry catalog context into CloudTrail. Token TTLs vary by data sensitivity.
4. Storage layer. S3 (or S3-compatible) enforces session policies on every request. Object tags provide defense-in-depth. No trust in compute — enforcement is at the storage boundary.
5. Compute layer. Spark, Trino, Flink, PyIceberg — all untrusted, all receiving scoped credentials through the REST catalog protocol. No static keys, no ambient permissions, no configuration drift.
Each layer is independently deployable, independently auditable, and independently replaceable. The Iceberg REST catalog specification is the contract between layers — any catalog that implements credential vending, any engine that supports the X-Iceberg-Access-Delegation header, and any storage that evaluates session policies can participate in the architecture.
Getting started
Implementing zero-trust for your Iceberg lakehouse is an incremental process. Start with the highest-risk workloads and expand.
Step 1: Deploy a REST catalog with credential vending. Apache Polaris is the open-source option. Deploy on Kubernetes with the Polaris Helm chart, configure a storage integration for your S3 warehouse bucket, and create a test catalog with a few tables. Verify that Spark can load tables with vended credentials before proceeding.
Step 2: Enable STS session tags. Set INCLUDE_SESSION_TAGS_IN_SUBSCOPED_CREDENTIAL: true in the Polaris configuration. Update the IAM role trust policy to allow sts:TagSession. Verify that CloudTrail events carry the session tags. This gives you the audit trail immediately, even before migrating all workloads.
Step 3: Migrate AI workloads to vended credentials. Start with training pipelines — they are typically batch jobs with clear start and end points. Remove static S3 credentials from Spark configurations and replace with REST catalog configuration. Monitor STS call volumes and adjust TTLs based on workload characteristics.
Step 4: Ensure table performance matches security posture. Zero-trust credentials are only viable if queries complete within the credential TTL. Optimize table health — compact small files, consolidate manifests, expire old snapshots — so that query planning and execution complete well within the 15–60 minute credential window.
Step 5: Extend to multi-tenant isolation. Create per-tenant catalogs with isolated IAM roles and storage integrations. Configure namespace-level permissions within catalogs. Implement per-tenant audit reporting using session tag filtering in CloudTrail.
Further reading
- Apache Iceberg on AWS S3: A Guide — architecture, AWS services, configuration, and maintenance for Iceberg on S3
- Reduce AWS S3 Costs with Apache Iceberg — compaction, manifest optimization, and storage lifecycle strategies
- Managed Iceberg — autonomous table maintenance for production Iceberg lakehouses
- Apache Polaris Configuration — AWS S3 — Polaris documentation for S3 credential vending
- Governing the Lakehouse: Zero-Trust with Ceph and Polaris — IBM's reference architecture for on-premises zero-trust lakehouse
- REST Catalog Credential Vending for Lakehouse Security — detailed comparison of credential vending across catalog implementations
