LakeOpsLakeOps
Demo
Back to all articles

Cloud Cost articles

Cloud cost optimization for data lakes and lakehouses — storage waste, compute spend, query pricing, and strategies to cut lakehouse costs.

AllCloud CostApache IcebergLakeOpsLakehouseData PlatformsData LakeCompactionObservabilityStreamingData Governance

11 articles

Apache Iceberg Orphan Files — safe cleanup without breaking tables, with shield and broom icons over an Iceberg table
Apache IcebergCloud CostLakeOpsAWS

Apache Iceberg Orphan Files: Safe Cleanup Without Breaking Tables

Orphan files are invisible to Iceberg but fully billable by cloud storage. They accumulate silently from failed writes, crashed compaction, and concurrent conflicts — and on mature lakes they can account for 25–40% of storage spend. This guide covers how orphan files are created, how to detect them safely, the retention window that prevents table corruption, and how to automate cleanup at lake scale without listing millions of objects.

David W
David W
26 min read
Apache Iceberg Retention Policy — how long to keep snapshots, from newest to oldest with expired snapshot cleanup
Apache IcebergCloud CostLakeOpsData Lake

Apache Iceberg Retention Policy: How Long Should You Keep Snapshots?

Every Iceberg commit creates a snapshot. Left unmanaged, snapshots pin storage, inflate metadata, and slow query planning. This guide covers what snapshots actually cost, retention strategies by workload type, the retain_last vs older_than tradeoffs, compliance via tags and branches, the cascade from snapshots to orphan files to manifest bloat, and how to set automated retention policies that match your operational reality.

Rob M
Rob M
32 min read
Apache Iceberg on AWS S3 — architecture diagram showing Iceberg metadata layers, AWS services, and the data lakehouse stack
Apache IcebergAWSData LakeLakeOps

Apache Iceberg on AWS S3: A Guide

Apache Iceberg on AWS S3 is the standard architecture for open lakehouses. This guide covers how Iceberg's metadata hierarchy maps to S3 objects, the AWS services ecosystem (Glue, Athena, EMR, Redshift, S3 Tables), configuration best practices, performance optimization, table maintenance, and the operational components needed for production deployments.

Rob M
Rob M
24 min read
Reducing AWS S3 cost with Apache Iceberg — diagram showing S3 storage and API cost vectors from Iceberg write patterns and the optimization strategies that address them
FinOpsApache IcebergAWSLakeOps

Reducing AWS S3 Cost with Iceberg: A Guide

AWS S3 bills for Iceberg lakehouses are inflated by small files, orphan data, retained snapshots, metadata overhead, and scan amplification. This guide quantifies each cost vector with S3 pricing mechanics and walks through five strategies — compaction, expiration, layout optimization, storage tiering, and engine routing — to cut storage and query spend.

Rob M
Rob M
20 min read
Snowflake to Iceberg migration — Snowflake tables flowing into an Apache Iceberg lakehouse, illustrating a hybrid multi-engine architecture where Snowflake remains a valued component
SnowflakeApache IcebergLakeOpsData Platforms

Snowflake to Iceberg Smooth Migration

A practical guide for senior data engineers expanding Snowflake into a multi-engine Iceberg lakehouse. Covers five production tools — LakeOps, managed Iceberg, Open Catalog sync, Spark, and AWS Glue — with migration patterns, operational trade-offs, and a phased rollout sequence.

David W
David W
17 min read
Annual cloud bill infographic showing Iceberg lakehouse spend doubling year over year — FinOps and cost reduction framing for data platform teams in 2026
FinOpsApache IcebergLakeOpsCloud Cost

State of Iceberg FinOps and Cost Reduction in 2026

State of Iceberg FinOps in 2026: where lakehouse spend leaks, what to measure, how autonomous management and optimization are replacing manual maintenance — and a practical survey of tools from cloud optimizers to control planes.

David W
David W
24 min read
Iceberg lakehouse cost reduction — cost waste flows through LakeOps autonomous operations to deliver 80% savings
Apache IcebergLakeOpsCloud CostFinOps

7 Iceberg Lakehouse Cost Reduction Strategies

Iceberg lakehouses silently accumulate cost from small files, dead snapshots, orphan data, unoptimized layouts, and over-provisioned compute. Seven practical strategies — from deploying an autonomous control plane to leveraging partition evolution — that production data teams use to cut lakehouse spend by up to 80%.

Jonathan Saring
Jonathan Saring
9 min read
LakeOps table metrics showing records distribution, file size distribution, and table size growth over the last 30 days
Apache IcebergLakeOpsFinOpsData Platforms

Autonomous Iceberg Table Maintenance for Data Lakes

Iceberg tables need continuous maintenance — compaction, snapshot expiration, manifest optimization, and orphan cleanup — but manual scripts break at scale. A deep look at what autonomous table maintenance means in practice: how telemetry-driven orchestration replaces reactive firefighting and keeps every table healthy without human intervention.

Rob M
Rob M
16 min read
LakeOps measured results on real Iceberg workloads: 95% faster compaction, 12x query performance improvement, 80% cost reduction
Apache IcebergLakeOpsCloud CostFinOps

Apache Iceberg Cost Optimization in 2026

Your Iceberg lake is overcharging you from four directions at once — storage bloat, query compute waste, compaction overhead, and engineering time. This post breaks down exactly where each dollar goes and how autonomous table management eliminates the waste without touching your pipelines.

David W
David W
22 min read
LakeOps dashboard showing optimization activity, key metrics, and recent operations across production Iceberg tables
Apache IcebergLakeOpsFinOpsData Platforms

Managed Iceberg in 2026: Autonomous Data Lake

Iceberg tables degrade silently — small files pile up, snapshots bloat metadata, and query latency creeps higher. A breakdown of the nine components every production data lake needs to stay healthy — starting with observability and telemetry collection, through compaction, snapshot management, and lake-wide policies, to multi-engine routing and agentic AI enablement.

Jonathan Saring
Jonathan Saring
23 min read
From 350TB to 230TB in 10 Minutes: The Hidden Weight of Stale DataExternal
Apache IcebergData LakeLakeOpsCloud Cost

From 350TB to 230TB in 10 Minutes: The Hidden Weight of Stale Data

See how a 350TB data lake shrank to 230TB in 10 minutes by removing stale data—saving 34% in AWS S3 costs and proving the need for a control plane.

Amit Gilad
5 min read