Apache Iceberg is an innovative data lakehouse table format designed to revolutionize how you manage large-scale data across various storage layers, such as Hadoop and AWS S3. By treating these diverse storage solutions as a cohesive, universal database, Apache Iceberg facilitates seamless integration with numerous tools, platforms, and interfaces, enhancing both flexibility and accessibility.
One of the standout features of Apache Iceberg is its support for ACID transactions, which ensures data integrity by enabling atomicity, consistency, isolation, and durability in data processing operations. Additionally, Apache Iceberg’s Time Travel capability allows users to query historical versions of data, providing valuable insights into data changes over time. The format also supports dynamic schema evolution and partitioning adjustments without downtime, which simplifies the management of data as it grows and changes.
Apache Iceberg also includes a host of other advanced features that enhance data usability and management. These include snapshot isolation for concurrent data access, upserts and deletes within tables, and handling large-scale metadata efficiently.
To help you harness the full potential of Apache Iceberg, our latest blog series gathers a collection of straightforward, hands-on tutorials.
These guides are designed to provide you with practical experience in working with Apache Iceberg, from setting up your first table to executing complex data operations. Whether you’re a data scientist, engineer, or analyst, these tutorials offer a valuable opportunity to enhance your data handling skills and leverage Apache Iceberg’s powerful features in your projects.
Self-Contained Exercises
These are completely self-contained and can be done from your laptop; all infrastructure is spun up as Docker containers.
End-to-End Basic Data Engineering Tutorial (Spark, Dremio, Superset)
From Postgres to Dashboards with Dremio and Apache Iceberg
From SQLServer to Dashboards with Dremio and Apache Iceberg
From MongoDB to Dashboards with Dremio and Apache Iceberg
Intro to Dremio, Nessie, and Apache Iceberg on Your Laptop
Using Flink with Apache Iceberg and Nessie
Getting Started with Project Nessie, Apache Iceberg, and Apache Spark Using Docker
Require Online Services
These are tutorials that will require signing up for services like Upsolver, Dremio, AWS, and others to complete.
How to Convert JSON Files Into an Apache Iceberg Table with Dremio
How to Convert CSV Files into an Apache Iceberg table with Dremio
Run Graph Queries on Apache Iceberg Tables with Dremio & Puppygraph
BI Dashboards with Apache Iceberg Using AWS Glue and Apache Superset
Streaming and Batch Data Lakehouses with Apache Iceberg, Dremio and Upsolver
Git for Data with Dremio’s Lakehouse Catalog: Easily Ensure Data Quality in Your Data Lakehouse
How to Create a Lakehouse with Airbyte, S3, Apache Iceberg, and Dremio
Video Playlists:
Apache Iceberg Lakehouse Engineering 101
In summary, Apache Iceberg is more than just a data storage format; it is a comprehensive solution that adapts to the complexities of modern data environments, ensuring robust data integrity, flexibility, and scalability.