Snowflake Iceberg Tables, Data Sharing, and the Open Lakehouse: External Volumes, Polaris Catalog, Zero-Copy Cloning, Time Travel, Cross-Cloud Replication, and Multi-Engine Interoperability

Table of Contents

• The Open Lakehouse Problem
• Apache Iceberg Tables in Snowflake
• External Volumes — Connecting to Cloud Storage
• Snowflake-Managed vs Externally-Managed Iceberg Tables
• Polaris Catalog — The Open Catalog for Iceberg
• Zero-Copy Cloning
• Time Travel
• Data Sharing — Snowflake’s Killer Feature
• Cross-Cloud and Cross-Region Replication
• Iceberg vs Native Tables — When to Use Which
• Common Mistakes
• Interview Questions
• Wrapping Up

In the previous post, we covered Snowpark for Python. So far, everything we have built lives inside Snowflake’s proprietary storage — micro-partitions that only Snowflake can read. But what if your Spark jobs on Databricks also need to read that data? What if a partner company on a different cloud needs access to your tables? What if you want to avoid vendor lock-in by storing data in an open format?

This post covers Snowflake’s answer to all three questions: Iceberg tables for open format interoperability, data sharing for zero-copy collaboration, and cloning and Time Travel for instant copies and point-in-time recovery.

Analogy — A universal power adapter. Snowflake’s native tables are like electronics designed for one country’s outlets — they work perfectly within Snowflake, but nothing else can plug in. Iceberg tables are like building your electronics with a universal power adapter — Snowflake, Spark, Trino, Flink, and any Iceberg-compatible engine can all plug into the same data without converters. Data sharing is like giving a friend a key to your apartment — they can come in and use your kitchen (query your data) without you copying any groceries (zero data movement). And Time Travel is a time machine for your data — accidentally drop a table on Tuesday and travel back to Monday to get it back.

The Open Lakehouse Problem

The problem with proprietary storage:

  Snowflake native tables:
    - Data stored in Snowflake's proprietary micro-partition format
    - Only Snowflake can read them
    - To use with Spark/Databricks: UNLOAD to Parquet -> copy to S3 -> read in Spark
    - To share with a partner: export CSV/Parquet -> transfer via S3 -> partner loads it
    - Vendor lock-in: migrating away means exporting everything

  The open format solution:
    - Store data in Apache Iceberg format (Parquet files + Iceberg metadata)
    - Any engine can read: Snowflake, Spark, Trino, Flink, Presto
    - No data copying between engines
    - No vendor lock-in -- your data is in open Parquet files on your cloud storage
    - Snowflake adds ACID transactions, Time Travel, and governance on top

Apache Iceberg Tables in Snowflake

Apache Iceberg is an open table format that provides ACID transactions, schema evolution, time travel, and partition evolution on top of Parquet data files. Snowflake added Iceberg support so you can create tables that live in your own cloud storage (S3, Azure Blob, GCS) in a format that any Iceberg-compatible engine can read.

-- Create an Iceberg table managed by Snowflake
CREATE ICEBERG TABLE ANALYTICS_DB.CORE.ORDERS_ICEBERG (
  order_id INTEGER,
  customer_name STRING,
  amount DECIMAL(10,2),
  order_date DATE,
  region STRING
)
  CATALOG = 'SNOWFLAKE'
  EXTERNAL_VOLUME = 'my_s3_volume'
  BASE_LOCATION = 'orders_iceberg/';

-- Insert data (same SQL as native tables)
INSERT INTO ANALYTICS_DB.CORE.ORDERS_ICEBERG
SELECT * FROM ANALYTICS_DB.CORE.ORDERS;

-- Query (identical to native tables)
SELECT region, COUNT(*), SUM(amount)
FROM ANALYTICS_DB.CORE.ORDERS_ICEBERG
GROUP BY region;

-- The data is now stored as Parquet files on YOUR S3 bucket
-- Spark, Trino, or any Iceberg engine can read it directly

What Makes Iceberg Tables Different from Native Tables

Native Snowflake tables:
  - Data in proprietary micro-partition format
  - Stored in Snowflake-managed storage
  - Only Snowflake can read/write
  - Full feature set (clustering, search optimization, etc.)

Iceberg tables:
  - Data in open Parquet format with Iceberg metadata
  - Stored in YOUR cloud storage (S3, Blob, GCS)
  - Any Iceberg engine can read (Spark, Trino, Flink, Presto)
  - Snowflake provides ACID, Time Travel, governance on top
  - Most Snowflake features work (DML, streams, Dynamic Tables)

External Volumes — Connecting to Cloud Storage

An external volume connects Snowflake to your cloud storage where Iceberg data files live. You create it once and reference it when creating Iceberg tables.

-- Create an external volume for S3
CREATE EXTERNAL VOLUME my_s3_volume
  STORAGE_LOCATIONS = (
    (
      NAME = 'primary_s3'
      STORAGE_BASE_URL = 's3://my-iceberg-bucket/data/'
      STORAGE_PROVIDER = 'S3'
      STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::123456789012:role/snowflake-iceberg-role'
    )
  );

-- Create an external volume for Azure
CREATE EXTERNAL VOLUME my_azure_volume
  STORAGE_LOCATIONS = (
    (
      NAME = 'primary_azure'
      STORAGE_BASE_URL = 'azure://myaccount.blob.core.windows.net/iceberg-data/'
      STORAGE_PROVIDER = 'AZURE'
      AZURE_TENANT_ID = 'your-tenant-id'
    )
  );

-- Describe volume to get IAM details for trust policy setup
DESC EXTERNAL VOLUME my_s3_volume;

Snowflake-Managed vs Externally-Managed Iceberg Tables

Snowflake supports two modes for Iceberg tables:

-- Snowflake-managed (Snowflake writes, others read)
CREATE ICEBERG TABLE orders_sf_managed (...)
  CATALOG = 'SNOWFLAKE'
  EXTERNAL_VOLUME = 'my_s3_volume'
  BASE_LOCATION = 'orders/';

-- Externally-managed via AWS Glue catalog
CREATE ICEBERG TABLE orders_ext_managed
  EXTERNAL_VOLUME = 'my_s3_volume'
  CATALOG = 'my_glue_catalog_integration'
  CATALOG_TABLE_NAME = 'orders';

-- Externally-managed via Polaris (REST catalog)
CREATE ICEBERG TABLE orders_polaris
  EXTERNAL_VOLUME = 'my_s3_volume'
  CATALOG = 'my_polaris_integration'
  CATALOG_TABLE_NAME = 'orders';

-- Refresh metadata for externally-managed tables
-- (when external engine writes new data)
ALTER ICEBERG TABLE orders_ext_managed REFRESH;

Decision rule:

  If Snowflake is the system of record (you write data here):
    -> Snowflake-managed Iceberg table
    -> Full DML, Time Travel, streams, clustering
    -> Other engines read via Iceberg format

  If another engine writes the data (Spark, Flink, CDC tool):
    -> Externally-managed Iceberg table
    -> Snowflake reads only
    -> Metadata refresh needed when external engine writes

Polaris Catalog — The Open Catalog for Iceberg

Polaris (donated by Snowflake to Apache Software Foundation in 2024) is an open-source REST catalog for Apache Iceberg. It provides a single catalog that multiple engines can use to discover and access Iceberg tables.

Why Polaris matters:

  Without Polaris:
    Snowflake has its own catalog
    Spark has its own catalog (Hive Metastore or Unity Catalog)
    Trino has its own catalog
    Each engine maintains separate metadata about the same tables
    Sync issues, stale metadata, inconsistent access control

  With Polaris:
    ONE catalog that all engines connect to
    Snowflake, Spark, Trino, Flink all see the same tables
    One set of permissions (RBAC)
    No metadata sync needed
    Engine-agnostic -- not tied to any vendor

  Snowflake Open Catalog:
    Snowflake's hosted version of Polaris
    Managed service (no infrastructure to run)
    Integrates natively with Snowflake
    Also accessible from Spark, Trino, Flink via REST API

Zero-Copy Cloning

Zero-copy cloning creates an instant metadata-only copy of a table, schema, or database. No data is physically copied — the clone shares the same underlying micro-partitions (or Iceberg data files). Storage costs are zero until data in the clone or original diverges.

Analogy — A shared photo album. Cloning is like creating a shortcut to a shared photo album. Both the original and the clone see the same photos. If you edit a photo in the clone (update a row), only the edited photo gets a new copy — all other photos remain shared. You do not pay for storage until you start making changes.

-- Clone a table (instant, zero storage cost)
CREATE TABLE orders_dev CLONE orders;

-- Clone with a point-in-time (Time Travel + clone)
CREATE TABLE orders_yesterday CLONE orders AT(OFFSET => -86400);

-- Clone an entire schema
CREATE SCHEMA dev_staging CLONE ANALYTICS_DB.STAGING;

-- Clone an entire database
CREATE DATABASE DEV_DB CLONE ANALYTICS_DB;

-- After cloning:
-- DEV_DB has all schemas, tables, views from ANALYTICS_DB
-- Storage cost: $0 (until data diverges)
-- Time: seconds (even for terabytes)

Zero-copy clone use cases:

  1. Development copies:
     Clone production -> develop on clone -> test -> promote changes
     No risk to production data, instant setup

  2. Pre-migration backup:
     CLONE the table before running a risky migration script
     If migration fails: DROP original, rename clone

  3. Analyst sandboxes:
     Give analysts their own clone to experiment freely
     They can INSERT, UPDATE, DELETE without affecting production

  4. Testing:
     Clone production data for integration tests
     Tear down after tests complete (DROP clone)

  5. Point-in-time snapshots:
     Clone AT a specific timestamp for auditing or compliance

Time Travel

Time Travel lets you query, clone, or restore data as it existed at any past point in time. Standard edition supports 1 day. Enterprise edition supports up to 90 days.

-- Query data as it was 1 hour ago
SELECT * FROM orders AT(OFFSET => -3600);

-- Query data at a specific timestamp
SELECT * FROM orders AT(TIMESTAMP => '2026-07-19 10:00:00'::TIMESTAMP);

-- Query data before a specific query was executed
SELECT * FROM orders BEFORE(STATEMENT => '01b2c3d4-0000-0000-0000-000000000001');

-- Restore an accidentally dropped table
DROP TABLE orders;          -- Oops
UNDROP TABLE orders;        -- Saved (within Time Travel retention period)

-- Restore an accidentally dropped database
DROP DATABASE ANALYTICS_DB;
UNDROP DATABASE ANALYTICS_DB;

-- Restore data after a bad UPDATE
-- Step 1: See what the data looked like before
SELECT * FROM orders AT(OFFSET => -600) WHERE order_id = 12345;

-- Step 2: Restore by creating a clone from the past
CREATE TABLE orders_restored CLONE orders AT(OFFSET => -600);

-- Step 3: Merge restored data back
INSERT OVERWRITE INTO orders
SELECT * FROM orders_restored;
DROP TABLE orders_restored;

Time Travel Configuration

-- Set retention period (Enterprise: up to 90 days)
ALTER TABLE orders SET DATA_RETENTION_TIME_IN_DAYS = 90;

-- Check current retention
SHOW TABLES LIKE 'orders';
-- Look for the "retention_time" column

-- Transient tables: max 1 day Time Travel, no Fail-safe
CREATE TRANSIENT TABLE staging_temp (...)
  DATA_RETENTION_TIME_IN_DAYS = 1;

Time Travel vs Fail-safe:

  Time Travel (1-90 days, configurable):
    - YOU can query and restore data
    - Uses UNDROP, AT, BEFORE, CLONE
    - Storage cost: retains old micro-partitions

  Fail-safe (7 days, automatic, not configurable):
    - Only SNOWFLAKE SUPPORT can restore data
    - Activates AFTER Time Travel expires
    - Last resort for disaster recovery
    - Additional storage cost

  Timeline:
    Day 0: data modified
    Day 1-90: Time Travel (you can restore)
    Day 91-97: Fail-safe (Snowflake support only)
    Day 98+: data permanently gone

Data Sharing — Snowflake’s Killer Feature

Data sharing lets you share live, read-only data with other Snowflake accounts without copying data. The consumer sees the data as if it were in their own account, always up-to-date, always consistent. Zero data movement, zero storage duplication.

Analogy — A library card for another library. Instead of photocopying books (exporting CSVs) and mailing them to another library (uploading to their system), you give them a library card (share) that lets them walk into your library and read any book you have authorized. The books never leave your shelves. Every time they visit, they see the latest editions. And you can revoke the card anytime.

-- PROVIDER side: create a share
CREATE SHARE revenue_share
  COMMENT = 'Monthly revenue data for partner analytics';

-- Grant access to specific objects
GRANT USAGE ON DATABASE ANALYTICS_DB TO SHARE revenue_share;
GRANT USAGE ON SCHEMA ANALYTICS_DB.MARTS TO SHARE revenue_share;
GRANT SELECT ON TABLE ANALYTICS_DB.MARTS.REVENUE_MONTHLY TO SHARE revenue_share;
GRANT SELECT ON VIEW ANALYTICS_DB.MARTS.CUSTOMER_SUMMARY TO SHARE revenue_share;

-- Add consumer accounts
ALTER SHARE revenue_share ADD ACCOUNTS = partner_account_id;

-- CONSUMER side: create a database from the share
CREATE DATABASE partner_revenue FROM SHARE provider_account.revenue_share;

-- Query shared data (always live, always current)
SELECT * FROM partner_revenue.MARTS.REVENUE_MONTHLY
WHERE month = '2026-07';

Listing — Snowflake Marketplace

Snowflake Marketplace:

  Public listings:
    - Share data publicly with any Snowflake customer
    - Free or paid listings
    - Examples: weather data, financial data, COVID data, demographics

  Private listings:
    - Share with specific accounts only
    - Used for partner data exchanges, inter-company sharing
    - Consumer sees a private listing in their Marketplace tab

  Data products:
    - Combine shared data with Snowpark UDFs, stored procedures, notebooks
    - Share not just data but also logic and applications

Direct Sharing vs Listing

Direct Share:
  - Provider creates a SHARE object
  - Adds specific consumer accounts
  - Consumer creates a database from the share
  - Best for: known partners, internal cross-account sharing

Listing (Marketplace):
  - Provider creates a listing (public or private)
  - Consumers discover and subscribe via Marketplace UI
  - Metadata, descriptions, sample queries included
  - Best for: data products, public datasets, many consumers

Cross-Cloud and Cross-Region Replication

Snowflake supports replicating databases, shares, and account objects across regions and clouds. This enables disaster recovery, data locality, and cross-cloud data sharing.

-- Enable replication for a database
ALTER DATABASE ANALYTICS_DB ENABLE REPLICATION TO ACCOUNTS
  my_org.account_eu_west    -- EU region
  my_org.account_ap_south;  -- Asia-Pacific region

-- On the target account: create a replica
CREATE DATABASE ANALYTICS_DB_REPLICA AS REPLICA OF
  my_org.source_account.ANALYTICS_DB;

-- Refresh the replica (pull latest changes)
ALTER DATABASE ANALYTICS_DB_REPLICA REFRESH;

-- Automate refresh with a task
CREATE TASK refresh_replica
  SCHEDULE = '60 MINUTE'
AS
  ALTER DATABASE ANALYTICS_DB_REPLICA REFRESH;

Replication use cases:

  Disaster recovery:
    Replicate to a different region
    If primary fails, promote replica

  Data locality:
    Replicate to regions closer to consumers
    EU analysts query EU replica (lower latency)

  Cross-cloud sharing:
    Primary on AWS, replica on Azure
    Azure consumers query the Azure replica

  Cost note:
    Replication charges for data transfer between regions/clouds
    Storage is charged in both source and target accounts

Iceberg vs Native Tables — When to Use Which

Decision guide:

  Use NATIVE tables when:
    - Snowflake is your only analytics engine
    - Maximum query performance is the priority
    - You need all Snowflake features (search optimization, zero-copy clone)
    - Vendor lock-in is not a concern

  Use ICEBERG tables when:
    - Multiple engines need to read the same data (Spark, Trino, Flink)
    - You want data in open format on your own storage
    - You are building an open lakehouse architecture
    - You want the option to migrate away from Snowflake
    - Another engine (Spark, CDC tool) writes data that Snowflake reads

Common Mistakes

1. Using Iceberg tables when native tables would be faster and simpler. If Snowflake is your only analytics engine and you do not need multi-engine access, native tables are faster (more optimization options) and simpler (no external volume setup). Iceberg adds complexity for interoperability — only use it when you need that interoperability.

2. Not refreshing externally-managed Iceberg tables. When an external engine (Spark) writes new data to an externally-managed Iceberg table, Snowflake does not know about it until you run ALTER ICEBERG TABLE … REFRESH. Without this, queries return stale data. Automate the refresh with a task.

3. Confusing data sharing with data copying. Data sharing does NOT copy data. The consumer queries live data in the provider’s account. This means the provider’s warehouse is NOT used — the consumer uses their own warehouse to query shared data. Changes by the provider are instantly visible to the consumer.

4. Forgetting that Time Travel has storage costs. Every UPDATE and DELETE creates new micro-partitions while retaining old ones for the Time Travel retention period. A table with frequent updates and 90-day retention accumulates significant storage. Set retention appropriately — 1 day for staging tables, 7-30 days for most production tables, 90 days only for critical audit tables.

5. Not using zero-copy clones for development. Teams that export production data to CSVs and import into dev environments waste time and storage. A zero-copy clone is instant, free until data diverges, and provides a perfect replica of production — including table structure, data, and statistics.

6. Using data sharing without understanding the consumer experience. Shared data appears as a read-only database in the consumer’s account. The consumer cannot modify the data, create views on top of it (without first creating their own database with views referencing shared tables), or grant access to sub-objects. Communicate these limitations to consumers upfront.

7. Not considering data transfer costs for cross-region replication. Replicating from us-east-1 to eu-west-1 charges per GB transferred. For large databases with frequent changes, these costs add up. Use replication only when data locality or disaster recovery justifies the cost. For one-off analysis, consider having the analyst connect to the source region instead.

8. Creating Iceberg tables without a clear multi-engine use case. Some teams adopt Iceberg because it is the latest trend. If no one outside Snowflake reads your data, Iceberg adds external volume management complexity for no benefit. Adopt Iceberg when you have a concrete multi-engine requirement.

Interview Questions

Q: What are Apache Iceberg tables in Snowflake and why would you use them? A: Iceberg tables store data in the open Apache Iceberg format (Parquet data files + Iceberg metadata) on your own cloud storage (S3, Azure Blob, GCS). Unlike native Snowflake tables (proprietary format), Iceberg tables can be read by any Iceberg-compatible engine: Spark, Trino, Flink, Presto. Use Iceberg tables when you need multi-engine access to the same data, want to avoid vendor lock-in, or need to integrate with an existing data lake that uses Iceberg. Snowflake adds ACID transactions, Time Travel, and governance on top of the open format.

Q: What is the difference between Snowflake-managed and externally-managed Iceberg tables? A: Snowflake-managed Iceberg tables use Snowflake as the catalog and allow full DML (INSERT, UPDATE, DELETE, MERGE), Time Travel, streams, and clustering. Externally-managed tables use an external catalog (AWS Glue, Polaris) and are primarily read-only from Snowflake — another engine (Spark, Flink) writes the data. Choose Snowflake-managed when Snowflake is the primary writer. Choose externally-managed when another engine owns the data and Snowflake is a reader.

Q: What is zero-copy cloning and how does it work? A: Zero-copy cloning creates an instant, metadata-only copy of a table, schema, or database without duplicating any data. The clone shares the same underlying storage (micro-partitions) as the original. Storage cost is zero until data in the clone or original diverges — at which point, only the changed portions get new storage. Clones are used for creating development copies of production, pre-migration backups, analyst sandboxes, and point-in-time snapshots (combined with Time Travel: CLONE … AT).

Q: How does Snowflake data sharing work? A: Data sharing creates a read-only, zero-copy connection between a provider account and one or more consumer accounts. The provider creates a SHARE object and grants SELECT on specific tables and views. The consumer creates a database from the share and queries the data using their own warehouse. No data is copied — the consumer reads live data from the provider’s storage. Changes by the provider are instantly visible. The consumer pays only for their compute (warehouse credits), not for the shared data storage.

Q: What is Time Travel and how does it differ from Fail-safe? A: Time Travel lets users query, clone, or restore data as it existed at any past point in time. Standard edition supports 1 day; Enterprise supports up to 90 days. Users access it via AT(OFFSET), AT(TIMESTAMP), BEFORE(STATEMENT), UNDROP, and CLONE AT. Fail-safe is a separate 7-day recovery window that activates after Time Travel expires. Only Snowflake support can recover data during Fail-safe. Both retain old micro-partitions, so both have storage costs.

Q: What is Polaris Catalog and how does it relate to Iceberg? A: Polaris is an open-source REST catalog for Apache Iceberg, originally created by Snowflake and donated to the Apache Software Foundation. It provides a single catalog that multiple engines (Snowflake, Spark, Trino, Flink) can use to discover and access Iceberg tables. This eliminates the need for separate catalogs per engine and ensures consistent metadata and access control. Snowflake offers a hosted version called Snowflake Open Catalog.

Q: When would you use data sharing vs data replication? A: Use data sharing when both provider and consumer are in the same Snowflake region — it is free, instant, and always current. Use replication when the consumer is in a different region or cloud — it copies data to the target region for lower-latency access. Replication has data transfer costs and a refresh delay (scheduled or manual). For cross-region sharing, Snowflake now supports listing-based sharing that handles replication automatically, but direct share with explicit replication gives more control.

Wrapping Up

Iceberg tables, data sharing, zero-copy cloning, and Time Travel represent Snowflake’s answer to the modern data platform challenges: interoperability, collaboration, agility, and resilience. Iceberg tables open Snowflake’s data to any engine. Data sharing enables zero-copy collaboration across accounts and organizations. Zero-copy cloning makes development and testing instant and free. And Time Travel provides a safety net for every accidental change.

In the next post, we will cover Snowflake performance optimization: clustering keys, search optimization, caching layers, warehouse scaling strategies, and query profiling.

Related posts: – Snowflake Overview & Architecture – Snowflake Transformations & CDC – Snowpark Python – Delta Lake Deep Dive – OneLake Shortcuts in Fabric