Catalogs and Query Engines

Tableflow has direct integration with some external catalogs. Additionally, most catalogs have some concept of an "external table" where the catalog will periodically scrape the object storage bucket where the table lives as the mode of integration, or fetch the latest metadata on-demand when running a query. For example, Snowflake calls this the "Iceberg files" integration method.

Snowflake Documentation: https://docs.snowflake.com/en/sql-reference/sql/create-iceberg-table-iceberg-files

Similar to catalogs, most query engines have a way to specify the location of a table in object storage and query that directly without going through a catalog. This method can be applied to ClickHouse and DuckDB, for example.