Metaxy Integrations¶
Orchestration¶
-
Orchestration • Data Platform
Seamlessly integrate Metaxy with Dagster with the power of
@metaxifyand theMetaxyIOManager.Recommended
Metaxy has been built with Dagster in mind. This integration is the best way to organize, materialize and observe multiple Metaxy features at scale.
Metadata Stores¶
Learn more about metadata stores here.
-
Database
Use Google BigQuery - scalable serverless analytical database on GCP.
-
Database
Leverage the lightning-fast analytical ClickHouse database for large metadata volume and high-throughput setups.
Recommended
Ideal for production.
-
Storage
Store metadata in Delta Lake format in local files or remote object stores (S3, GCS, and others). (1)
Recommended
Ideal for dev environments.
-
Database • Storage
Use DuckDB - a fast analytical database with support for local and remote compute. DuckLake is available as well.
Warning
DuckDB is not recommended for production due to parallel writes limitations.
-
Database • Storage
Use the multi-modal LanceDB database or Lance storage format. (2)
-
uses a local versioning engine implemented in Polars and
polars-hash -
uses a local versioning engine implemented in Polars and
polars-hash
Plugins¶
-
ORM • Database
Retrieve SQLAlchemy URLs and
MetaDatafor the current Metaxy project from MetaxyMetadataStoreobjects. -
ORM • Database
Adds
SQLModelcapabilities tometaxy.BaseFeatureclass.