Built for ease

Meteor is a plugin driven agent for collecting metadata. Meteor has plugins to source metadata from a variety of data stores, services and message queues. It also has sink plugins to send metadata to variety of third party APIs and catalog services.

Zero dependency

Meteor is written in Go and compiles into a single binary with no external dependencies, and requires a very minimal memory footprint.

Coverage

With 50+ plugins and many more coming soon to extract and sink metadata, it is easy to start collecting metadata from various sources.

Extensible

With the ease of plugin development you can build your own plugin to fit with your needs. It allows new sources, processors and sinks to be easily added.

CLI

Meteor comes with a CLI which allows you to interact with agent effectively. You can list all plugins, start and stop agent, and more.

Proven

Battle tested at large scale across multiple companies. Largest deployment collect metadata from thousands of data sources.

Runtime

Meteor can run from your local machine, cloud server machine or containers with minium efforts required for deployment.

Framework

Meteor agent uses recipes as a set of instructions which are configured by user. Recipes contains configurations about the source from which the metadata will be fetched, information about metadata processors and the destination to where the metadata will be sent.

Extraction

Extraction is the process of extracting data from a source and transforming it into a format that can be consumed by the agent. Extractors are the set of plugins that are source of our metadata and include databases, dashboards, users, etc.

Processing

Processing is the process of transforming the extracted data into a format that can be consumed by the agent. Processors are the set of plugins that perform the enrichment or data processing for the metadata after extraction..

Sink

Sink is the process of sending the processed data to a single or multiple destinations as defined in recipes. Sinks are the set of plugins that act as the destination of our metadata after extraction and processing is done by agent.

Ecosystem

Meteor’s plugin system allows new plugins to be easily added. With 50+ plugins and many more coming soon to extract and sink metadata, it is easy to start collecting metadata from various sources and sink to any data catalog or store.

Extractors

Meteor supports source plugins to extract metadata from a variety of datastores services, and message queues, including BigQuery, InfluxDB, Kafka, Metabase, and many others.

Processors

Meteor has in-built processors including enrichment and others. It is easy to add your own processors as well using custom plugins.

Sinks

Meteor supports sink plugins to send metadata to a variety of third party APIs and catalog services, including Compass, HTTP, BigQuery, Kafka, and many others.