README
¶
Marmot
Discover any data asset across your entire org in seconds
Open-source catalog for all your data assets. Search everything - tables, topics, queues, buckets, and more.
What is Marmot?
Marmot is an open-source data catalog designed for teams who want powerful data discovery without enterprise complexity. Built with a focus on simplicity and speed, Marmot helps you catalog assets across your entire data stack - from databases and APIs to message queues and data pipelines.
Unlike traditional catalogs that require extensive infrastructure and configuration, Marmot ships as a single binary with an intuitive UI, making it easy to deploy and start cataloging in minutes.
Built for Modern Data Teams
- Deploy in Minutes: Single binary, Docker, or Kubernetes - no complex setup required
- Powerful Search: Powerful query language with full-text, metadata, and boolean operators
- Track Lineage: Interactive dependency graphs to understand data flows and impact
- Flexible Integrations: CLI, REST API, Terraform, and Pulumi - catalog assets your way
- Lightweight: PostgreSQL-backed with minimal resource requirements
Key Features
Search Everything
Find any data asset across your entire organisation in seconds. Combine full-text search with structured queries using metadata filters, boolean logic, and comparison operators.
Interactive Lineage Visualisation
Trace data flows from source to destination with interactive dependency graphs. Understand upstream and downstream dependencies, identify bottlenecks, and analyse impact before making changes.
Metadata-First Architecture
Store rich metadata for any asset type. From tables and topics to APIs and dashboards.
Team Collaboration
Assign ownership, document business context, and create glossaries. Keep your entire team aligned with centralised knowledge about your data assets.
Quick Start
New to Marmot? Follow the Quickstart Guide for a guided setup.
Interested in exploring Marmot? Check out the live demo
Development
See Local Development for how to get started developing locally.
Contributing
All types of contributions are encouraged and valued!
Ways to Contribute:
- Report bugs or suggest features via GitHub Issues
- Improve documentation
- Build new plugins for data sources
Before contributing, please check out the Contributing Guide.
License
Marmot is open-source software licensed under the MIT License.
Directories
¶
| Path | Synopsis |
|---|---|
|
Package docs Code generated by swaggo/swag.
|
Package docs Code generated by swaggo/swag. |
|
internal
|
|
|
docgen/cmd
command
|
|
|
plugin/providers/asyncapi
+marmot:name=AsyncAPI +marmot:description=This plugin enables fetching data from AsyncAPI specifications.
|
+marmot:name=AsyncAPI +marmot:description=This plugin enables fetching data from AsyncAPI specifications. |
|
plugin/providers/bigquery
+marmot:name=BigQuery +marmot:description=This plugin discovers datasets and tables from Google BigQuery projects.
|
+marmot:name=BigQuery +marmot:description=This plugin discovers datasets and tables from Google BigQuery projects. |
|
plugin/providers/kafka
+marmot:name=Kafka +marmot:description=This plugin discovers Kafka topics from Kafka clusters.
|
+marmot:name=Kafka +marmot:description=This plugin discovers Kafka topics from Kafka clusters. |
|
plugin/providers/mongodb
+marmot:name=MongoDB +marmot:description=This plugin discovers databases and collections from MongoDB instances.
|
+marmot:name=MongoDB +marmot:description=This plugin discovers databases and collections from MongoDB instances. |
|
plugin/providers/mysql
+marmot:name=MySQL +marmot:description=This plugin discovers databases and tables from MySQL instances.
|
+marmot:name=MySQL +marmot:description=This plugin discovers databases and tables from MySQL instances. |
|
plugin/providers/openapi
+marmot:name=OpenAPI +marmot:description=This plugin discovers OpenAPI v3 specifications.
|
+marmot:name=OpenAPI +marmot:description=This plugin discovers OpenAPI v3 specifications. |
|
plugin/providers/postgresql
+marmot:name=PostgreSQL +marmot:description=This plugin discovers databases and tables from PostgreSQL instances.
|
+marmot:name=PostgreSQL +marmot:description=This plugin discovers databases and tables from PostgreSQL instances. |
|
plugin/providers/s3
+marmot:name=S3 +marmot:description=This plugin discovers S3 buckets from AWS accounts.
|
+marmot:name=S3 +marmot:description=This plugin discovers S3 buckets from AWS accounts. |
|
plugin/providers/sns
+marmot:name=SNS +marmot:description=This plugin discovers SNS topics from AWS accounts.
|
+marmot:name=SNS +marmot:description=This plugin discovers SNS topics from AWS accounts. |
|
plugin/providers/sqs
+marmot:name=SQS +marmot:description=This plugin discovers SQS queues from AWS accounts.
|
+marmot:name=SQS +marmot:description=This plugin discovers SQS queues from AWS accounts. |