Authors: Akshaya Jayakumar & Srutimala Deka & Sanskriti Tripathi
In this blog, we will first take a look at what the metrics layer is and where it fits into our data stack. Then we will have a brief look at all the available metrics tools and move on to two specific tools - Metriql and Transform.
The comparison chart between the two will aid in selecting the best tool for any particular use case.
What is the Metrics Layer
Metrics are an essential part of analytics. These are the numbers that indicate the overall performance of a business or an organization.
In the traditional (and widely used) scenario, any tool which makes use of metrics, say a visualization tool or a CRM software, would have its own metrics definition. These are not portable to other tools which might use the same set of metrics in a different way.
So there comes the point when the metrics or KPIs have evolved over time. The numerous tools using those metrics would either fail to remain updated, or it may not be consistent across all the consumption tools. This can heavily affect large businesses when decisions are made swiftly after one look at the numbers.
That’s where the metrics layer comes in!
It is a single layer that sits between the consumption layer and the data source. This layer is nothing more than a metrics store where metrics are defined, managed, and cataloged by a selective group of data experts.
A data-driven organization would employ its data engineers, analysts, and business experts to collaboratively define the metrics they are most interested in.
When is it time to adopt a metrics layer?
In case the following conditions are met, one may consider introducing a metrics layer in their data stack.
The collection of consumption tools is getting larger and mismanaged.
There are key metrics that are reused in most of the downstream tools.
Large organizations have a mature data team.
Introduction to some BI-Metrics tools currently in the spotlight
LookML: LookML is a tool from Looker that allows analysts to collaborate, test, and document data. It is an enterprise product. LookML’s primary advantage lies in providing a framework for converting every metric request into a query to extract that metric from the database consistently and accurately.
Airbnb Minerva: Airbnb has come up with its own metric tool that addresses its internal metrics inconsistency problem. It is custom-made for querying customer data uniformly with a user interface and metrics store.
Transform: Transform is a brand new metric tool that provides not only a metric layer but also a UI and APIs for consumption tools to use those metrics. It has an open-source component (MetricFlow) and a 14-day trial to explore the UI and some APIs.
Metriql: Metriql is a metrics store built on top of DBT. This allows users to build, test, and document data by leveraging DBT’s features. Metriql is open-source and allows integration with many consumption tools.
Comparison Between Metriql and Transform
​ | Criteria | Description | Metriql | Transform | Comments |
Product Offerings | End to End suite | Components for every facet of the tool | No | Yes | Transform has 3 components - the core metric layer, the UI/catalog, APIs for downstream tools |
Product Offerings | Querying method | Metrics retrieval approach | Uses subset of ANSI SQL, Trino's query interface | Uses its own MQL Server and UI for querying; CLI querying is also possible in local system | ​ |
Product Offerings | Saas offering | Software packages that are available to the end users on the cloud | No | No | Both Metriql and MetricFlow component require on-prem installation |
Requirements to use the tool | Memory Requirement | System memory requirements for local installation | 64-bit processor, 4GB for Windows 10 and above | 2GB - 4 GB for local installations and Windows 8 and above | ​ |
Requirements to use the tool | Data Source/ Warehouse Supported | Snowflake | Yes | Yes | ​ |
Requirements to use the tool | Data Source/ Warehouse Supported | Redshift | Yes | Yes | ​ |
Requirements to use the tool | Data Source/ Warehouse Supported | Big Query | Yes | Yes | ​ |
Requirements to use the tool | Data Source/ Warehouse Supported | Postgres | Yes | No | ​ |
Requirements to use the tool | Data Source/ Warehouse Supported | Presto | Yes | No | ​ |
Pricing | Offering type | Licensed or Open Source | Open Source | The Metric Layer is Open Source ; Catalog and API are not. | 1.Metriql uses DBT, whose CLI is open source and free to use but DBT Cloud is licensed. |
Pricing | License | ​ | Apache 2.0 | AGPL | ​ |
Pricing | Licensed Cost/Pricing model | Cost and credits required to use the tool | Metriql is free of cost and incurs cost of DBT DBT CLI is free DBT cloud has 14 days free trial period | Transform UI and APIs comes with 14 day trial with limited functionality | DBT: Subscription based on pack of users 1.Developer users - > $1000/user/month 2.Operator users $500/10-user/month |
Pricing | GUI | ​ | Yes; it has dashboard for integrations | Yes; It is the catalog component | ​ |
Pricing | CLI | ​ | Trino's CLI | Local CLI | ​ |
​ | Criteria | Description | Metriql | Transform | Comments |
Downstream Integration | BI Tools | ​ | Mode Analytics, Tableau, Thoughtspot, Google Data Studio, Looker and many more | Mode Analytics, Tableau, Python | ​ |
Downstream Integration | Third Party Tools | ​ | Google Sheets, Jetbrains DataGrip, Dbeaver | Google Sheets, Slack, Excel | ​ |
Compute | ​ | In reference to processing power, memory and other resources required for the computation | DBT - Data Warehouse | Source Data Warehouse compute | ​ |
Metrics Storage | ​ | Storing metrics definitions or intermediate tables used for computation | Source Data Warehouse | Source Data Warehouse | ​ |
Quality Assurance | Automated Testing | Leveraging automation tools to maintain, execute tests, and analyze data | Yes | No | DBT creates automated test cases like not null, unique, etc. |
Quality Assurance | Data Model Validation | ​ | Yes | Yes | Validation of transform model after configuring yaml file (from CLI) |
Data Governance | Data Lineage | Tracking the source and flow of data from sources to consumption | Yes | Yes | Transform UI allows annotation and tracking of metrics when changes are made. Metriql makes use of dbt Cloud to understand the data flow |
Data Governance | Documentation | Descriptive information about the data | Yes | Yes | In DTB Great documentation is created for each model with DAG |
Data Security | ​ | ​ | Same as the connected Warehouse security + DTB | Same as the connected Warehouse security | ​ |
Conclusion
Metriql and Transform are two tools with quite similar methods of defining metrics, namely, yaml files.
Querying metrics requires no new language to be picked up by the user. This is true for both Metriql and Transform.
For a team of technical and non-technical professionals, Transform’s interface acts as an easy-to-use platform for collaboration.
To leverage the advantages of DBT, Metriql would be the tool to adopt.