Launching an Internal Data Marketplace with Atlan
The Active Metadata Pioneers series features Atlan customers who have recently completed a thorough evaluation of the Active Metadata Management market. Paying forward what you’ve learned to the next data leader is the true spirit of the Atlan community! So, they’re here to share their hard-earned perspective on an evolving market, what makes up their modern data stack, innovative use cases for metadata, and more.
In this installment of the series, we meet Cristina Perez Martinez, Data Engineer and Architect, and Ezequiel Barbero, Market & Business Intelligence Manager at Telefónica Tech, who share how a modern data cataloging experience and column-level lineage will support a broad vision for data democratization.
This interview has been edited for brevity and clarity.
Could you tell us a bit about yourself, your background, and what drew you to Data & Analytics?
Ezequiel Barbero:
I’ve got a Masters in Big Data and have worked in Data & Analytics since 2002. I started at Telefónica in Argentina with the BI Data Team working on ETLs based in SQL. Then I worked in Data Engineering helping with Data Science, working with the head of that team in Argentina.
In 2019, I came to Spain to work with their Data Science team on Marketing Intelligence, and in 2021 I joined Telefónica Tech to start the BI Team.
Cristina Perez Martinez:
I started working at Telefónica in 2019 as a Python developer, and I moved to Telefónica Tech in 2021. My team has mainly been working as Data Engineers and Data Architects for the BI team.
Would you mind describing Telefónica, and how your data team supports the organization?
Cristina:
Telefónica is divided into quite a few different companies, but as a whole, it’s a Telecommunications Enterprise. Here, in Telefónica Tech, the digital business unit, we’ve been focused on digital technologies such as AI & BD, connectivity and IoT, Cybersecurity, Cloud, and Blockchain.
Our team is divided into two, with part of the team focused on architecture and engineering, getting raw data, then standardizing and transforming it until it goes into Snowflake, our Data Warehouse. The rest of the team is focused on Data Analysis, based in Snowflake and coding in SQL. From there, they develop dashboards in PowerBI.
Ezequiel:
Telefónica Tech has a team working on IoT and Big Data for external use cases, but our team is responsible for internal use cases, supporting the company. We support infrastructure, transformation, and for almost a year now, Data Governance.
What does your data stack look like?
Ezequiel:
Our stack is based on Microsoft Azure, and we use Data Factory for Orchestration. We use Databricks’ ETL tool, blob storage, and data lake. Snowflake is Telefónica’s data warehouse.
Why search for an Active Metadata Management solution? What was missing?
Ezequiel:
Our company has over 6,200 people, but our team is small relative to the entire organization. So if it’s important to improve data democratization, then that wouldn’t be possible without self-service, and without Data Governance.
Why was Atlan a good fit? Did anything stand out during your evaluation process?
Ezequiel:
We were first looking for a cloud-based SaaS solution that was easy to deploy and easy to set up.
Cristina:
Our goal was to have a place where we could create a catalog of data that was accessible enough to the rest of the company. It was also important to understand the lineage between Snowflake and PowerBI. Our primary goal was to understand the impact that modifying a source would have on our data warehouse, so column-level lineage ensures end-to-end visibility and traceability. Additionally, we recognize the need for a robust tool to strengthen security of our data platform, allowing us to assign roles and permissions to ensure that only authorized people have access to specific information, as well as the ability to perform audits which is essential to maintain the integrity and compliance of our data operations.
What do you intend on creating with Atlan? Do you have an idea of what use cases you’ll build, and the value you’ll drive?
Cristina:
One of the requirements we had is to create somewhat of a marketplace for our data, with everything based on Atlan assets, and we’re working on launching that in the beginning of this year. From there, we’re looking forward to populating a lot more metadata in Atlan and Snowflake.
In the future, we’re excited about the possibility of using Atlan AI. Our goal is to make accessing data even easier for people, and being able to chat with Atlan about data would make it easy for people to find what they need.
Photo by Mario Caruso on Unsplash
