AWS re:Invent is one of my favorite trade shows. It is one of the biggest technology conferences of the year and is an opportunity to have hundreds of conversations with customers and prospects, listen to their priorities and challenges, hopes, and give them a Cloudera tote bag or a pair of orange sunglasses.
What follows is a collection of just a few things I learned and observed during my week in Las Vegas.
It’s All About AI
Over the past couple of years, AI has skyrocketed to the top of the “Peak of Inflated Expectations” in Gartner’s Hype Cycle and, not surprisingly, many of the sessions, demonstrations, and conversations at the conference were focused on leveraging AI.
Cloudera partnered with NVIDIA on two sessions where we shared our AI Inference service, which uses NVIDIA NIM microservices to accelerate the development and deployment of AI models, and supports the scaling of those models. We also shared our Accelerators for Machine Learning Projects, or AMPs, which are templates for machine learning/AI models that customers can deploy with the click of a button and start to customize, reducing the time it takes to get models into production.
Finally, we hosted a hands-on workshop to walk attendees through a Retrieval-Augmented Generation (RAG) workflow within Cloudera AI to show how easy it is to deploy contextualized models based on organizational data..
Our goal is to make it easy, fast, and safe for customers to get started with AI and see value from their projects, and that was a message that resonated with virtually everyone who stopped by our sessions and our booth.
Sustainable AI Will Be a Core Competency for Enterprises
Sustainability was a key theme throughout the conference. As companies think about where and how they deploy AI workloads, they must consider both the financial and environmental impacts of those decisions. Cloudera is proud to partner with AWS to help mutual customers deploy sustainable AI solutions by leveraging AWS Graviton processors, reducing consumption and costs while improving performance for AI workloads.
You can read more about the partnership and its implications here.
Distributed Data is Here to Stay
re:Invent is obviously a cloud conference, but most of the customers we spoke with had much more than just their AWS environment. Customers are dealing with data stores across multiple clouds and on-premises environments that, for a variety of reasons, may never move to a cloud. However, they still need to provide access to a unified view of that data for many use cases–from customer analytics to operational efficiency.
A key consideration for customers who find themselves in this scenario is to simplify as much as possible: choose platforms that provide a consistent experience, leverage tools that span multiple environments, and invest in open standards, technologies, and processes to ensure maximum flexibility now and in the future.
Trusted Data is Critical for AI Success
Data practitioners know that while the current hype is around AI, the real work of generating positive outcomes from AI models starts with providing secure and governed access to trusted data. Inevitably, the majority of companies will find themselves managing distributed systems, often in multiple clouds and on-premises. Cloudera Shared Data Experience (SDX) remains the only solution that provides consistent security, governance, accessibility, and observability across data stores wherever they reside, while also providing the engines to process and analyze that data.
This is a critical component of the AI workflow, and if I were going to bet on some of the reasons AI might not ultimately live up to the hype, my first guess would be that trusted data is harder to achieve than most organizations realize without a focus on metadata and a single source of security and governance.
Iceberg is the Winner
Apache Iceberg was everywhere this week, with big announcements from AWS, Cloudera, and others related to supporting our customers’ transition to the open table format. It’s been clear for most of 2024 that Iceberg is now the consensus choice for open table format, and the market is coalescing around it.
Cloudera’s investment in and support for open metadata standards, our true hybrid architecture, and our native Spark offering for Iceberg combine to make us the ideal Iceberg data lakehouse. Spark is the best engine for data processing, including ingestion and transformation, and while we provide many execution engines for data workloads on top of Iceberg, we also support data sharing via the Iceberg REST catalog specification, as well as connections to third-party engines. The result is true engine freedom, and reduced data copies and data movement.
It’s an exciting time for everyone in the Iceberg ecosystem. In past conferences, I have spent a lot more time introducing and educating customers on the benefits of Iceberg and open table formats. There was still a lack of awareness around the project. This year, it seemed the table format needed no introduction. We’re looking forward to working with AWS and others to build the best possible Iceberg lakehouse.
Flo Rida’s Still Got It
At re:Invent, we partnered with Mission Cloud to co-host IGNITE24 for our customers featuring Flo Rida, and it did not disappoint. The event embodied the collaborative spirit that defines our work with AWS and our partners. During the partner keynote at AWS re:Invent, Dr. Ruba Borno, Vice President, Global Specialists and Partners, compared the AWS partner ecosystem to a symphony. Just as a symphony requires diverse instruments to create a harmonious masterpiece, digital transformation relies on the orchestration of expertise and innovation from partners across the ecosystem.
At IGNITE24, we celebrated the importance of working together to achieve something greater than the sum of its parts—a principle AWS echoed in its messages about the power of partnership, shared goals, and mutual success.
Events like these remind us that while the work of transforming businesses with data is challenging, it’s also an opportunity to connect, collaborate, and celebrate our shared journey.
