The Eventing Issue: Navigating Event-Driven Innovation
Welcome to the third issue of Follow the Stream! Specially for Christmas, this edition is dedicated to Event Streaming! In software, any significant action can be recorded as an event. Also known as event stream processing (ESP), event streaming patterns can process a continuous flow of data as soon as an event or change happens. This scroll explores how event stream processing works, its major benefits, and how to get started building event-driven architectures.
Explore the basics behind events in Apache Kafka®.
Heading 1
Subtitle 1
Heading 2
Subtitle 2
Heading 3
Subtitle 3
First looking at terminology, review concepts like event design, event streaming, and event-driven design, to get an understanding of the role that events play in each of these approaches to using Apache Kafka®. Moving onto developing event-first thinking, consider the benefits of event-based systems, and try to understand how event-driven programming changes everything.
Tis’ the Season of Streaming, which means it's time to celebrate and innovate! Starting on December 9th, Confluent’s five-day learning event is an expert introduction to building with real-time and event-driven data flows. With 15 speakers and sessions on Kafka, Flink, data mesh, and GenAI, the event offers opportunities to enhance diverse skills, win attendance-based prizes, and support tech education with a $10 donation per registration.
Connect with Confluent: Celebrating One Year and 50+ Integrations
In just 12 short months, the Connect with Confluent (CwC) technology partner program has transformed from a new, ambitious initiative to expand the data streaming ecosystem into a thriving portfolio that’s rapidly increasing the breadth and value of real-time data. To celebrate this first anniversary, Confluent is excited to introduce the newest partners joining in Q3 2024 and reflect on the major milestones achieved through CwC integrations throughout the year.
Confluent’s WarpStream Deal Reflects Market Shift in Data Streaming
Confluent's acquisition of WarpStream underscores a significant shift in the data streaming market, as companies increasingly favor "bring your own cloud" (BYOC) solutions to manage cloud infrastructure costs. This approach reflects the industry's broader push toward diversified and scalable data streaming strategies, to meet the evolving demands of high-volume, cost-sensitive environments.
Empowering the First Generation of Real-Time AI Startups
The AI Accelerator Program from Confluent for Startups is designed to support the next wave of innovation in artificial intelligence. This highly selective, 10-week virtual program seeks to collaborate with 10 to 15 early-stage AI startups that are building real-time applications utilizing the power of data streaming. As real-time AI continues to transform industries, Confluent is uniquely positioned to help startups harness the potential of data streaming to drive intelligent, automated decisions at scale.
In today’s fast-paced business environment, real-time data insights are crucial, but traditional batch processing often results in complex, tangled integrations. To resolve this, companies are shifting to a universal data products mindset, which means treating data as a reusable, discoverable asset. A data streaming platform enables businesses to build reliable data products that support real-time experiences efficiently and cost effectively.
The Data Streaming Organization: Driving Value & Competitive Advantage From Data Streaming
A framework that allows organizations to meet increasingly complex customer demands can realize value from governed, shared, accessible, and real-time data. Organizations must extract maximum value from their generated data, which is achieved by providing a unified technology platform, established ways of working, and a guiding data streaming strategy and vision.
Real-Time or Real Value? Assessing the Benefits of Event Streaming
Event streaming, particularly in the service of event-driven architectures, is not just about reducing latency, but also about enabling new ways of structuring software and teams. It allows companies to optimize data for universal consumption rather than specific query patterns, which supports innovation, improves responsiveness to business needs, and transforms how teams work together.
Virta Powers EV Charging with Confluent’s Real-Time Data Streaming Platform
To support its rapidly expanding EV charging network, Virta adopted a fully managed data streaming platform, optimizing real-time data processing. This shift allows Virta to seamlessly handle 45 million messages per hour across 350,000+ charging points, delivering 100% uptime and instant updates.
Quick Thinking Report: Striking the Balance Between Instinct and Insight
Today’s business leaders are under growing pressure to make snap decisions as data and technology accelerate business operations. Confluent surveyed 200 UK executives, and found that 43% of C-level leaders believe difficulties with accessing real-time data is hindering decision-making, while 58% want to "completely overhaul" their data approach in 2025.
GEP Boosts AI-Powered Supply Chain, Processing 1B Events Per Month
Although the initial Kafka infrastructure at GEP addressed some challenges of batch processing, the setup required a 20-person team and the micromanagement of data logistics. By implementing Confluent's fully managed streaming platform, GEP scaled to over a billion events per month, streamlined the management of 500 microservices, and allowed developers to focus on successfully embedding AI into daily operations.
Penske Unlocks Vehicle Uptime and Supply Chain Visibility with Data Streaming and AI
Penske collects vast amounts of data from its over 400,000 vehicles, using sensors and IoT devices to monitor real-time metrics like location and engine status. AI-powered diagnostics analyze this data to predict vehicle failures before they happen, minimizing breakdowns and improving fleet management. The combination of real-time data streaming and AI allows Penske to improve supply chain visibility, and strengthens its ability to meet customer demands for on-time delivery.
Data Makes the World Go Round: 4 Key Trends for Supply Chains in 2025
“Data is the lifeblood that moves global commerce and the modern supply chain.” It also sets the tone for key supply chain trends in 2025, with the global focus shifting towards traceability and transparency, sustainability, and resilience. Another key trend of real-time data, automation and AI for more dynamic and responsive operations, highlights the overall need for high-quality data to power these advancements.
Event-Driven Architecture (EDA) vs. Request/Response (RR)
Adam Bellemare compares and contrasts Event-Driven and Request-Driven Architectures to give a better idea of the tradeoffs and benefits involved with each. Event-Driven Architectures (EDA) provide a powerful way to decouple services in both time and space, letting businesses subscribe and react to the events that matter most.
Unleashing the Potential of Demand Forecasting With Data Streaming
Demand forecasting is crucial for businesses, enabling them to leverage data for strategic growth and informed decision-making. Retailers can harness data to identify potential demand spikes and tailor marketing campaigns around major events. The case study of Acme Inc. illustrates how retailers aggregate and analyze diverse data sources, leading to a more accurate understanding of consumer behavior and market trends.
Leveraging Data Streaming To Reinvent BMW Group’s Omnichannel Sales Journey
Companies are increasingly embracing direct-to-consumer (D2C) strategies to enhance customer satisfaction and strengthen relationships. BMW, for instance, has announced plans to launch its D2C operations imminently. To support this shift, the BMW Group prioritized 360-degree data transparency across the customer journey, requiring real-time data access organization wide.
How To Migrate From Kafka To Confluent Cloud With Limited Downtime
Migrating from open-source Apache Kafka to Confluent can feel challenging, but success stories from companies like BigCommerce and Cerved highlight it as an effective and streamlined alternative to other deployment options. The migration process must be tailored to each environment and follows three simple phases: planning, setup, and migration/validation.
Apache Kafka in Manufacturing at Automotive Supplier Brose for Industrial IoT Use Cases
“Data streaming unifies OT/IT workloads by connecting information from sensors, PLCs, robotics and other manufacturing systems at the edge with business applications and the big data analytics world in the cloud.” Kai Waehner, Global Field CTO at Confluent, “explores how the global automotive supplier Brose deploys a hybrid industrial IoT architecture using Apache Kafka in combination with Eclipse Kura, OPC-UA, MuleSoft and SAP.”
Everything You’ve Ever Wanted To Ask About Event-Driven Architectures
Anna McDonald (the Duchess) and Matthias J. Sax (the Doctor), from Confluent, answer user-submitted questions on all things events and eventing, including Apache Kafka, its ecosystem, and beyond! The discussion highlights why events are a mindset, why the Duchess thinks event immutability is relaxing, and why event streams sometimes need windows.
Shift Left: Preventing and Fixing Bad Data in Event Streams
At a high level, bad data refers to any data that doesn’t conform to expected formats or standards. It can creep into data sets in a variety of ways, and cause serious issues for all downstream data users. In Apache Kafka®, event streams are built on an immutable log, meaning that once data is written, it cannot be edited or deleted. While this immutability is a core feature, it also introduces unique challenges, and requires extra caution when producing to, and managing data in Kafka.
How Apache Iceberg and Flink Can Ease Developer Pain
Adi Polak, Director at Confluent, highlights the distinction between operational and analytical data estates, emphasizing challenges like "schema evolution" that complicate downstream analytics. Technologies like Apache Iceberg and Apache Flink address these issues by optimizing data lakes and enabling real-time stream processing, improving system reliability and reducing latency.
The journey of transitioning from a monolithic to a microservice-based architecture starts with an exploration of asynchronous events, publish-subscribe frameworks, and patterns such as Strangler Fig and Branch by Abstraction. Using a real-world banking case study, Confluent’s Staff Software Practice Lead Wade Waldron highlights essentials for building systems that are scalable, resilient, and adaptable.
Building an Event-Driven Architecture? Here Are Five Ways to Get It Done Right
Despite the widespread adoption of Apache Kafka, its integration with event-driven systems continues to present challenges for developers and architects. Some key factors to consider are the importance of schema management, when to use stream processing over bespoke consumers, and how to ensure systems scale elastically for the future.
Real-Time Data and AI Thrust Manufacturing Into the Future
As factories become increasingly equipped with sensors, data-streaming technology, and AI to monitor every stage of manufacturing, “the production line is going online with dramatic effect on processes, operations and efficiencies.” Rolls-Royce, for example, demonstrates how AI and real-time data enable computer vision to minimize downtime and emissions, allowing for proactive service and greater scalability.
Data Meets Luxury: How Mercedes-Benz Leverages Event Streaming for Personalized Driving Experiences
Mercedes-Benz is transforming the luxury car experience by using a data streaming platform for hyper-personalized, AI-driven features. Event streaming simplifies data collection, cleaning, and enrichment, delivering real-time insights and AI-powered recommendations. This approach empowers R&D teams to craft dynamic, personalized driving experiences, accelerate innovation, and maintain compliance with global data regulations.
Organizing Data for Real-Time Manufacturing Insights
Today’s technology allows real-time data collection across production lines, but Smart Manufacturing requires more than just data gathering. To optimize production, the data must be strategically organized and contextualized. A Smart Manufacturing platform addresses this by using data mesh or data fabric architecture, providing shared data infrastructure and more effective internal networks.
