Anchor: Navigation

Manufacturing Issue 2.0 - October 2024 Follow the Stream Your Dedicated Community Content Scroll 

Unlocking Real-Time Insights: The Power of Data Streaming in Action

Welcome to the second issue of Follow The Stream! In this edition, we continue our journey into how organizations harness the power of real-time data to foster innovation. We’re excited to present Confluent's first-ever verified use cases, showcasing transformative applications of data streaming within the manufacturing sector. Let’s explore how to turn data into a strategic asset!
See Industry Updates

Data Streaming Fundamentals

Data Streaming 101 Infographic

The modern world is defined by speed. Grocery delivery, rideshare apps, and payments for just about anything can happen instantly using a mobile device and its apps. Every action of every consumer creates data, and businesses must make sense of it quickly to take advantage in real time.
Data streaming platforms like Confluent use real-time pipelines to collect, store, and use data continuously using event-driven principles. Each change in the data, known as an event, triggers new data to be received and integrated into the processing function.

Heading 1

Subtitle 1

Heading 2

Subtitle 2

Heading 3

Subtitle 3
Get Infographic

Confluent Crash Course

Explore the basics of Apache Kafka®, Apache Flink®, and DSP.

Heading 1

Subtitle 1

Heading 2

Subtitle 2

Heading 3

Subtitle 3
Data in Motion is the foundation for everything at Confluent.
Check out the links below to get an understanding of how real-time data processing powers business productivity and innovation:
Subtitle 1
Subtitle 2
Subtitle 3
Anchor: General-Updates

Industry in Motion: General Updates

Data Streaming ROI & Investments Soar Higher

As businesses learn to do more with less, proving tech investment ROI has become even more critical. This ensures they are achieving value, despite limited resources. 84% of IT leaders cite 2x to 10x return on data streaming investments as it continues to drive significant benefits across a wide range of business functions and industries.
37% of IT leaders in the manufacturing industry see a 5x or more ROI on data streaming investments.
Get Data Streaming Report 2024
Available in German, French, and Portuguese!

Data streaming fundamentally changed the way modern companies do business

“Confluent CEO and co-creator of Apache Kafka®, Jay Kreps, shares his insights on the transformative power of data streaming for modern enterprises. As businesses strive to harness real-time data for better decision-making and competitive advantage, Kreps offers a deep dive into the current state of data streaming adoption, the challenges organizations face, and the pivotal role data streaming plays in enabling cutting-edge applications like real-time AI.”
Read Article

Stream Processing is a Journey Over Decades…

"Stream processing has existed for decades. However, it really kicks off in the 2020s thanks to the adoption of open source frameworks like Apache Kafka and Flink. Fully managed cloud services make it easy to configure and deploy stream processing in a cloud-native way; even without the need to write any code". Confluent’s Field CTO and data streaming expert Kai Waehner explores the past, present and future of stream processing, "including the relation of machine learning and GenAI, streaming databases, and the integration between data streaming and data lakes with Apache Iceberg".
Read Blog Post

Currect 2024 Recap: Convening With Data Streaming Engineers

This year’s Current may be over, but all the mind-blowing work the data streaming community shared in Austin lives on. At Current 2024, we explored the evolution of data streaming, emphasizing the critical role of data streaming engineers amid a significant paradigm shift in data management, the transition to AI-powered applications necessitating a broader definition of roles in AI/ML engineering, and discussed technologies like Kafka and Apache Flink®, highlighting insights into future challenges and innovations.

Confluent Named a Leader in two IDC MarketScape Reports

The IDC MarketScape vendor analysis model is designed to provide an overview of the competitive fitness of ICT suppliers in a given market. In 2024, Confluent was recognized as a Leader in the IDC MarketScape for Worldwide Analytic Stream Processing Software 2024 Vendor Assessment and the IDC MarketScape for Worldwide Event Brokering Software 2024 Vendor Assessment. 
The first assessment highlights stream governance as well as partner program and multicloud support as Confluent’s competitive strengths, while the second recognizes Confluent for strong analytical and operational capabilities and out-of-the-box monitoring and security capabilities.
Read Article

The Data in Motion Tour 2024 is Stopping by Your City! 


Join data streaming experts for an opportunity to network with peers and ecosystem partners, hear directly from customers, learn from Confluent's team of Apache Kafka® experts, and roll up your sleeves in interactive demonstrations and hands-on labs.
Learn More

Upcoming dates:
October 9th: Vienna - RSVP October 15th: Delhi - RSVP October 15th: London - RSVP October 15th: Madrid - RSVP October 22nd: Riyadh - RSVP October 23rd: Johannesburg - RSVP October 29th: Zürich - RSVP November 6th: Frankfurt - RSVP November 13th: Milan - RSVP November 19th: Stockholm - RSVP November 21st: Copenhagen - RSVP November 28th: Dubai - RSVP

Anchor: Executives-Brief

Executive's Brief: Data Strategy & Leadership

How a Truly Cloud-Native Kafka Solution Can Modernize Architecture

Confluent has invested 5M+ engineering hours in completely re-architecting Apache Kafka® to harness the full power of the cloud and support all mission-critical enterprise workloads with 10x faster scaling, infinite storage, and built-in resiliency. There are three attributes that set Confluent apart from Kafka and other solutions for data streaming:
  • Confluent is cloud-native, providing a 10x better, fully managed service to free you from operational burdens
  • Confluent is complete with all the essential tools for developers to build quickly, reliably, and securely
  • Confluent exists everywhere to connect all data in real time, from on-prem to cloud and across clouds
Read Ebook
Available in German, French, Spanish, and Italian!

Confluent: Reusable Data Streams Are Rising C-Level IT Agendas

Executives are increasingly prioritizing reusable data streams due to their benefits in agility, cost-efficiency, as well as cleaner code and greener data practices. Platforms like Confluent Cloud for Apache Flink® support the creation of these high-quality streams, helping organizations to streamline operations and improve decision-making. 
Read Article

Moving Up the Curve: 5 Tips for Enabling Enterprise-Wide Data Streaming

The Confluent maturity curve outlines the five stages of Apache Kafka® adoption. It highlights how deeply data streaming is integrated within a business, which can help identify opportunities and challenges. Here are some tips for moving to the next stage of the maturity curve:
  • Show what a level 4 & 5 data streaming platform looks like
  • Make the business case (specifically for level 4+)
  • Show how to implement enterprise-wide data streaming with a Target Operating Model
  • List all stakeholders involved
  • Confirm when to implement this—specifically at level 3

Heading 1

Subtitle 1
keyboard

Heading 2

Subtitle 2
keyboard

Heading 3

Subtitle 3
keyboard
Read Blog Post
Anchor: MustRead

How an EV Manufacturer Drives Operational Efficiency With Real-Time Streaming

As electric vehicle manufacturers face complex integration challenges across diverse systems, real-time data visibility becomes crucial for optimizing production and meeting demand. An EV manufacturer turned to Allata to implement a data mesh using Confluent Cloud, enabling seamless integration across key systems and enhancing operational efficiency. This approach transformed the company's data management by providing real-time insights and reducing overhead, setting the stage for future improvements in data-driven decision-making and customer experiences.
Read Blog Post

Why Global Manufacturing Companies Rely on Data Streaming?

Learn about data streaming applications within the manufacturing industry directly from Confluent’s Global Field CTO Kai Waehner. In this overview video, he walks through how to get end-to-end visibility into OEM supply chain by leveraging data streaming for OT and IT data and how it helps to modernize and innovate the entire engineering and sales lifecycle. All of the data is managed in one visibility infrastructure where you can take action on the data, like enabling predictive maintenance or better customer service. 
Watch on YouTube
Anchor: Supply-Chain-Resilience

Data Streaming for Supply Chain Resilience

Anchor: Highlight-2

Michelin’s Evolution to Cloud-Native Microservices and Data Streaming

Michelin modernized its global supply chain by transitioning to real-time inventory management using Confluent Cloud and Apache Kafka®. By replacing unreliable batch reporting with a data streaming platform, Michelin improved visibility across its entire supply chain, reducing downtime and enabling faster, data-driven decisions that optimized logistics and cut operational costs.
Watch Online Talk

The Magic of AI in Supply Chain: Hype vs. Reality

“Industry experts Jeff Winter and Lora Cecere, founder of Supply Chain Insights, discuss current trends and the rise of enterprise technology and AI in today’s supply chain industry.”
Watch on YouTube

How FourKites Built Real-Time Visibility and AI for the Global Supply Chain

In 2023, FourKites, the platform behind the world’s largest supply chain data network, launched MyWorkspace – a centralized intelligence solution that enables customers to create tailored dashboards for interconnected decision-making across enterprise and supply chain data. Arpit Garg, Senior Principal Architect, and Goutham Elangesh, Staff Data Engineer at FourKites, explain how the engineering team implemented MyWorkspace using Confluent Cloud Kafka and Rockset. 
Watch Session
Anchor: Architects-Blueprint

Architect's Blueprint: Data Systems Design

Stream SAP Data Anywhere It's Needed with Confluent Cloud

Harnessing SAP data in real time is the key to reinventing customer experiences, building efficiencies in back-end operations, and launching new business models and services. Ultimately, it is creating a more responsive business that keeps pace with the modern, digital-first world. To achieve this, more businesses are tapping into data streaming to keep all systems that rely on SAP data in sync with high-value, real-time data.

Apache Kafka in Manufacturing at Automotive Supplier Brose for Industrial IoT Use Cases

Data streaming unifies OT/IT workloads by connecting information from sensors, PLCs, robotics and other manufacturing systems at the edge with business applications and the big data analytics world in the cloud.  Kai Waehner, Global Field CTO at Confluent, explores how the global automotive supplier Brose deploys a hybrid industrial IoT architecture using Apache Kafka in combination with Eclipse Kura, OPC-UA, MuleSoft and SAP.

Heading 1

Subtitle 1
keyboard

Heading 2

Subtitle 2
keyboard

Heading 3

Subtitle 3
keyboard
Read Blog Post

Analyze Factory Floor Telemetry Data for Anomalies in Real Time

Modern automated manufacturing factory floors can have hundreds of robots performing various tasks, including picking and placing objects, assembling parts, packaging, welding, painting, and cleaning. By effectively utilizing telemetry data and leveraging the power of real-time streaming analytics platforms like Confluent Cloud’s Apache Kafka® clusters and Apache Flink® SQL, valuable insights into robot health and performance can be gained.
See Use Case

Manufacturing 4.0 Connected Vehicles, Mobility Services, and Smart City 
Real-Time Processing and Analytics powered by Apache Kafka and MQTT

Apache Kafka® and MQTT are a perfect combination for many IoT use cases. Kai Waehner, Global Field CTO at Confluent, looks at the pros and cons of both technologies, and how they are complementary. Various real-world case studies across industries will be discussed, including connected vehicles, manufacturing, mobility services, and smart city. The examples use different architectures and best practices for lightweight edge scenarios, hybrid integrations, or serverless cloud solutions.
Watch on YouTube

The Top Five Trends for Data Streaming in 2024

Data streaming with Apache Kafka® and Apache Flink® is a cutting-edge technology reshaping the digital landscape, garnering significant attention worldwide. Kai Waehner, global field CTO at Confluent, explores the latest developments in data sharing, data contracts, serverless stream processing, multicloud solutions, and AI integrations, providing a comprehensive overview of what's next in this dynamic field.
Watch Online Talk
Anchor: Developers-Desk

Developer's Desk: Building Applications

The Data Streaming Revolution: The Force of Kafka + Flink Awakens

Explore data streaming and Apache Flink® in the second issue of Confluent's Comic Book. Follow Ada, Jax, and the data architecture team as they navigate the challenges of stream processing to prepare a shoe retail titan for the biggest athletic event of the year.
Join them to explore how Flink® and Kafka can be leveraged in tandem. After all, some things are just better together.
Get the Comic
Available in German, French, Spanish, Korean, and Japanese!

How Producers Work: Kafka Producer and Consumer Internals

Apache Kafka is a powerful event streaming platform that efficiently stores and delivers event data with minimal effort from developers, functioning like a black box. However, when issues arise with Kafka, the challenging task of debugging requires a deeper understanding of the inner workings of Kafka, to see how we actually interact with the black box through producers and consumers.
Read Blog Post

Uniting the Machine Learning and Data Streaming Ecosystems

Uniting machine learning (ML) and data streaming requires overcoming both technical and socio-technical barriers. Organizations need to adopt decentralized, domain-driven approaches like data mesh and feature-oriented teams. Utilizing Apache Kafka® as the backbone for real-time data integration can operationalize ML by providing the speed and scalability required for real-time analytics.

Heading 1

Subtitle 1

Heading 2

Subtitle 2

Heading 3

Subtitle 3
Read Blog Post

Let Flink Cook: Mastering Real-Time Retrieval-Augmented Generation (RAG) with Flink

Large language models are advancing rapidly, but to move from prototypes to production, they need real-time, domain-specific data and strong security. RAG addresses this by integrating live data streams with AI models for more accurate and relevant responses. Apache Kafka and Apache Flink can be used to implement RAG for various applications, enhancing the performance and reliability of generative AI systems.

Heading 1

Subtitle 1

Heading 2

Subtitle 2

Heading 3

Subtitle 3

3 Best Practices for Building Cloud-Native Systems with Kafka

Building a complete, enterprise-ready data streaming platform yourself with Apache Kafka® is a complex undertaking. Although this task only becomes more challenging—and more resource-intensive—when done in the cloud, organizations that go further and build a cloud-native data streaming platform can design an architecture that unlocks business value, speeds innovation, and reduces total cost of ownership (TCO). 

Heading 1

Subtitle 1

Heading 2

Subtitle 2

Heading 3

Subtitle 3
Get Brief

Learn About Data Streaming With Apache Kafka® and Apache Flink®

High-throughput low latency distributed event streaming platform. Available locally or fully-managed via Apache Kafka on Confluent Cloud.

High-performance stream processing at any scale. Available via Confluent Cloud for Apache Flink.

Explore Developer Hub

Request Flink Workshop or Tech Talk

Anchor: Innovation-Research

Data Streaming for Innovation & Research

Real-Time Data Streaming for Smart Warehouses

Smart warehouses are transforming operations through advanced technologies like IoT, robotics, and data streaming. Real-time data streaming, in particular, enhances efficiency and accuracy by enabling seamless inventory management, predictive maintenance, and automation, ultimately meeting the high demands of customer expectations.

Heading 1

Subtitle 1

Heading 2

Subtitle 2

Heading 3

Subtitle 3
Read Blog Post

Modernizing SCADA Systems and OT/IT Integration with Data Streaming

“SCADA control systems are a vital component of IT/OT modernization. The old IT/OT infrastructure and SCADA system are monolithic, proprietary, not scalable, and miss open APIs based on standard interfaces.” Kai Waehner, global field CTO at Confluent, explains the modernization of such a system based on the real-life use case of 50Hertz, a transmission system operator for electricity in Germany.

Data-Driven Manufacturing: Dialogue with Dr. Ralf M. Wagner

Dr. Ralf M. Wagner, senior vice president of Insights Hub, "discusses the role of data-driven manufacturing in manufacturing, the challenges and advancements in industrial IoT solutions, and generative AI technologies".
Watch on Youtube

Join the Community

Sign up for updates below and check out previous issues!

Share content suggestions and new uses cases in the Comments Section