Streaming - Michelin IT Engineering Blog

Sign in Subscribe

Streaming

A collection of 8 posts

Processing Error Handling in Kafka Streams

Processing Error Handling in Kafka Streams

Introduction A major concern when developing Kafka Streams applications is handling processing errors. Processing errors occur when the implemented logic in a Kafka Streams application fails to process a record correctly. Let’s illustrate what processing errors are with a concrete example from our Supply Chain domain: a DeliveryBooked event

Data in motion, myth or reality

Data in motion, myth or reality

When I started to work in IT a couple decades ago, the center of the universe for a software engineer was databases, mostly relational ones. We were relying heavily on their transactional capabilities to ensure data consistency. Integrating an app with the rest of the world was quite a challenge.

Designing Kafka Streams Applications

Designing Kafka Streams Applications

For the past 4 years, our journey into the heart of Kafka's capabilities has been shaped by two pivotal concepts: Master Topologies and Micro Topologies. These conceptual frameworks have become the backbone of our Kafka Streams application design, offering a comprehensive and granular understanding of our end-to-end communication.

The Michelin Guide: an unexpected event driven use case

The Michelin Guide: an unexpected event driven use case

tl;dr content publication is a challenge for an institution like the Michelin guide. Event streaming between SaaS solutions using Kafka hosted in Confluent cloud was a solution for us to move from a 2 days delay publication to near real time. Although it was not easy especially from an

Moving from orchestration to choreography - Part 2

Moving from orchestration to choreography - Part 2

In the first article [https://blogit.michelin.io/choregraphy-or-orchestration-thats-the-question/] of this serie, we've been through the preliminary work to convince our organization to move from a central orchestrator (implemented with a Business Process Management tool) to a choreography of micro-services. This second article will focus on a critical