Michelin loves Open Source Software and we can prove it - Part 2
.jpg)
I began my software developer journey 10 years ago. From the beginning and still today, open-source has been an integral part of my work. Yet, it was only as consumer, and I never completely embraced the model because I never contributed to any projects. In this post, retrospective of the past year, we will cover the reasons behind a new Michelin contribution, how to start and why OSS is accessible to all.
The genesis: a problem or a need
Michelin deployed an Event-Driven Architecture approach thanks to Kafka since 2018. Clusters have been deployed on-premise in our factories, others in the cloud and we needed to propose a tool to help teams supervising their topics, consumer groups, connectors, etc. The clusters or resources management is delegated to ns4kafka, our open-source solution that brings Kubernetes best practices (namespaces and resources management) on top of Kafka.
There are several competitors that aim to help managing Kafka resources. We can quote RedPanda console, Conduktor console, Confluent Control Center or UI for Apache Kafka. We chose another one, AKHQ. Michelin started to contribute in 2021, especially on authentication and authorization, to provide for instance an external claim provider for AKHQ that we could link to ns4kafka, that manages namespaces with Kafka ACL. Since then, AKHQ is used widely at Michelin.
AKHQ offers everything you need to manage your Kafka resources. It abstracts Kafka clients features, allows you to browse your topics, search for messages simply on a GUI. In figures, it represents +3.1K stars on GitHub, 189 contributors and 23 millions Docker image pulls.
tchiotludoIn 2023, I moved to another position in Michelin and started to work on Kafka-related topics for manufacturing. I quickly faced some issues and heard complains from colleagues about AKHQ. Not the kind of issue that blocks you using the tool, but several small issues that makes you waste time. A survey has been conducted to collect all the feedback, evaluate the satisfaction/dissatisfaction level of AKHQ and list the different issues people had. Results were definitively bad, the tool was producing frustration and some people even started to think about designing a new solution. Disclaimer: most of the issues were due at this time to a specific setup we had on our AKHQ instances, some edge cases the tool was not handling yet.
On my side, I knew that there was an intermediate solution, that will prevent us from either deploying a new tool (with all the considerations it leads to: choosing the tool among all the possibilities, qualification for our usage, trainings, etc.) or developing our internal solution.
The opportunity: a win-win
At that time, we had a clear overview of all the pain points and started to imagine the next steps. It was the opportunity for me to propose something else: why can't we contribute to the project ?
When you start thinking about the benefits/risks balance, open-source quickly makes sense. If you choose the statu quo, people continues to be frustrated every time they use the tool, it's clearly not an option. If you decide to search for a replacement, you will probably spend weeks to test several other solutions, do the necessary to deploy it on your infrastructure, manage the ramp-up by training people on this new tool, answer questions/issues people have the first time they use something new, etc. Of course, it depends on how big your company is.
Contributing to open-source is the best compromise. You spend minutes, hours, days, depending on your capacity and the result is almost immediate. I mean, you won't be forced to write code if you are not capable of. Contributing by raising an issue on a project, with enough details to help a maintainer reproducing and fixing it are the first steps into open-source.
In my case, I felt confident to fix the issues by myself. I talked to my manager and proposed to spend few days to setup my environment, dig into the source code to understand how AKHQ was developed before checking the issues we had. No objections, so let's start !
It can't be this easy for everyone to get a GO from a manager to spend time on a product, which is not your internal product. However, try to expose the benefits for the company and yourself of contributing to open-source. Personally, I prefer to work in a company which is open-minded and find in open-source, technical conferences, etc. a way to give their developers a sense of fulfilment. And developers looking for a job will probably think the same.
The beginning: start small
Obviously, I didn't start to contribute on AKHQ by taking an issue that would impact the "core" of the app because I wasn't able at this time to identify all the impacts a change could have. I started by simpler one, an issue that was causing people to login twice sometimes. 2 lines of code later, my first pull request was created and merged 🥳
After several other fixes and a release done by Ludovic, the project's owner, we were able to deploy it on our instances. The situation started to become stable, we had less complains and people even started to propose improvements.
The ramp-up: think big
The biggest improvement submitted by users was to have fine-grained permissions to be able to give access to Kafka resources on some clusters, but not all. In our context, where a single AKHQ instance by zone (Europe, Asia, Americas) was managing all our Kafka clusters within the zone, it will allow to give access on resources to people from each plant, without giving access to resources everywhere.
This need was of course impacting all the AKHQ features and all the supported authorisation mechanisms (LDAP groups mapping, OIDC mapping, external claim provider, etc.). This feature was in the AKHQ backlog for a while and the target solution was almost defined. After a call with Ludovic, where we ended imagining the solution to answer this need, I started to work on it. With 31 commits and 104 files changed, this is my biggest pull request ever merged (for now) 🤯
Bonus: implementation details
The major update in this PR is the introduction of the AKHQSecured annotation. This annotation decorates API endpoints to define the required Resource (Topic, TopicData, Schema, etc.) and Action (Create, Read, Update, etc.) a user must be granted to access this endpoint. The security check is done in a Micronaut SecurityRule. This rule retrieves the expected Resource/Action from the annotated endpoint and compares it to the user's authorization details in the JWT token.
This "high-level" check is completed by another check at the endpoint level. We needed to take into account a possible resource and/or cluster filtering applied on the user's groups. I wrote some generic functions that every endpoint can use to 1/define if the user can call it 2/ build query filters from the user's permissions and inputs. Like this, a new endpoint won't have to reinvent the well, but simply call one of these function to complete the authorization (or get an AuthorizationException). These functions are simple to use, the implementation (and some introspection) retrieves the calling method details (AKHQSecured annotation) as well as the user permissions.
void checkIfClusterAllowed(String cluster);
void checkIfClusterAndResourceAllowed(String cluster, String resource);
void checkIfClusterAndResourceAllowed(String cluster, List<String> resources);
List<String> buildUserBasedResourceFilters(String cluster);
Conclusion
I learnt several things during this journey. First, you can't blame an open-source product for the issues you had. Behind that, there are usually people working on it on their free time to propose this tool to you. Second, you can contribute at your level simply by raising an issue on a project. Of course, you are welcome if you can fix issues and develop new features. Then, don't be afraid to propose to contribute if you feel that your company (and the community) can benefit from it. It is a win-win.
Thanks to Michelin, my manager and the OSS community I can proudly say that I am now an open-source contributor. Michelin is now recognized as core contributor on AKHQ:
