Data Mesh in Practice: Kafka for 50 Teams with FlixBus
Lessons learned at Flixbus from building and maintaining a distributed self-service streaming platform
Introduction
Join us to hear from Taras Slipets, Staff Data Engineer at FlixBus, about the lessons learned from building and maintaining a distributed self-service streaming platform over the last 4 years.
Flixbus are leaders in the global green passenger mobility market, servicing more than 400,000 daily connections. Real-time data is critical for decision making in this dynamic environment, facilitating fleet tracking, route optimization, delivering precise arrival times and leveraging ML/AI to optimize the customer experience.
The session will deep-dive into:
Kafka usage at FlixBus
Data mesh core principles
Naming conventions for domain-oriented ownership
Kafka's prefixed ACL pattern type
Lessons learned from enabling 10 domains and 50 internal teams
Self-service for resource management (architecture, CI/CD, permissions workflow)
We'll also take a look at how Conduktor is committed to providing a scalable, vendor-agnostic solution for self-service on Kafka, facilitating collaboration between Platform Teams and Application Teams.
Hosts
James
White
Taras
Slipets
Recording date
21st of February, 2024
duration
50 min.