Data is everything – this is the dogma that virtually every business follows these days. Large insurance companies are no exception, and it is here that data-driven approaches are essential. With Telematik Plus, for example, HUK offers a car insurance rate that records the driving behavior of policyholders via smartphone. The data is then analyzed in the backend. Responsible driving positively influences the insurance premium in this way. Through a smartphone app, the driver always has full transparency over his driving behavior.

An important building block of this backend is Apache Kafka, which mainly serves as a central data pipeline. Kafka has also become a fixed and central component of the Datalake infrastructure at HUK.
In addition, HUK is currently using Kafka to complete the transformation from batch to event-based real-time streaming processing.

Like all large and complex infrastructure components, Kafka clusters are best installed and managed automatically. DevOps tools and principles such as Ansible and GitOps meet requirements from the BigData area at this point.

In this talk we will go into more detail about the architecture of the HUK Kafka backend and share our experiences of the last 1.5 years, from planning to the first PoC to the production go-live.