Simplifying Real-Time ML Pipelines with Quix Streams: An Open Source Python Library for ML Engineers

As data volume and velocity continue to increase, the need for real-time machine learning (ML) is becoming more pressing. However, building real-time ML pipelines can be complex and time-consuming, requiring expertise in both ML and streaming application development.

This talk will address this problem by introducing Quix Streams, an open-source Python library that makes it easy for data scientists and ML engineers to build real-time ML pipelines without having to learn the intricacies of building a streaming application from scratch.

In this talk, we’ll cover:

• The growing importance of real-time ML in today's application stack, and the use cases for real-time ML processing.
• A comparison of different ML architectures (batch, request-response, stream, and hybrid) and their pros and cons
• The current state of streaming architecture, which is typically Java-based, and the challenges this poses for data scientists and ML engineers who primarily work in Python
• An overview of Quix Streams and its features, including a demo of how to use it to build real-time ML pipelines

This talk is relevant for data scientists, ML engineers, and software engineers who are looking to adopt new technologies and practices in order to build real-time ML pipelines and stay current in their field.