Quix Announces New Open Source Library to Process Streaming Data in Python

LONDON, March 2, 2023 (Newswire.com) - Quix, a real-time telemetry data platform founded by Formula 1 data engineers, is open-sourcing their data streaming library called "Quix Streams" on the platform's three-year anniversary.

Quix Streams makes it quick and easy to build real-time applications that process high volumes of telemetry data when developers need a quick response and guaranteed reliability at scale.

The library is written in C#, available in Python, and designed to be easily extended to other programming languages.

It includes many useful features, such as:

• Streaming context: allows teams to bundle data from one data source into the same scope with supplementary metadata—thus enabling workloads to be reliably scaled horizontally across multiple replicas.
• Built-in buffers: make it easier to run processing operations on windows of time-series data and reduce the operational cost of producing and consuming data at high frequencies.
• State management: easy-to-use state store combining blob storage and Kubernetes persistent volumes that ensures quick recovery from any outages or disruptions.
• Support for data frames: with a tabular protocol that efficiently processes multiple parameters with one timestamp (rather than publishing them separately).
• Message splitting: automatically splits and merges large messages on the producer and consumer sides, respectively.
• Built-in data serialization/deserialization: automatically serializes data from native types using built-in codecs for better performance.
• Multiple data type support: enable various data to be attached to timestamps, such as numbers, string and binary data—reducing the extra data handling required in the code.
• Checkpointing: perform manual checkpointing when subscribing to a topic and inform the message broker what has already been processed (thus increasing performance).
• Support for raw messages: for custom scenarios the where Quix Streams protocol does not suffice, developers can publish and subscribe to raw messages instead of using the Quix format.

Software, ML, and Data Engineers can now manage the ever-greater volume and velocity of time-series data with ease, as Quix Streams makes streaming data more easily accessible.

"Our goal is to empower engineers and data scientists from all parts of the data ecosystem," said Founder and CTO Tomas Neubauer. "With the release of Quix Streams, we want Python developers to enjoy the scalability and resiliency of Java and Scala-based technologies that have been traditionally used to process data streams but without the hassle of working with JVM environments. We also believe that open-sourcing our library will encourage collaboration and innovation across the industry, and we're excited to see what people will do with it."

In the next version, the Quix team plans to introduce a new feature called "streaming data frames" that simplifies stateful stream processing for users coming from a batch processing environment.

Learn more about Quix by visiting quix.io or reading the developer documentation at docs.quix.io. To book a product demo, reach out to an expert.

Source: Quix