AI offers organizations a more efficient way to analyze large volumes of data in real time. In many cases, running inference in Modzy on discrete single inputs or discrete batches of inputs is sufficient. However, in other situations, users need actionable insights instantaneously, and require the ability to process streaming data as it’s captured. Modzy now supports stream inferencing, allowing users to continuously pass input data such as live video to models and receive inference results and actionable insights in real time.

AI and streaming in action

There are many applications when users need to process streaming data in real time:

  • Traffic monitoring to inform infrastructure projects
  • Perimeter security to detect suspicious activity
  • Video captioning for live events to improve viewer accessibility
  • Equipment status monitoring to perform predictive maintenance

For example, AI could be used to automate captioning for a live event, such as a meeting where a company’s executive is giving a speech to employees. In this case, Modzy’s streaming capability can be used to ingest the audio feed during the speech, feed the audio to a Modzy processing engine to perform speech transcription, and return transcribed text for display to the audience in real time.

Consider another situation where local government officials are monitoring traffic to inform infrastructure projects and reduce traffic. Similarly, Modzy’s streaming capability takes in the traffic camera feeds to perform vehicle detection. Based on the vehicle detection inference results, real-time traffic flow statistics can be used to inform infrastructure planning efforts.

Modzy’s real-time stream inferencing detects vehicles in full motion video.

Modzy’s streaming solution

Using Modzy’s streaming capability is simple:

  • User deploys a model adhering to our gRPC model container specification which performs batch inference
  • User submits a job to the model, specifying the input type as “stream,” and provide the source stream URL
  • Modzy reserves a processing engine for the streaming job, decodes the stream, passes the decoded data to the model, and returns the inference results – in real time.

The ability to perform real-time inference on streaming data is advantageous when input data requires continuous analysis, providing actionable insights from machine learning models the moment you need them. Modzy is built with a robust, easy-to-use capability that abstracts away the intricacies of stream processing. Bottom line: Harnessing the power of AI to build streaming applications is here.