We are proud to present you with a new Savant version — 0.2.5. We worked on the release for more than 2.5 months. It contains significant changes, new features, and bug fixes in several fields, but primarily, we improved developer experience and deployment features.
Nvidia hardware! Here’s what’s new:
🔗 Join our Discord and visit GitHub to learn more about Savant’s cutting-edge features and to stay updated with our latest advancements. Let’s shape the future of computer vision together.
Please go to Medium to read the original release announcement with more pictures, videos and links.
Brand-New DeepStream 6.3
Nvidia has unveiled the latest DeepStream SDK, version 6.3, which forms the foundation for Savant 0.2.5. With the introduction of DeepStream 6.3, several essential bug fixes were incorporated, prompting our transition from version 6.2 to 6.3. Notable updates include:
- Resolution of the H.264 encoder issue.
- Enhancement of the JPEG encoder, which now benefits from dGPU-assisted encoding, a shift from its previous software-based operation.
Nevertheless, there are still some persistent issues in DS 6.3:
- Streams containing B-frames might experience frame disarray on Nvidia Jetson, though Savant has devised a solution.
- A glitch in the nvstreammux component results in the dispatch of repeated events, but Savant offers a remedy.
Crafting the Developer Experience
We’re staunch advocates for the notion that creating computer vision pipelines should be a straightforward process. However, the age-old tug-of-war between complexity yielding performance and simplicity leading to inefficiency remains.
DeepStream, while immensely powerful, isn’t the most user-friendly. Those accustomed to elementary computer vision platforms, such as PyTorch, might find themselves in a maze when navigating DeepStream’s intricacies. This complexity isn’t without reason. DeepStream stands as a beacon of parallel computing, offering cutting-edge stream processing logic. But the journey from grasping GStreamer to wrestling with its less-than-intuitive tools and integration gaps can be daunting with the raw DeepStream.
Enter Savant. Our mission is to simplify this intricate landscape. We’ve infused our platform with five pivotal features aimed at boosting developer efficiency. This latest release brings to the fore sophisticated tools, enabling seamless pipeline development directly from IDEs like PyCharm or VS Code. Additionally, we’ve enriched our documentation, offering hands-on demonstrations of diverse functionalities, complete with video tutorials.
Embracing OpenTelemetry Integration
Savant has taken a leap forward by integrating OpenTelemetry. This integration empowers users to meticulously instrument each frame that flows through a pipeline. The benefits are manifold: from code profiling and attaching logs and attributes to specific traces, to delving deep into pipeline glitches and identifying potential chokepoints. For those keen on exploring this feature, we’ve curated a hands-on sample that showcases the prowess of OpenTelemetry within Savant. Users have the flexibility to deploy OpenTelemetry in either sampled or comprehensive modes.
DevServer
The feature enables hot Python code reloading upon change. It helps to experiment, debug, and develop faster with just IDE. You don’t need to rebuild and restart the pipeline to have an effect.
Introducing the Client SDK
The Client SDK, a Python-centric library, is designed to streamline the process of interfacing with a pipeline. With just a handful of straightforward Python lines, users can both feed data into and extract processed results from the pipeline. A standout feature is its seamless integration with OpenTelemetry, granting users programmatic access to specific frame processing logs.
Beyond this, the Client SDK is a versatile tool. Whether you’re crafting unit tests, comprehensive integration tests, or bespoke adapters, this SDK has got you covered. Catering to diverse needs, it offers both synchronous and asynchronous variants.
Fancy Logging
We added color-decorated logs to help developers analyze them faster.
URI-Input Script
We packed Savant with a handy script to ingest video or images to the pipeline right from the development environment without the need for launching standalone adapters.
Improvements In Adapters
We implemented several enhancements to adapters.
🆕 Kafka/Redis Adapters
We published technology preview Kafka/Redis adapters in the release. They enable scalable processing required to handle many streams in high-load, scalable environments. The adapters handle metadata with Kafka and video with Kafka or Redis (or other Redis-compatible storage like KeyDB).
🆕 Multi-Stream Source Test Adapter
To help developers measure the performance of their pipelines and tune parameters, we implemented a utility adapter ingesting multiple streams in the pipeline. The adapter can create a desired parallel workload to help understand how the pipeline behaves under the workload and discover optimal performance configuration (batching, number of streams, etc.).
Upon completion, it can send a shutdown message to a pipeline to signal that testing is over.
🆕 Custom Adapter Development Toolkit (Client SDK)
Client SDK, described in the previous section, gives excellent tools to develop custom adapters quickly.
Multi-Streaming Always-On RTSP Sink Adapter
Previously, developers needed to launch an instance of AO-RTSP Sink Adapter per stream to cast the resulting videos. Now, you can launch a single adapter and specify multiple streams simultaneously.
HEVC Encoding For Industrial GigE Cams
With the feature, streams retrieved from GigE Vision cams can be HEVC-encoded and sent with narrow band channels if necessary. The feature may improve performance when video compression doesn’t decrease the quality.
Deployment
We improved how the module Docker image signals the container environment about its initialization process. Now, developers can utilize the keep-alive features of Docker to understand when the pipeline is ready to handle data.
Health status is available with the filesystem object and HTTP URL.
Demos and Samples
We believe samples are crucial to new users who want to dive into Savant fast; thus, every new release of Savant comes with additional samples. We are improving previously published samples to reflect industry changes.
Instance Segmentation Demo
The demo features the YOLOV8M-Seg model. The pipeline can reach 100 FPS and more on a modern GPU like RTX4000 or A4000.
Facial ReID Demo
The demo features YOLOV8-Face, AdaFace, and HNSWLib to recognize people on video.
Age/Gender Recognition Demo
The demo is based on YOLOV8-Face and a custom model based on MobileNet V2.
Traffic Meter Sample Enhancement
We also extended the traffic meter demo to use YOLOV8S for detection. Now it supports PeopleNet, YOLOV8M, and YOLOV8S on your choice.
Utility Samples
- The OpenTelemetry example shows how to use it.
- The client SDK, OpenTelemetry, and DevServer features are demonstrated in the template project.
- RTSP Cam Compatibility check sample. We created a sample showing how to test an RTSP cam for compatibility and stream quality.
- Kafka/Redis Ecosystem Sample. The sample shows how to deploy Kafka/Redis source, sink adapters, ingest and access data.
Bugs
The release comes with 14 bug fixes.
Documentation
A vast documentation update comes with Savant 0.2.5. Among the most important updates are:
- development environment configuration for VS Code;
- developer tutorial covering our approach to develop pipelines for Savant efficiently in PyCharm and VS Code (+ video);
- advanced topics covering non-trivial problems of pipeline development;
- adapter documentation updated.
Other Enhancements
Rust-based Transport Protocol. Previous Savant versions were based on Avro. However, we have found that AVRO is not fast enough for our needs, and we have concerns about the GIL release with Python AVRO. To address the possible problems, we re-implemented the protocol with the Rkyv crate and implemented proper GIL releasing when necessary.
Element Groups. The feature is described in detail in the documentation.
Advanced Logging Configuration. Now, log levels can be configured per code scope. Find out more in the documentation.
H264 Software Encoder. Hardware encoder is unavailable in dGPUs like A100, V100, and Jetson Nano V2. To fully support these devices, we implemented support for the software-based encoder. More on the software encoder here.
Frame Sequence Validation. When real-time adapters begin to lose data because of pipeline bottleneck, a special WARN message appears in logs.