In News|5 Minutes

Efficient Multiple Object Tracking with MHT

Efficient Multiple Object Tracking with MHT

In the field of computer vision, the race for performance is often defined by bigger models and more powerful GPUs. This is also true for Multiple Object Tracking (MOT), the task of knowing who / what is where - at all times.

MOT is one of the few classical computer vision tasks that we still consider unsolved after the advent of deep learning. The complexity comes from the peculiar mix of perception challenges (e.g. occlusions, ambiguities) and suboptimal optimization choices, which are at the heart of every tracking method.

State-of-the-art benchmarks for tasks like Multiple Object Tracking (MOT) are dominated by deep learning architectures that, while incredibly accurate, demand immense computational resources (e.g. see MOTChallenge for an always up to date list). But what happens when a high-end GPU isn’t an option? For a growing number of real-world applications, algorithmic efficiency is the most critical feature.

At Deep Vision Consulting, we specialize in solving exactly this kind of problem. So we applied our skills to MOT as well and built a high-performance tracking systems that deliver real-time results on resource-constrained hardware, proving that deep algorithmic expertise can (sometimes) outperform brute-force computation.

Figure 1. Representation of the core idea behind MHT. As the name suggests, every detection can be associated to different identities in time, creating an exponential number of possible trajectories. The key challenge is thus how to make this problem computationally tractable.

Our solution is built on the robust foundation of Multiple Hypothesis Tracking (MHT). Unlike many modern approaches that are tightly coupled to specific deep learning backbones, MHT is a powerful and flexible framework for data association. It elegantly formulates tracking as a mathematical optimization problem: finding the most probable set of trajectories to explain the observed detections over time.

The MHT approach places its roots back to a seminal paper of D.B. Reid (IEEE Transaction on Automatic Controls, 1979), which is a suggested read for anyone new to the topic.

By focusing on the core logic of tracking, and just picking our preferred lightweighted visual feature embedder, we can create solutions that are both highly accurate and incredibly efficient. This allows us to deploy state-of-the-art tracking capabilities in environments where it was previously thought impossible.

1. Our Approach

Our expertise allows us to engineer and deploy MHT-based solutions across a wide spectrum of hardware, tailored to our clients’ specific needs.

for low power edge devices

We have developed a lightweight MHT implementation, inspired by Cox and Hingorani (IEEE Transactions on Pattern Analysis and Machine Intelligence, 1996), specifically for ARM-based embedded systems. Running on a device as modest as a single-core 1Ghz ARM processor, our system can track objects in real-time at 30 FPS while consuming less than 10% of the CPU. This is ideal for on-device intelligence in smart cameras, drones, and mobile robotics.

FOR HIGH-DENSITY CPU ENVIRONMENTS

Even when GPU resources are unavailable or fully utilized by other tasks, our advanced C++ implementation of MHT shines. This time our approach was guided by MHT Revisited (Kim et al., Proceedings of ICCV 2015). By leveraging powerful mathematical solvers like Gurobi and SCIP, we turn tracking into a highly optimized graph problem. This system achieves between 30 to 100 FPS on a standard Intel i7 CPU, delivering exceptional performance for applications like server-side video analytics without dedicated graphics hardware.

2. The Future Is Efficient

As more devices become intelligent, the need for on-device processing will only grow. The ability to run complex algorithms like MOT directly on the edge reduces latency, lowers infrastructure costs, and enhances data privacy. Our work in efficient tracking is perfectly aligned with this future.

While the academic benchmarks will continue to be a showcase for what’s possible with unlimited resources, we remain focused on delivering what’s practical and effective for real-world deployment. If you’re building a product that requires reliable object tracking without the overhead of a GPU, our expertise can provide a decisive advantage.

Before leaving, take a look down here to some of the MHT Revisited results on MOT17 challenge.

detection embedded multiple object tracking tracking
Related Posts