3D+AI is a software library for real-time dense 3D reconstruction from stereo cameras, using deep learning.
It was designed with the goal to accelerate the integration of metric 3D vision for OEMs in mind. This solution provides incredibly detailed 3D metric data in real time on embedded devices, making high-fidelity 3D vision accessible for various industrial applications.
3D vision means having depth, point clouds and metric data, that are necessary in many use cases from industries like automotive, robotics, agritech, industrial automation and others.
Approaching 3D vision with deep learning techniques is possible and unleashes greater opportunities if compared to 3D vision with traditional techniques; however, technical and computational complexity increases significantly.
And when deep learning for 3D vision needs to be applied at the edge, the matter can get extremely intricate.
3D+AI addresses these 3 elements together (3D reconstrucion with deep learning at the edge) and brings to OEMs a ready-to-use solution.
Input left-right pair
OPENCV traiditional stereo matching algorithm vs 3D+AI depthmap
3D+AI pointcloud
KEY FEATURES
3D+AI enables OEMs to create their own custom stereo camera, without having to invest time and money in actually developing it. What follows is a list of all things that can be customized if 3D+AI is employed:
Optics
Choose the lenses more suitable for your use case.
Sensors
Pick your preferred camera modules with hw-sync.
Baseline
It can be adjusted to arbitrary values to capture fine details or cover more distance.
The 3D+AI software library is optimized to work on devices supporting the Hailo AI accelerator, such as:
- NXP i.MX 8M Plus with Hailo-8 (embedded solution)
- Hailo-15™ (SOM solution)
- Intel® Core with Hailo-8 (PC solution)
Regardless of the accompanying hardware, the stereo matching step is offloaded to the Hailo AI accelerator, leaving the CPU free for further processing of the point clouds. As opposed to other commercially available stereo cameras, this means that there is no need for additional external computation (e.g. a PC). As a result, you can create your 3D vision logics directly on the chosen hardware platform or, for more complex tasks, get custom 3D vision developments from our expert team.
The following table contains performance results for different choices of hardware and input resolution.
| Input Resolution | Max Disparity | Point Cloud Size | FPS with Hailo-8™ and Intel® i7 | FPS with Hailo-8™ and NXP i.MX 8M | FPS with Hailo-15 |
|---|---|---|---|---|---|
| 512x384 | 128 | 197.000 | 40 | 24 | 26 |
| 768x576 | 128 | 442.000 | 23 | 14 | 15 |
EXAMPLES OF 3D+AI IN ACTION
Check below the variety of use cases where 3D+AI had a strong edge over traditional solutions, without requiring external PCs or expensive hardware / GPUs. Everything runs on low power, industrial grade, deployment ready embedded boards.
Images from a robot patrolling a greenhouse and checking size and count of vegetables. Robots running on battery cannot affort power intensive GPUs or traditional PCs. Our 3D+AI algorithm can run on low power embedded devices, without draining battery life.
A driving scene for vehicles with need of 3D perception for either autonomy or increased safety features. Real time metric information are critical in such scenarios. Missing information (e.g. as it might happen with more traditional algorithm) can lead to unexpected behaviors.
Disparity maps from an industrial pick-and-place application. Traditional image processing on 2D images struggles with complex shapes and unfortunate illumination.
Application of 3D+AI algorithm in an agriculture use case. 3D analysis can greatly simplify following vision algorithms.
Result on random pattern projector + IR camera over texture-less objects. 3D+AI algorithm can seamlessly run on traditional RGB images and images from other spectrums (NIR in the example above). With proper illuminators / projectors, can work in the dark as well.
HOW TO GET IT
3D+AI is licensed with an unlimited perpetual license model. See details below.
For pricing information, get in touch through the contact page.
Unlimited Perpetual License
- Up to 1000 devices per year, per project
- On single hardware platform (Hailo-8 on i.MX 8M Plus or Intel Core or Hailo-15)
- On single software dependency configuration (Hailo SDK version, Yocto based BSD or Ubuntu version, etc)
- 10h jumpstart support
Dependency Update Pack
- Port the Unlimited Perpetual License to a different software dependency configuration among the supported ones
Support Pack
- 20h additional support
- examples: extra support over documentation, design of stereo camera parameters, stereo calibration, other vision tasks.