A robotic inspection system is often described as a robot carrying a camera and an AI model. In practice, making such a system work requires much more.
The robot must understand its surroundings, acquire consistent visual information, move safely, communicate its findings, and fit into the way people already make quality decisions.
This article looks beneath the visible AI layer. Using a real industrial project as a practical example, it explores the engineering that connects calibration, localization, robot motion, image analysis, spatial feedback, human judgment, and reporting. The central question is not how to automate every decision, but how to combine machine repeatability with the experience people bring when an inspection result is not clear-cut.
This balance was explored in RE-PAIR, developed by Deep Vision Consulting with Bonfiglioli as the industrial end user. RE-PAIR was selected through an open call of JARVIS, an EU-funded Horizon Europe project focused on AI-driven, multimodal interaction for advanced user-centric human–robot collaboration. The pilot applied JARVIS path-planning and augmented-reality tools to the inspection of returned forklift axle and motor-reducer assemblies.
The industrial task was remanufacturing: before a returned axle can be repaired and put back into service, Bonfiglioli must understand its condition and determine which components can be recovered. The resulting robotic inspection system supported this decision by automating repetitive activities such as moving between predefined viewpoints, acquiring images, identifying obvious visual differences, and consolidating the collected information into a report. AI was only the most visible layer of a broader system encompassing calibration, localization, path planning, projection, and integration, while human expertise remained central whenever findings were ambiguous and required an understanding of acceptable wear, repairability, and operational context.
TABLE OF CONTENTS:
1. Why a Robotic Inspection System Goes Beyond Image Analysis
2. The End-to-End Inspection Pipeline
3. Calibration: Building a Common 3D World
4. Pose Estimation: Letting the Recipe Follow the Component
5. Path Planning: Turning Viewpoints into Safe Robot Motion
6. Repeatable Acquisition and AI-Based Anomaly Detection
7. AR Projection: Connecting Digital Results to the Physical Component
8. Human Validation and Automated Reporting: Closing the Loop
9. Industrial AI Is a Human-Centered Systems Problem
1. Why a Robotic Inspection System Goes Beyond Image Analysis
Remanufacturing begins with understanding the condition of a returned component before deciding what can be repaired, recovered, or reused. For a large mechanical assembly, this traditionally means that an operator must move or reposition the component, work around its complex geometry, photograph relevant areas, identify possible defects, document the evidence, and prepare a report.
Image analysis can help identify visual differences, but it addresses only one step in that workflow. If the operator must still position the assembly, find each inspection area, reproduce suitable camera views, connect every finding to a physical location, and organize the results, most of the inspection burden remains unchanged.
A complete robotic inspection system links those activities into a repeatable process. It localizes the component, adapts to its position, moves safely around complex geometry, acquires comparable images from defined viewpoints, analyzes them, returns each finding to the correct physical location, and collects the validated evidence in a structured report.
This does not make the human operator less important but it makes better use of the operator’s expertise. The system takes on repetitive work such as revisiting predefined areas, acquiring images, flagging clear visual deviations, organizing results, and preparing reporting evidence, while the operator determines what ambiguous findings mean in practice.
That judgment is essential because ambiguity is not an exceptional edge case in industrial inspection. A scratch may be irrelevant in one location and critical in another, while a change in surface appearance may be acceptable, repairable, or evidence that a component should not be reused. The most useful automation therefore concentrates human attention where knowledge of tolerances, repair criteria, component history, and operational consequences has the highest value.
2. The End-to-End Inspection Pipeline
Turning this division of responsibility into a working inspection process requires two connected phases: configuration and execution. The process begins with the operator defining what must be inspected and recording the required viewpoints on a reference component. For each returned component, the system then localizes the assembly, adapts those viewpoints to its position, plans the robot’s movements, acquires and analyzes the images, presents the findings for human validation, and generates the final report.
During configuration, an operator creates an inspection recipe using a known, defect-free component. The operator manually guides a collaborative robot carrying a tool equipped with a 2D camera and a projector. At each relevant area, the system stores the robot viewpoint and a reference image.
The system links each inspection view to a specific area of the axle rather than to one fixed position in the work area. As a result, the same inspection procedure can be reused even when the next axle is placed differently, reducing setup time and avoiding repeated robot programming.
Projection supports the operator while the recipe is being created. The system can display the camera’s field of view directly on the axle and show in green the areas included in the planned analysis. The operator can therefore understand the inspection coverage while interacting with the physical component instead of continually looking at a separate monitor.
During execution, when a returned axle is ready for inspection, the system:
- localizes the component;
- adapts the stored viewpoints to its current position;
- plans safe robot movements;
- acquires images from repeatable viewpoints;
- applies AI-based analysis;
- projects the results onto the axle;
- asks the operator to validate the findings;
- generates the inspection report.
Viewed as a whole, the robotic inspection system is a chain of spatial, robotic, visual, and human decisions. A failure to preserve context at one stage reduces the usefulness of the next. Accurate detection, for example, has limited value when the operator cannot determine where the reported anomaly is located on the physical component.
3. Calibration: Building a Common 3D World
The pipeline described above can work only if every part of the system agrees on where the axle and its inspection areas are located. The camera observes the component, the robot positions the camera-projector tool, the projector places guidance on the physical surface, and the axle’s CAD or mesh model represents that same component digitally. Each starts with its own way of describing position, so their information cannot be combined reliably until those spatial relationships are established.
Spatial accuracy at this stage influences every result that follows. Even a small mismatch can lead the robot to a slightly different viewpoint, make images less comparable, or cause a defect detected correctly in an image to be projected onto the wrong area of the axle. In that situation, even an accurate AI model produces results that are difficult for the operator to trust or use.
Calibration establishes this accuracy by measuring how the system’s elements are positioned relative to one another. The camera provides the central link:
- markers help the camera determine where the physical axle is located;
- a prior camera-robot calibration connects what the camera sees to the robot’s workspace;
- a camera-projector calibration tells the system where projected information will appear.
Together, these relationships create a common 3D understanding shared by the robot, camera, projector, physical component, and digital model. A finding that begins in a camera image can then be connected to the correct area of the axle, used within the robot workspace, shown to the operator on the physical surface, and recorded with its location intact.
Calibration is therefore not a peripheral setup task. It is what turns several separate devices into one coherent robotic inspection system and preserves spatial accuracy from image acquisition to operator feedback.
4. Pose Estimation: Letting the Recipe Follow the Component
Calibration gives the devices a common spatial understanding, but the system must still determine where each newly returned axle is positioned within that shared 3D space. This is necessary because operators cannot be expected to place every large component in exactly the same location and orientation.
The inspection recipe created during configuration is a reusable record of which areas must be examined, from which views, and against which reference images. Pose estimation allows that recipe to follow the component by determining its current three-dimensional position and orientation. To provide a reliable visual reference, the operator places an ArUco marker in a known position relative to the axle. The camera recognizes its distinctive pattern, allowing the software to locate the axle relative to the camera and robot. Because the inspection views are linked to specific areas of the axle, the system can adjust them to its current placement. The inspection procedure therefore does not need to be recreated or the robot manually reprogrammed each time another axle of the same type arrives. For the robotic inspection system, this separates two concepts that are often confused:
- Configuration defines which areas should be inspected for a component type
- Localization determines where the current component instance is positioned
This separation makes expert knowledge reusable. A quality-control specialist can define the inspection intent once, while the software manages later changes in placement.
The broader lesson applies beyond axle remanufacturing. Whenever a robot must interact with variable component positions, task knowledge should be expressed relative to the object whenever possible, not embedded exclusively in fixed machine coordinates.
5. Path Planning: Turning Viewpoints into Safe Robot Motion
Pose estimation tells the system where the axle and its inspection areas are located. The next challenge is turning that information into actual robot movements, because knowing where the camera must capture an image does not determine how the robot should reach that position.
Moving around a large mechanical assembly requires more than sending the robot directly from one inspection point to another. The route must account for the axle’s geometry, the robot’s possible configurations, the camera-projector tool, and other obstacles in the work area. A movement that appears short may otherwise bring part of the robot or its tool too close to the component.
Path planning calculates feasible, collision-aware movements between the required inspection views. Within RE-PAIR, the JARVIS Robot Trajectory and Path Planning Interfaces supported these movements and enabled the configured inspection procedure to run automatically.
This creates a clear division of responsibility. The operator decides which surfaces and views matter for quality control, while the planning software determines how the robot can move between them. As a result, inspection expertise does not have to become robot-programming expertise. Quality-control personnel can focus on the component and the inspection requirements instead of manually defining every movement, while procedures can be adapted more easily for different component variants. This separation is particularly valuable in high-mix environments, where inspection procedures change more frequently than on a production line dedicated to a single product. Reusing inspection intent while calculating the required motion automatically reduces the effort needed to introduce or update a procedure.
For another example of trajectory planning applied to industrial robotics, see DVC’s case study on dynamic trajectory planning for anthropomorphic and Cartesian robots.
6. Repeatable Acquisition and AI-Based Anomaly Detection
Path planning allows the robot to return the camera to the required inspection views. This repeatability matters because changes in angle, distance, framing, focus, illumination, or occlusion can appear to the AI as changes in the component itself.
Once images are acquired under comparable conditions, anomaly detection can highlight candidate differences such as missing elements, surface damage, or unexpected visual features. The system directs the operator’s attention to these deviations, while the operator decides whether they are acceptable, repairable, or relevant to the component’s reuse.
For manufacturers facing similar visual-inspection challenges, DVC also offers its proprietary Defect Visual Inspection (DVI) system. DVI is designed for defects that are visually recognizable but difficult to define through fixed rules, and it can be trained from a small set of nominal images without requiring examples of every possible defect. As a deployable AI inspection solution, it can complement controlled image-acquisition systems by highlighting candidate anomalies and refining its performance through operator feedback.
7. AR Projection: Connecting Digital Results to the Physical Component
Anomaly detection identifies a region in a camera image, but the operator must act on the corresponding location on the physical axle. On a large component with similar surfaces and inspection points distributed around its geometry, finding that location from an image alone can be difficult.
AR projection closes this gap by using the calibrated relationship among the camera, projector, robot, and component to place digital guidance directly on the axle. In RE-PAIR, JARVIS AR interfaces supported this interaction. During configuration, the projector shows the camera’s field of view and highlights the areas included in the inspection recipe. During review, green indicates an inspected area with no identified anomaly, while a blinking red marker directs the operator to a finding that requires attention.


Figure 5. Augmented reality projection indicating valid inspected areas (in green) and highlighting detected anomalies (in red) directly on the component.
Projection therefore returns digital evidence to the place where the decision must be made. The operator no longer has to interpret an image on a monitor, infer its orientation, and then search for the same feature on the physical component.
8. Human Validation and Automated Reporting: Closing the Loop
Projection brings each candidate anomaly to the operator’s attention on the physical component, but the final interpretation still depends on human expertise. By combining the acquired image, the highlighted location, and knowledge of repair criteria, the operator can dismiss a finding, request a repair, or determine whether it affects the axle’s suitability for remanufacturing.
Once the findings are validated, the system generates a structured report containing the component information, inspection images, annotations, and confirmed observations. Close-up images show the analyzed areas, while a 3D representation of the axle preserves the wider location of each finding.
Automated reporting removes the need to gather images, transfer notes, and reconstruct the inspection record manually. More importantly, it closes the evidence chain by turning AI outputs and human decisions into information that can be reused during repair, quality control, remanufacturing, and audits.
9. Industrial AI Is a Human-Centered Systems Problem
RE-PAIR shows that the central challenge in industrial inspection is not simply detecting a difference in an image. It is carrying trustworthy evidence from a large physical component to a decision that can guide repair, reuse, or rejection.
A robotic inspection system creates value when that chain remains intact. The inspection procedure follows the component, the robot acquires comparable images, AI directs attention to possible anomalies, projection connects those findings to the physical surface, and the validated result becomes a usable record. Weakness at any point reduces the value of everything that follows, regardless of the model’s accuracy.
Human involvement is therefore not a limitation of the system but part of its design. Automation takes on repeatable movement, evidence collection, initial screening, and documentation, while operators define what matters and interpret findings whose significance depends on repair criteria and operational context. Their expertise becomes easier to apply consistently rather than being removed from the process.
For industrial teams, the broader implication is that an AI inspection initiative should be evaluated as a complete workflow, not as a model purchase. The important questions include how the system adapts to component placement and variation, controls image acquisition, communicates findings, supports judgment, and preserves the resulting decisions.
Purposeful automation is not measured by how many human decisions it eliminates. It is measured by how effectively it removes repetitive burden while giving people better evidence for the decisions that still require experience. To explore how robotics and computer vision could support that balance in your inspection process, see DVC’s computer vision consulting services.
Frequently Asked Questions
To complement the discussion above, here are concise answers to some common questions about the topic, from basic definitions and key distinctions to practical engineering considerations.
A collaborative robot is designed for applications in which people and automated equipment operate in the same work area. In this inspection workflow, the operator can manually guide the robot while defining which areas must be inspected, after which the robot can return to those views automatically. Safe operation still depends on the complete application (including the tool, movement speeds, workspace, and appropriate risk assessment), not on the robot label alone.
Visual markers provide a reference pattern that a camera can recognize reliably. An ArUco marker uses a distinctive black-and-white design; when its relationship to the axle is known, detecting it allows the system to calculate where the axle is positioned. Markers are one practical localization method, although other applications may use component geometry or additional sensors instead.
Anomaly detection is useful when acceptable appearance is easier to demonstrate than every possible defect is to enumerate. The model learns or references expected visual patterns and highlights deviations for review, but a deviation is not automatically a defect: acceptable wear, repairability, and operational relevance may still require human judgment. Consistent camera viewpoints and acquisition conditions are essential so the model responds to changes in the component rather than changes in the image.
Reliable performance depends on the complete inspection workflow rather than on the AI model alone. The system needs accurate calibration and localization, safe and repeatable robot motion, consistent image-acquisition conditions, and a clear process for human validation. These elements must work together so that findings remain connected to the correct physical location and can be turned into trustworthy inspection records.
Related Posts
Why a Timeline Profiler Finds What a Code Profiler Misses
The right profiler is not the most powerful one in the abstract. It is the one that can see the…
How We Built Metrically Accurate 3D Annotations from a Single Camera
We show how markerless camera calibration can be approached in urban environments and beyond.
AI and 3D Vision on the Renesas RZ/V2H: From Edge Processing to Smart Camera Reality
A complete AI and 3D stereo vision pipeline running on a single embedded SoC to lay the foundation…
Multi-modal foundation models out of the lab: a reality check
In this post we discuss the applicability of multi-modal foundation models (VLM) to solve real…
Onsemi Hyperlux ID AF0130: Industrial iToF Depth Sensor Evaluation
Hands-on evaluation of the Onsemi Hyperlux ID AF0130 iToF sensor. Real-world testing reveals…
Beyond the Frame: What Are Event Cameras and Why Do They Matter?
A deep dive into event cameras: the hardware, software challenges, and real-world applications of…















