Performance debugging usually begins where developers have the most control: the application code. A model may meet its target in a local benchmark, a function-level profiler may show no clear slow function, and the first round of configuration changes may look reasonable.

The first case came from an industrial computer vision system with a production throughput target in the mid-80 FPS range per camera. Two camera streams ran on the same hardware, each in its own application process, while neural network inference was executed through ONNX Runtime on a single shared NVIDIA GPU.

That shared GPU was the central constraint. Because the system used pipelining across CPU cores, the full processing path did not necessarily need to fit inside one frame interval. What mattered was whether GPU-side inference, data movement, and runtime scheduling could keep up with the combined workload without starving the rest of the system of CPU headroom.

This made a timeline view essential. A function-level profiler could show how long the application code took, but it could not explain how the runtime submitted work, when the GPU executed it, or whether the two processes were competing for the same device in a way that limited throughput.

The system initially used the CUDA Execution Provider in ONNX Runtime. An Execution Provider is the backend that decides how ONNX Runtime maps and runs a model onto available hardware. On NVIDIA hardware, CUDA EP is the standard general-purpose GPU backend, while TensorRT EP is a more specialized optimizing backend that can build device-specific engines for faster inference.

The backend change created a second problem: startup became long and occasionally unstable. Nsight was useful again, this time not for inference latency but for startup diagnosis. The timeline showed repeated TensorRT engine-building work on the GPU, performed across multiple sessions in sequence. This was consistent with how TensorRT EP works: during engine construction, it profiles candidate kernels and builds an optimized engine for the model and device. The fix was not another round of application refactoring. It was caching. In this deployment, enabling timing and engine caching preserved the expensive build work across runs, so startup no longer repeated the same kernel-selection process every time. The startup instability disappeared, and the two camera applications could run together within the required throughput envelope.

Finally, Nsight helped characterize robustness rather than raw speed. The team used it to observe behavior under different contention patterns: workers hitting the GPU at the same time versus workers alternating. The exact latency of an individual inference mattered less than the bounded behavior of the full system. Under contention, GPU work serializes; under alternation, individual inferences stay shorter. The value of profiling was to measure both regimes clearly enough for the deployment team to accept the system, not to pretend there was a single magic latency number. This matters because industrial AI systems are judged by sustained operation. A backend change that improves a benchmark but destabilizes startup or fails under concurrency is not a production win. In this case, Nsight Systems provided the evidence for the change, the diagnosis for the side effect, and the robustness data needed to close the loop.

The second case came from a web product that needed browser-native GPU inference. The goal was to run on the user’s device, directly in the browser, without a cloud round trip or native installation, while staying close to a real-time 60 FPS interaction target. That combination is commercially attractive: easier distribution, lower infrastructure dependency, better privacy posture, and lower user friction.

The pipeline used WebGPU through ONNX Runtime’s WebGPU backend. WebGPU is the modern web API for GPU compute and rendering. Conceptually, it exposes GPU adapters, devices, queues, buffers, textures, and command submission to web applications, while the browser implementation maps that work onto native graphics APIs such as Direct3D 12, Metal, or Vulkan.

That architecture is exactly why debugging WebGPU performance is not the same as profiling CUDA. With Nsight on a native NVIDIA stack, you can often see CUDA kernels, memory transfers, and device activity with high fidelity. In a browser, WebGPU intentionally abstracts the hardware and mediates access for security, and portability. Any timeline profiler may show process coordination and GPU-track events, but it will not automatically give the same kernel-level view of what a native CUDA profiler would expose.

The reported issue was a latency gap in the in-browser pipeline. The natural starting point was to investigate the application-owned GPU work and the model execution path. For this kind of browser problem, Perfetto was the appropriate tool because Chrome emits trace events across the renderer process, GPU process, and related subsystems. Perfetto could show the timing relationship between those components on a single timeline.

The delay is documented in source comments as intentional: it helps avoid unstable alternating fast/slow frame behavior when multiple command buffer stubs compete for GPU time. In other words, the latency was not a bug in the application shader, a missing model optimization, or a driver mystery. It was a browser scheduling choice made for system-level stability.

That finding changed the optimization plan. The application-owned GPU work was too small to recover the missing frame budget by itself. ONNX inference remained the dominant model-side cost, and the browser polling interval was external to the application. Optimizing the small shader more aggressively would have produced a technically satisfying improvement in a place that did not move the total system enough.

The practical lesson was about synchronization boundaries. If a pipeline submits GPU work, waits, submits another small piece of GPU work, and waits again, it can pay browser coordination costs multiple times.

The two investigations looked different on the surface: one was a native NVIDIA deployment, the other a browser-based WebGPU product. But the working pattern was the same. The first useful question was not “which line is slow?” but “which layer owns the time we are missing?”

If your team is hitting performance targets in development but losing them in production, or seeing behavior that application-level profiling cannot explain, this is the kind of production-level performance investigation we carry out regularly. Reach out at info@deepvisionconsulting.com to discuss your specific situation.