arXiv subjects

Raj Mali

Publications and source records attributed to Raj Mali.

3 recordsLinked to original sources

arXiv2026

Hardwired Distributed Processing Units for Inference: Neural Decisions at the Sensor that Free the Host

We show that a hardwired distributed processing unit (HDPU) can take a neural decision at the sensor and so remove the inference from the host. Built for zebra-crossing detection on a low-cost Artix-7 FPGA, the unit decides on every camera frame in a fixed 2,269,477 clock cycles (34.04 ms depends upon Fpga clk ), inside the 37.49-ms frame period, and on the board it returns the same scores and decisions as its bit-exact software model on all 639 test images. The whole detection path, from camera pixels to decision, takes 1,814 LUTs, with no processor, no operating system and no full-resolution frame store. When the unit answers instead of reporting "not sure", 95.7% of its answers on street photographs are correct, and a hardware contrast stage lifts accuracy on camera-like dashcam frames from 47.7% to 78.2%. The host is spared all of the inference: 8.13 ms of a core in every frame period, where a full-precision ResNet-18 would need more than a frame period even on 16 threads. It also receives 16 bytes per frame instead of 614,400, 38,400 times less. The unit is an 80x60-pixel pipeline feeding a four-layer binarized CNN with 93,408 one-bit weights, verified bit-exactly from a Python model through RTL simulation to the board. Earlier HDPUs did pre-processing (edges, corners, steering); this one shows that the decision itself can move to the sensor.

cs.AR↗

arXiv2026

Hardwired Distributed Processing Units at the Sensor: Truly Parallel Vision Pre-Processing that Frees the Host CPU

Conventional vision pipelines route every pixel to a host processor, which must then wait for a complete frame before it can act and devote part of each frame period to routine pre-processing. We examine an alternative organisation in which this work is distributed across hardwired distributed processing units (HDPUs): small, fixed datapaths attached directly to the sensor, each committed to a single always-on task, operating truly in parallel on the same pixel stream and delivering compact results to the host. From one library of 3x3 operators sharing a common two-clock latency, we construct three HDPUs (Canny edges, Harris corners, and a line tracker that issues a steering decision) and run them side by side on a single camera stream in a low-cost Artix-7 FPGA, at a cost of 2,568 LUTs and an estimated 0.318 W. Every stage of every unit is bit-exact with its software model. In a full-system simulation with a datasheet camera model, each decision leaves its HDPU within 11.03 camera lines of the pixel that produced it, 22-67x sooner than the earliest moment at which a frame-based program on the host could respond; the HDPU latency varies by no more than 17 ns, whereas the host's own run times reached 12.5x their median. Offloading the three tasks returns 18-30% of a core to the host in every frame period at 640x480, and 63% of the frame period, with all threads in use, at 1080p. The cost of the units grows with frame width only through their line buffers, and in a closed-loop model of a line-following vehicle the HDPU raises the highest speed at which the line is held from 4.75 to 10.00 m/s.

cs.DC↗

arXiv2026

Fast Path, Exact Path: One Library of Hardwired Operators for Streaming and Exact Canny Edge Detection on a Low-Cost FPGA

Hysteresis edge tracking, the final stage of Canny edge detection, follows edge chains of unbounded length and therefore resists single-pass streaming; FPGA designs either approximate it or dedicate separate labelling hardware to it. We show that a single library of hardwired 3x3 operators, each with an identical two-clock latency, delivers both a camera-rate approximation and exact hysteresis on a low-cost Artix-7 (XC7A35T). The fast path streams camera pixels through the operators and K local hysteresis stages and commits each edge decision 262.15-720.63 microseconds after the pixel arrives (K = 1-8; system simulation with a datasheet camera model), that is, 3 + K camera lines later and 46-127x sooner than a frame-based processor can even begin at 30 frames per second. It provably never reports an edge that exact hysteresis rejects, recovers 76.6-96.1% of exact edges at thresholds 20/40, and on the 200 BSDS500 test images retains at least 98.6% of the exact path's boundary F-measure (ODS 0.470-0.477 versus 0.476). The exact path holds 128x128 tiles in block RAM, is exact within each tile, and replaces two-buffer iteration with in-place forward and reverse sweeps built into the same operator. This reduces hysteresis from 38-82 passes to 3-7 on pattern and photo tiles, accelerating the hysteresis step 6.6-21.7x (40x on a worst-case tile) and a complete tile 4.5-9.9x, with identical output. Both paths are bit-exact with software golden models at every pipeline stage and close timing at 50 MHz in at most 1455 LUTs. An optimised multi-core CPU remains faster on stored images, but it cannot begin until the frame is complete.

cs.AR↗