How Loom Compares

Loom is an edge accelerator. It is built for where GPUs cannot go, or don't pay: embedded systems, real-time control, direct radio I/O, and power-constrained autonomy, the envelopes where power, thermal, latency, and host-independence rule out a GPU. An autonomous inference engine for the physical world.

Modern inference is more than matrix math. Loom is built around the work that surrounds the matmul as much as the matmul itself: KV cache updates, gather/scatter, quant/dequant, reductions, TopK and sampling, masking, routing, low-batch decode, host-free loops, sensor- and event-triggered execution, and model-specific tensor plumbing. The comparisons that follow are organized around what each architecture does with that whole pipeline.

Most accelerators optimize for one of two goals: throughput (GPUs, TPUs) or rigid standard-compliance (DSPs). Loom targets low-power edge AI across scales, from compact embedded chips to Loom Node-class systems. The matrix shorthand below is the architecture that follows from that target. The sections after it work through the major comparisons in detail.

Datacenter inference processors win when the workload is rigid and the deployment is the rack. Two design choices define the category, and Loom takes a different position on each.

The Static-Schedule Trap

A common approach to datacenter inference latency is a fully static compiler that pre-calculates the location of every byte of data on every clock cycle before the model runs. This works for rigid, dense models. Frontier AI is conditional. Mixture of Experts, speculative decoding, and agentic branching depend on compute paths that are impossible to know until runtime data arrives. When a static architecture meets a dynamic workload, the options are: pad sequences, execute dead code, or stall the chip while the host recomputes the graph.

Loom embraces the conditional, data-dependent control flow of modern model architectures. Control flow lives in silicon. The on-die ACP evaluates state and gates decisions at silicon speed. When a speculative token draft fails or a sparse router selects 2 experts out of 256, the ACP evaluates a conditional skip and re-routes the MXU pipeline in flight.

HBM Dependency

Some inference processors target a 150-watt TDP but remain chained to High-Bandwidth Memory. HBM locks the hardware into expensive, heavily constrained CoWoS-class packaging, setting a high baseline for manufacturing cost and limiting where the part can be deployed.

Loom is designed to reach HBM-class effective bandwidth on commodity memory by manipulating data density. The MXU integrates hardware-native structured sparsity directly into the memory fetch path. Structural zeros are dropped in hardware with no software decompression overhead and no sparse-indexing penalty. Sparse weights physically occupy 50% less RAM and fetch at roughly 2x the effective bandwidth. For bandwidth-bound workloads like autoregressive LLM decoding, the design target is to match the throughput of HBM-equipped datacenter chips while operating in an embedded power envelope.

Classical DSPs and FPGAs anchored real-time signal processing for decades. They still have real strengths. What is changing is that the workloads sitting on top of them have shifted from fixed-function math to hybrid pipelines where neural inference, spectral transforms, and explicit data movement share one flow. Loom is built for those hybrid pipelines, with the determinism the classical parts are known for.

vs. Traditional DSPs

Tight inner loops vs. coordinated compute-and-memory pipelines

Where DSPs still win: A traditional DSP runs very small inner loops with extremely low overhead and extremely predictable timing. For a short FIR filter, a per-sample control loop, or a tight saturating fixed-point kernel, a classical DSP often feels more natural. DSPs offer friendlier fixed-point ergonomics, hardware loop support, and specialized addressing for ring buffers and FFT-heavy code.

Where Loom wins: Loom is designed for a different class of work: moving tiles of data through matrix math, vector and FFT-style operations, explicit data movement, and on-device control. A DSP is usually optimized for one tight loop. Loom is optimized for a whole pipeline. It is the better fit when low-precision AI numerics, software-managed local memory, and queue-driven orchestration matter more than squeezing one more multiply into a hand-tuned loop.

vs. Traditional FPGAs

LUT emulation vs. hardened datapaths · engineering velocity · talent pool

The reconfigurable silicon tax: FPGAs are the gold standard for cycle-accurate, hard real-time signal processing in defense and telecom. They achieve this by physically emulating logic using Look-Up Tables, limited DSP slices, and soft-routed wire fabrics. Building a large neural network or a complex spectral pipeline out of soft logic burns die area and power, and traps the engineering team in slow Verilog development, routing congestion, and overnight timing closures.

The engineering tax: Defense primes and telecom integrators struggle to staff enough Verilog and VHDL engineers to meet demand. A senior RTL engineer can spend months writing and verifying a custom DSP block that a PyTorch developer can prototype quickly in Python.

Loom's approach: Deliver the same deterministic execution as a bleeding-edge FPGA, with hardened, hyper-efficient systolic and spectral datapaths in place of the soft-routed fabric. Loom binds digital sensor interfaces directly to the neural compute graph without an OS or scheduler in the middle. Engineering velocity comes back: the team writes standard Python and Triton kernels instead of compiling hardware descriptions for 12 hours to test a new network layer. Stop compiling silicon. Start compiling software.

Most NPUs in this category are accelerators that require a host CPU to feed them work or manage control flow. Loom's ACP moves that logic on-die.

System Power, Not Chip Power

Supervisory host vs. real-time bottleneck

The NPU reality: Most edge AI chips market a 2-watt or 5-watt thermal design power. The number that doesn't appear on the data sheet is system power. These accelerators are coprocessors. They require a heavy application processor constantly awake just to fetch sensor data, manage the OS scheduler, pack the PCIe bus, and feed them matrices frame-by-frame. A low-power NPU can still demand a substantially larger host CPU just to keep it fed. The system power can be a multiple of the chip number.

Loom's approach: The on-die Autonomous Command Processor runs the entire real-time mission loop (Sensor → Inference → Action) natively. The host CPU drops to a slow-path supervisor: networking, telemetry export, OTA updates, deep sleep otherwise. Loom streams data directly from the sensor via DMA, executes the routing logic, and only sends a hardware wake interrupt to the host when a critical event needs broader system attention. The PCIe streaming power tax and constant OS polling overhead disappear, and the application processor downgrades to a lightweight management core. System power drops, sensor-to-action latency drops with it.

The matrix and the comparator sections are general. The argument lands when the matrix meets a specific physical deployment.

Robotics: Bypassing the OS Jitter Penalty

High-frequency control · hardware-native triggers · RTOS elimination

The Linux jitter tax: Modern robotics stacks (like Jetson GPUs running ROS2) are crippled by software bloat. When an IMU detects a drone tilting, that signal must fight its way through a hardware interrupt, the Linux OS scheduler, a ROS middleware node, a GPU driver, and a CUDA kernel launch. This pipeline can inject millisecond-scale, unpredictable jitter into the control cycle. Engineers are forced to bolt on separate microcontrollers running an RTOS just to handle the safety-critical timing.

Loom's approach: The Loom command dispatcher physically stalls at the silicon level until a physical signal (camera VSYNC, motor encoder pulse, IMU interrupt) arrives. The instant the pin goes high, the hardware interrupt forces a deterministic context switch, launching the inference graph in less than a microsecond. Hard, cycle-accurate control without an RTOS.

Streaming Sensors: Eradicating the Capture Buffer

Sub-frame inference · continuous DMA · row-by-row vision and live RF

The frame-buffer limbo: GPUs and NPUs are batch processors. To run vision or RF inference, an ADC or camera must capture an entire frame, push it through an Image Signal Processor, write it to system DRAM, and trigger an OS interrupt to wake the accelerator. This capture-then-compute pipeline guarantees tens of milliseconds of dead latency before a single math operation occurs.

Loom's approach: Hardware streaming DMA pipes video or raw I/Q radio samples directly from the sensor into an autonomous circular ring buffer inside the chip's SRAM, bypassing the host CPU, OS, and system RAM. The on-die ACP chases the write pointer and launches neural kernels on fresh sensor lines before the physical capture is complete. End-to-end inference latency can collapse, making ultra-tight closed-loop robotics and live RF channel equalization possible.

Private Edge: Generative AI at Batch-1

Interactive serving · on-prem appliances · air-gapped deployments

The batch-throughput trap: Datacenter GPUs are throughput engines. Frameworks like vLLM and TensorRT-LLM hit their marketed utilization by hoarding 64 to 128+ user requests into massive monolithic batches. For a single, real-time interactive agent at batch-1, the GPU architecture collapses. The core arrays can sit mostly idle, while generation speed is choked by the host CPU and PCIe bus orchestrating the autoregressive loop. For an on-prem or air-gapped deployment with a handful of concurrent users, this is the wrong cost curve.

Loom's approach: The MXU fast-paths GEMV (matrix-vector) operations to maintain peak utilization at batch-1. The on-die ACP handles Top-K sorting and token sampling locally, severing the host CPU and PCIe bus from the autoregressive loop. Loom natively executes the new wave of linear-time sequence models (Mamba, State Space Models, and Linear RNNs); the SCE processes continuous complex-phase state updates directly in the vector lanes, removing KV-cache bloat. 70B-class local inference becomes practical in a telecom closet, a forward-operating base, or a factory floor.

4-bit Compute: Mathematical Elegance Over Floating-Point Bloat

Silicon complexity · data-independent rotation · pure integer efficiency

The brute-force FP4 path: The common way to reach 4-bit inference is a complex block-scaled floating-point format such as NVFP4 (E2M1 with block-16 scaling). Because LLM activations contain large outliers that destroy naive quantization, these formats brute-force the problem with floating-point ALUs and a memory controller that constantly fetches fine-grained scaling factors for every block of elements. That requires offline calibration passes, burns silicon area on floating-point logic, and pushes power up hard.

Loom's approach: Fix the data at compile time, smooth the activations at runtime. The Loom compiler rotates the model weights offline and quantizes them once, so the stored weights are already in an incoherent space with the outliers spread out before they ever reach DRAM or ModelROM. At runtime, the Sparse Compute Engine applies a hardware Fast Walsh-Hadamard Transform to incoming activations in the vector lanes, rotating them into the same space so the products line up. The activation transform costs O(N log N), and that is the only runtime overhead. No offline calibration, no block-scaling, no floating-point rescale. Standard INT4 integer math becomes near-lossless, and the MXU runs it at integer speed. Frontier-class 4-bit fidelity using a fraction of the silicon area, at a fraction of the power.

Mixture of Experts: Routing in Hardware

Conditional execution · sparse activation · dynamic routing

The standard approach: On GPUs, MoE works well at high batch, where many concurrent requests amortize the cost of fetching scattered expert weights across HBM. The mechanism that makes it work is exactly what fails at low concurrency: to keep the parallel pipeline full, the GPU pads batches toward the worst-case expert count, and at batch-1 that padding is almost all structural zeros and idle cores. The router selects a few experts, but the hardware still pays for many, and multi-device MoE adds NVLink scatter-gather and host orchestration on top. For an on-prem appliance serving a small number of live users, that is the fatal case.

Loom's approach: MoE routing is solved in the memory controller, and it costs the same at batch-1 as it does at batch-many. The MXU computes the gating layer; the on-die ACP evaluates the Top-K active experts; the ACP hands those indices directly to the Smart DMA, which autonomously chases the scattered expert weights across commodity DRAM and packs only the selected experts into a continuous, dense stream for the systolic array. No padding to a worst-case count, no host round-trips, no batch needed to hide the latency. The MXU stays fully utilized on a single request through a dynamically routed model, so fine-grained architectures like DeepSeek-V3 run at silicon speed on a local box.