Loom Node

The GPU Infrastructure Bottleneck

Large-model inference is often limited by memory system design before it is limited by raw math. A 70B model at FP16-class weight storage is roughly 140 GB before context and runtime state. A 405B dense model is roughly 810 GB at the same precision class. That pushes conventional deployments toward multi-card systems, HBM allocation constraints, complex chassis, and expensive cooling.

The supply chain problem compounds the architecture problem. Interposer packaging, HBM allocation, and proprietary scale-up fabrics are procurement bottlenecks.

The Loom Node

The Loom Node takes a different path. Four Loom 128-class SoCs sit on a standard board, each with its own local memory domain. Loom provides board-level aggregate capacity and bandwidth without HBM, coherency hardware, or exotic packaging.

It is a deliberate pipeline-parallel and peer-DMA design: each chip keeps its own weights and working set local, and only the data that has to move crosses the switch.

Who this is for

The demand for large local memory is not limited to hyperscale inference. Research labs, creative teams, field programs, industrial deployments, and secure on-prem environments all need more model capacity and more local working memory than conventional single-accelerator systems expose.

This reference design is the starting point for that chassis: the topology, the power and thermal envelope, and the integration surface, built entirely from commodity parts around the Loom SoC. The same topology drops into a rackmount inference server, a multi-service edge appliance, a retrieval-heavy on-prem box, or a field-deployable analysis unit; the integrator picks the enclosure, cooling class, I/O, and socketed or direct-attach commodity memory, and the chip-to-chip data path stays the same.

These are fit classes, not blanket guarantees. Actual deployment headroom depends on precision, sequence length, KV/state size, service concurrency, and the chosen pipeline split.

HBM scarcity forces the industry into small models and short contexts. A terabyte of standard off-package memory lets you stop compromising. Capacity alone is not enough: memory bandwidth and compute still have to be managed.

The examples below are a selected set of long-context and large-model workload concepts that fit on a single board with 1 TB of local memory.

In-Memory Vector Search

1 TB holds ~500 million FP16 vectors

Disk-based RAG relies on slow I/O. But a brute-force linear scan of 500 million vectors takes less than a second, limited only by DRAM bandwidth. 1 TB lets you keep the embedding index resident. You can do fast coarse filtering in memory and then exact cosine reranking on a candidate set natively in the MXU.

The Omni-Router

Global BGP and packet graphing

A standard core router like a high-end Cisco ASR maxes out at 8-16 GB of RAM just to hold a few full views of the global BGP routing table, roughly 1.2 million active IPv4/IPv6 prefixes. With 1 TB of RAM and a 100GbE NIC dumping traffic directly into memory via streaming DMA, the Loom Node holds the active routing table, the entire historical BGP routing graph, the last 24 hours of flap history, and a massive ring buffer of raw PCAP data simultaneously. You can continuously run graph neural networks or sequence models over the routing table to detect BGP hijacking, anomalous route leaks, or peering failures. For a wireless operator or ISP, the entire routing picture, live and historical, sits on one node that can reason over it continuously.

Line-Rate Network Inspection

Full payload analysis at wire speed

At sustained 100GbE, 1 TB buffers on the order of 1-1.5 minutes of full payload; longer windows are possible at lower average rates or with selective capture. The Loom Node executes wire-speed capture and indexing in RAM, plus real-time detection on metadata and flows. Deep payload inference can then be applied to suspicious traffic or traffic windows, avoiding the compute wall of running sequence models over every raw byte. No cloud. No sampling. No static firewall signatures.

Multi-Agent Swarms

Massive concurrent simulation

1 TB provides the capacity to host a large population of agent models resident in RAM: a hundred instances of an 8B model, for example. Execution requires running subsets concurrently or time-slicing across many agents depending on your specific throughput needs, avoiding the disk-swapping that kills local multi-agent frameworks.

The RF Time Machine

Buffer the spectrum, analyze the past

With one chip dedicated to a software-defined radio baseband via continuous streaming DMA, 1 TB of memory acts as a massive ring buffer for raw I/Q spectrum data. Buffer minutes of uncompressed 100 MHz-wide radio spectrum in RAM. If an anomaly, jammer, or brief signal spike occurs, the reasoning chips can pause, rewind the memory buffer, and perform deep spectral analysis on events that already happened.

City-Scale RF Propagation

3D Fourier Neural Operators at scale

Enables large 3D RF simulations that would otherwise crash GPUs. By holding high-resolution topographical data in memory and utilizing the Sparse Compute Engine's FFT block, the user receives performance at interactive runtimes for practical city grid tiles.

Wide-Area Situational Awareness

Every camera in one resident tensor

Multi-camera video maps time, space, and view into a massive 4D attention space, and memory consumption scales hard with the number of feeds and the length of the window held resident. With a terabyte of local memory, the Loom Node can keep an immense 4D spatial-temporal tensor of high-definition feeds in RAM at once, enabling real-time multi-camera tracking and trajectory reasoning across an entire facility or deployment zone without frame drop or disk thrashing. The reasoning chips can hold minutes of every camera at once, correlate a track as it moves between views, and reason over the recent past of the whole scene rather than one frame of one feed.

Deep Reasoning at Scale

Trees of Thought without HBM exhaustion

The industry is pivoting from "predict the next word" to "think for 5 minutes before answering." Models like DeepSeek R1 and OpenAI o1 generate thousands of hidden reasoning tokens before responding. They branch out into massive Trees of Thought. Branching a context state rapidly exhausts standard GPU memory. The Loom Node enables many branch states to be stored and revisited without exhausting memory. Practical active-branch throughput reflects model size and chosen Monte Carlo Tree Search strategy.

Inline Interpretability

Read the model's mind in real time

Sparse autoencoders project a model's hidden state into a 100x wider dimension to find human-interpretable concepts like deception, financial risk, or toxicity features. Running an SAE alongside a 70B model requires hundreds of gigabytes of extra memory just for the sparse dictionary weights. The Loom Node can hold the base model, the KV cache, and the massive SAE dictionary simultaneously, monitoring or steering the model's internal representations live, token by token.

Continuous Edge Learning and Self-Play

Closing the AI loop offline

Full-model continuous training remains a heavyweight compute workload. However, 1 TB provides enough memory headroom for parameter-efficient fine-tuning (LoRA) and periodic on-device adaptation. It supports self-contained reinforcement learning and self-play experiments without cloud dependence. Reasoning models are trained with reinforcement learning from self-play: generate millions of synthetic reasoning trajectories, score them with a reward model, and train on the best ones. Achievable scale depends on model size and your training time budget.

Billion-Node Graph Neural Networks

Supply chains and fraud detection

Fraud detection, supply chain logistics, and drug discovery rely on massive Graph Neural Networks. These workloads are notoriously hostile to disk access. Keeping the full graph resident avoids disk thrash and enables high-throughput graph analytics.

Genomic Sequence Workloads

Resident cohorts without I/O wait

1 TB keeps large reference datasets and cohorts resident, reducing disk I/O. Loom can accelerate alignment and variant workflows and run sequence models locally. Protein-structure prediction is a separate, compute-heavy workload with its own distinct performance profile.

Hyper-Fuzzing and Kernel State Modeling

Autonomous vulnerability discovery

Discovering zero-day vulnerabilities in operating systems, specifically within complex memory management units, hypervisors, or system shells, has traditionally relied on brute-force fuzzing. The newest frontier of large context models is pushing windows from 1 million to 100 million tokens. You can fit an entire OS codebase, thousands of hypervisor state snapshots, and millions of execution traces into a single active context. Instead of throwing random data at a kernel, the Loom Node holds the entire semantic state of the operating system in RAM. The model analyzes exact C source code alongside live memory dumps, steering the search toward the command sequences and edge-case allocations most likely to break it.

Lifelong Context

100M-token working memory for agents and copilots

Standard attention scales quadratically with context length, making million-token windows impractical on conventional hardware. 1 TB enables a tiered memory architecture: recent tokens stay in dense formats for high-fidelity attention, while older context is aggressively compressed into ultra-dense representations and retrieved on demand via sparse routing. The result is a working memory measured in tens of millions of tokens, enough for a legal copilot to hold an entire case history, or a field analyst to maintain weeks of continuous operational context without eviction.

Latent-Space Digital Twins

Predictive control for reactors, process plants, and plasma systems

Controlling small modular reactors, chemical process plants, or plasma confinement experiments requires predicting chaotic thermal and fluid dynamics faster than the physics can evolve. 1 TB holds a continuous 3D voxel grid of temperatures, pressures, or magnetic fields alongside the control model. The ACP can fork the current state, simulate candidate control actions in a latent-space sandbox, and commit the best response, all on-chip, at microsecond cadence, without waiting for a remote HPC cluster.

On-Board SAR Processing

Satellite radar pipeline without ground station compute

Micro-satellites taking synthetic aperture radar images currently beam down massive volumes of raw data for ground-based processing. 1 TB holds the full SAR pipeline locally: time-and-space radar histories, phase-history processing, and change-detection models run on-board. The satellite transmits actionable intelligence rather than raw data, reducing downlink bandwidth by orders of magnitude.

Edge Molecular and Materials Search

Drug candidates, battery electrolytes, catalyst discovery

Discovering new molecules typically requires massive cloud compute to simulate molecular interactions. 1 TB holds a base molecular structure library alongside a generative model that spawns thousands of structural perturbations, scores them locally, and iterates, all without network connectivity. A lab-in-a-box for forward-deployed research teams, pharmaceutical field studies, or materials science at remote facilities.

Federated Swarm Intelligence

Fleet learning with differential privacy

Swarms of autonomous agents (subsea robots, exploration drones, distributed medical devices, must share learned intelligence without leaking private data or requiring broadband connectivity. Each Loom Node node trains locally, diffs its state against a baseline, injects calibrated noise for differential privacy, and compresses the update to a minimal payload. Fleet-wide learning emerges from sparse, privacy-safe exchanges rather than centralized data collection.

The integration moves off the silicon and onto the PCB

The expensive, low-yield part of a large-memory accelerator is not the silicon, it is the packaging. HBM means a TSMC CoWoS allocation, a silicon interposer, and a high-layer-count substrate full of micro-vias, every one of which is a yield risk and a procurement bottleneck before the part ever reaches a board.

This design does none of that. The Loom SoC is a conventional package. The multi-chip routing that would normally live inside an exotic substrate is pulled out onto a standard system PCB, where the switch is an off-the-shelf PCIe Gen5 part from Broadcom or Microchip, the host is a stock ARM64 SoC, and the memory rides published commodity interfaces. The hard integration problem moves from a process only a few fabs can run to a board your team already builds at yield.

The aggregate result is datacenter-class capacity and bandwidth with no HBM allocation queue, no interposer, and no CoWoS slot to wait for. The performance comes from the architecture.

Thermal Packaging

GPU accelerators drawing 700 watts or more require exotic vapor chambers, direct-to-chip liquid cooling, or high-RPM fans. At a projected 36-47 watts per module (SoC plus DRAM), a Loom chip should dissipate heat like a mid-range laptop processor. Extruded aluminum heatsink. Standard low-RPM case airflow. No heat pipes. Silent operation in an office, a telecom edge box, or a field enclosure. Take thermal design off the table as a deployment constraint and cut chassis BOM cost.

PCIe Gen5 P2P Fabric

A 96-lane PCIe Gen5 switch sits at the center of the board. Each Loom chip connects via x16 (~64 GB/s bidirectional). The switch is configured for direct peer-to-peer routing: any chip can DMA directly into any other chip's SRAM or DRAM without routing through the host. The host CPU boots Linux or OpenBSD, loads the models, and gets out of the way.

Cross-Chip Signaling

Loom keeps multi-chip pipelines moving in hardware. The host sets up memory and launches the workload, then the on-chip Command Processors take over. Each chip publishes completion, exposes the next handoff through peer-visible state, and the following chip continues without per-hop polling, interrupt handling, or host babysitting. Low-latency device-to-device flow at hardware speed.

Pipeline and Expert Parallelism

PCIe Gen5 x16 delivers ~64 GB/s. Pipeline parallelism patterns that move little data: each chip holds a contiguous range of model layers, processes them at full local bandwidth, and passes only activations, a few megabytes per token, to the next chip. Weights never cross the bus.

The same fabric is well-suited to expert parallelism for MoE models. With one expert range resident per chip, only the routed tokens cross the switch, and the peer-to-peer fabric runs the dispatch-and-combine collectives directly between chips without host processor intervention. That is what keeps the full-expert-table-resident MoE case in the table above fed.

This aggregate memory footprint is what makes full-precision large models practical. A 405B dense model at FP16 is ~810 GB, impossible on any single device, but it splits cleanly across four 256 GB chips with room for KV cache. No quantization required. Run the model the way it was trained.

Continuous batching keeps the pipeline full. While Chip 3 generates a token for one request, Chip 2 is processing the next token, Chip 1 is processing the one after that, and Chip 0 is ingesting a new prompt from a different user. Four chips at 100% utilization.