Untether AI
At-memory AI inference chip company
Untether AI is a semiconductor company that designs at-memory computing chips built specifically for running trained neural networks in data centers and edge devices. Instead of shuttling data back and forth between separate memory and…
Definition
Untether AI is a semiconductor company that designs at-memory computing chips built specifically for running trained neural networks in data centers and edge devices. Instead of shuttling data back and forth between separate memory and compute units the way conventional processors do, its chips place small processing elements directly beside each memory cell, so weights are read and multiplied in place rather than moved across a chip. The approach targets one of the main bottlenecks in AI inference: the energy and time cost of moving model weights, which on standard architectures often dwarfs the cost of the arithmetic itself.
Overview
Untether AI was founded to address a structural inefficiency in how conventional processors run neural networks: in a typical GPU or CPU, moving weight values from memory to a distant arithmetic unit consumes far more energy than the multiply-accumulate operation performed once the data arrives. The company's response is an architecture it calls at-memory computing, in which thousands of small processing elements are distributed directly among memory banks across the chip, so each element can read and compute on the weights stored right next to it. This removes the long, energy-hungry data path that dominates power consumption in traditional designs built around a central compute core and separate memory hierarchy. Mechanically, Untether's chips arrange memory and compute in a tightly interleaved grid rather than the usual separation of a processor die from external or on-die memory blocks. Each processing element handles a slice of a neural network's weights, performing its multiply-accumulate work locally and passing partial results along the grid rather than routing full weight tensors across a bus. Because most of the data movement happens over very short distances between adjacent tiles, the chip can achieve high computational throughput per watt on inference workloads, which are dominated by repeated weight-multiplication operations rather than the more varied instruction mix seen in general-purpose computing. This places Untether AI among a wave of inference-specialized chip designers, alongside companies pursuing digital in-memory compute, transformer-specific ASICs, and neuromorphic designs, all responding to the same observation: general-purpose GPUs, while flexible and dominant for training, are not the most energy-efficient way to run an already-trained model repeatedly. Untether differs from transformer-specific ASIC vendors by keeping its architecture broadly applicable across convolutional and other network types rather than betting entirely on the attention mechanism, and it differs from neuromorphic designs by staying within a conventional digital, weight-stationary computing model rather than adopting spiking or event-driven neuron models. In practice, Untether AI's chips are aimed at inference deployments where power and cost per inference matter more than raw flexibility, such as data center appliances handling high volumes of vision or recommendation workloads, and edge or embedded systems where a power budget rules out a full GPU. Customers integrate the chips through a software toolchain that compiles trained models, typically exported from mainstream frameworks, into a form the at-memory grid can execute, since these chips generally cannot run the training process itself. The trade-off inherent in the design is specialization: at-memory inference chips are not built to train models, and because weights sit fixed in memory tiles, adapting to a wildly different network architecture or frequently updated model can require re-provisioning the chip rather than a simple software update. Organizations that need one platform for both training and inference, or that iterate on model architecture very rapidly, typically still rely on GPUs; Untether's chips fit best once a model has stabilized and is being deployed at scale where the inference-time energy savings compound across millions of requests.
Key Features
- At-memory architecture places compute elements directly beside stored weights
- Distributed grid of thousands of small processing tiles across the chip
- Targets inference workloads rather than model training
- High computational throughput per watt on repeated matrix multiplication
- Software toolchain compiles models from mainstream frameworks onto the chip
- Applicable across convolutional and other non-transformer network types
- Designed for both data center appliances and power-constrained edge systems