Extropics TPU??

The Tensor array in TSU doesn't differ from matrices, which comes with the benefit of using each memory address for both layer loading and KV caching. GPU waste a lot of resources because they can only do sequential processing as they have no buffers other than going down the tree in memory tree. At least that is my understanding. A very simple ascii art representation of both shows the difference GPU Processing including KV cache: ``` Compute Units (Active) Wasted Idle Resources +------------------+ +---------------------+ | ████ ████ ██ | | ░░░░░░░░░░░░░░░░░░░ | | ███ ████ ██ | | ░░░░░░░░░░░░░░░░░░░ | | ██ ████ ██ | | ░░░░░░░░░░░░░░░░░░░ | | ██ ████ | | ░░░░░░░░░░░░░░ | +------------------+ +---------------------+ █ = Compute Active (warps/threads executing matrix ops) ░ = Waste (memory reserved but unused) ``` TSU including KV cache: ``` TSU Compute Units (Active) Wasted Idle Resources +------------------------------+------------------------+ | ██████████████████████████ | ░░░░░░░ | | ██████████████████████████ | ░░░░░░░ | | ██████████████████████████ | | | ██████████████████████████ | | +--------------------------------+----------------------+ █ = Compute Active (highly utilized tensor streaming & fused KV cache) ░ = Waste (minimal idle resources) ``` I hope this highlights how, despite powerful parallel compute units, GPUs often suffer from memory over-allocation, fragmentation, and bandwidth bottlenecks that leave many resources underutilized in LLM inference workloads. The streamlined, tiled, and instruction-controlled design of TSUs achieves much higher hardware utilization with less waste compared to the layered and often over-provisioned GPU KV cache systems. Waste in idle resources but still using a lot of power versus highly condensed access writes without loss of resources will deliver substantial energy savings of watts vs token processing (joules).