In modern AI systems, performance is no longer governed by processor speeds, but is fundamentally constrained by the energy and latency of moving data across memory, accelerators, interconnects, and network nodes. 

Every transfer consumes bandwidth, drains power, and often requires data to be reshaped, converted, or aggregated before useful computation can even begin. As AI clusters scale, this data movement bottleneck is forcing a complete re-evaluation of data center architecture. 

The Data Movement Crisis in Modern AI 

While modern GPUs and XPUs possess immense raw compute capability, they can only deliver value if data arrives at the right place, in the right format, at the right time. 

Hitting the memory wall 

The memory wall has created a widening gap between what processors can compute and what memory subsystems can supply. Large Language Model (LLM) inference is notoriously memory-bound. During the decode phase, tokens are generated sequentially. Generation of each new token requires repeated access to very large arrays of weight data and the retrieval of Key-Value (KV) cache elements from remote memory. As context lengths expand, the data footprint grows rapidly, crippling system throughput and energy efficiency. 

The cost of idle silicon – GPU & XPU starvation 

Whilst energy is consumed moving and staging data, expensive GPUs sit idle. When memory bandwidth, interconnect latency, or data preparation cannot keep pace, high-performance compute engines stall waiting for input. Some estimates place the numbers as high as 95% of enterprise GPU capacity sitting unused. 

Commercially, this represents an extraordinary waste of capital: an underfed GPU consumes rack space, power, cooling, and capital expenditure while producing zero useful work. This translates directly to lower inference throughput, higher latency, and an inflated cost per token. Buying faster XPUs alone will not solve the problem – operators must optimize the memory and transport paths around them. 

The energy density challenge 

Data movement is a massive energy sink. Massive amounts of power are consumed inefficiently shuffling data across memory tiers, converting domain signals, synchronizing distributed nodes, and cooling dense server racks. 

According to the International Energy Agency (IEA), global data center electricity consumption is projected to double from ~485 TWh in 2025 to around 950 TWh by 2030, with AI data centers driving the steepest trajectory. For AI operators and hyperscalers, efficiency is no longer just a sustainability metric, but a physical capacity constraint. 

The Optalysys solution: Photonic Compute-in-Transit 

Optical interconnects are already moving deep into AI infrastructure to deliver the raw bandwidth required by scaling clusters. 

Rather than treating optical transport as a passive pipe, Optalysys is transforming the optical data path into an active computational layer. 

Hybrid photonic-electronic architectures 

Rather than building an all-photonic system, we’re developing a hybrid photonic-electronic architecture that can integrate into existing systems without a stack overhaul. Structured, deterministic mathematical operations are executed directly within the optical data path as signals propagate, while digital electronics handle control, accumulation, precision calibration, and protocol management. 

The goal is to identify mathematical operations that naturally align with streaming data and execute them where that data is already in motion. 

Engineered around a 32 GSps serial data path, the Optalysys architecture features receiver-side photonic compute capable of executing programmable operation classes, including mathematical transforms, matrix-vector multiplications, and multiply-accumulate functions. 

Rethinking the dataflow pipeline 

By embedding computation into transmission, the fundamental dataflow of AI workloads changes: 

• Conventional flow: Memory → Transport → XPU Operations 

• Compute-in-Transit flow: Memory → Transport + Operation → XPU Operations 

Data arrives at the destination processor already prepared, reduced, or transformed. 

Workload alignment: Where Compute-in-Transit excels 

Compute-in-Transit is targeted at specific, high-impact pipeline stages. A prime candidate workload meets three core criteria: 

1. The data must move anyway across memory tiers or network interconnects 

2. The operation is structured and streamable (e.g. linear transforms, reductions) 

3. Offloading the operation eliminates meaningful network traffic or endpoint processor burden 

Use case explorations 

• Long-context LLM inference & KV cache – filtering, selecting, or converting KV cache data as it streams between disaggregated memory tiers and XPUs 

• Distributed AI & collective communication – performing online gradient accumulation or tensor reduction directly in transit during XPU-to-XPU communication 

• Heterogeneous pipeline conversion – reshaping tensor layouts or converting data between precision formats (e.g., FP8, FP16, INT8) in flight 

• Streaming linear operations – executing progressive mathematical operations on massive datasets en route to the accelerator 

System-level infrastructure benefits 

Integrating photonic Compute-in-Transit into AI clusters could yield distinct advantages for data center operators: 

• Maximized XPU utilization – offloading preliminary preparation or reduction leaves 100% of GPU silicon focused on dense matrix math 

• Reduced data movement overhead – eliminates intermediate buffer stages and downstream compute passes after transport 

• Enabled disaggregated & tiered memory – pooled memory architectures become vastly more practical when data can be filtered or pre-processed during transfer 

• Minimal power footprint – our architecture targets an estimated 2.76 pJ/bit link energy on 45nm silicon photonics (excluding laser), adding minimal energy overhead for receiver-side computation 

The future of AI compute 

The future of AI infrastructure is heterogeneous: the data centers of the future will combine classical silicon, neuromorphic chips, quantum processors with optical interconnects and processors while an intelligent orchestration layer directs each workload to the compute best suited to it. 

Photonics has already revolutionized how AI infrastructure moves information – Compute-in-Transit opens up the next frontier: if data must travel, light should compute on it while it moves. 

---

At Optalysys we’re pioneering the architectural revolution enabling photonic Compute-in-Transit. Get in touch with us to find out how we can bring efficiency gains to your use case →