Loading...
Technical Insight

Magazine Feature
This article was originally featured in the edition:
Issue 5 2026

Why AI infrastructure must compute while data moves

News

AI’s next infrastructure bottleneck is not arithmetic alone but data movement. Optalysys explores how photonic Compute-in-Transit could turn the data path itself into a useful computational resource for future AI systems.

By Flavio Bergamaschi, VP Algorithms & Imon Kundu, Director of Photonics, Optalysys


Artificial intelligence has become one of the strongest drivers of computing infrastructure. GPUs have grown more capable, memory systems faster and interconnects wider as increasingly large models demand more from every part of the system.

But AI scaling is increasingly constrained by data movement and memory limits. Improving the rate at which we perform arithmetic only addresses part of the problem.

AI workloads create and transform large volumes of intermediate data — data that must move between memory and processing elements and, increasingly, across distributed computing resources. The cost of moving data is becoming as much of an architectural concern as the cost of computing on it.

What we need are smarter systems.

Most computing infrastructure follows a familiar pattern: move data to where computation takes place, perform the operation, then move the result onwards. Faster memory, specialised accelerators and higher-bandwidth interconnects have made that process considerably more efficient, but the underlying relationship remains largely unchanged.

Computation and data movement are still treated as distinct, separate activities.

This raises a different architectural question: what if some computation could happen while the data was already moving?


Figure 1: Conventional architectures move data to discrete processing elements; Compute-in-Transit applies selected transformations along the data path itself.

Making the data path do useful work
Compute-in-Transit is an architectural model in which selected transformations are applied to data as it moves through a system, rather than only once it reaches a separate processing element.

It’s not just about moving computation closer to the data, as in near-memory or in-network approaches, but developing new compute capability as the data path itself becomes part of the computation.

Rather than always asking how quickly we can transport data to another processor, we ask: which transformations could take place on the way? For which parts of the workload can movement and useful computation become part of the same process to ease the burden on processors?

Finding the right fit in an AI workload
So where would the Compute-in-Transit approach make sense?

An AI model contains many different types of computation, dependencies, and intermediate data, and they do not all suit the same execution model. Looking at individual operators tells us what computation is taking place, but it does not necessarily tell us enough about how data moves between operations.

Instead, we can examine the workload as a dataflow.

Mapping a workload in this way reveals where data enters a region of computation, which intermediate results are created and consumed inside it, what data leaves, and which operations take place along the way.

That makes it possible to identify areas where compatible transformations could be organised into a continuous stream.

For suitable parts of a workload, serial and vectorised photonic computing could then apply selected transformations as data propagates through the transport path, rather than transporting it to another processor, carrying out the operation and moving it again.

Compute-in-Transit is not intended to force an entire AI workload into a new architecture — it offers another execution model for the portions of a workload whose dataflow makes it useful.


Figure 2: A bounded workload region can be analysed in terms of ingress, external dependencies, internal logical transfers and egress. This exposes where data is created, consumed and moved before any architectural mapping is attempted.

How photonics makes this possible
Photonics offers unusual properties that are well suited to this type of architecture.

An optical signal carries information in both amplitude and phase, and passive photonic structures can transform those properties as the signal propagates.

Interference and diffraction can therefore implement linear mathematical operations within the optical path itself. Rather than requiring a signal to be received, processed electronically and then transmitted again, selected transformations can occur in the same physical path that is already carrying the data.

Electronics remain essential; control, programmability, error management and nonlinear functions still need electronic logic. The opportunity comes from combining the strengths of both domains.

Where a workload exposes compatible vectorised dataflows, blocks of data can be presented as streams to compatible mathematical operations while remaining in transit.

For those parts of the workload, the transport path stops behaving purely as a link between processors. It begins doing useful computational work of its own.


Figure 3: Serial photonic computing combines streamed data movement with programmable transformations in the optical path, while electronic logic retains control and supporting functions.

Putting Compute-in-Transit into hardware
At Optalysys, we have been developing the technologies required to put this architectural idea into hardware.

Our current prototype packages a photonic integrated circuit into a pluggable module connected through a standard QSFP56 port to an AMD Alveo V80 FPGA.

At its heart is our proprietary passive diffractive device, which performs an optical Fourier transform.

High-speed modulators, phase control and balanced photodetectors form the photonic path, while FPGA-based electronic logic manages serialisation, synchronisation, backpressure and error correction. A microcontroller handles calibration and signal-integrity control.

Together, these components create a programmable serial pipeline capable of supporting operations including Fourier and number theoretic transforms, multiplication, multiply-accumulate and modular arithmetic.

The prototype has processed hundreds of gigabytes of data streamed through the interface, with transform-stage results consistent with the native CPU implementation. The current module draws 9.7 W, with its photonic devices operating at 14 GBaud and the surrounding driver and receiver electronics supporting up to 56 GBaud.

These figures are important not because the prototype is intended to replace a GPU, but because they demonstrate that programmable mathematical transformations can be integrated directly into a high-speed data path and exercised on sustained data streams.



Figure 4: Optalysys programmable photonic prototype, showing the pluggable module and its photonic and electronic subsystems.

From prototype to integrated platform
Programmability is an important part of the architecture. The current module uses foundry process-design-kit components, helping portability across silicon-photonics platforms, while the photonic function is exposed through software and our FPGA intellectual property rather than being fixed around a single algorithm. We are developing an implementation that combines the photonic circuit, analogue front end and high-speed digital logic.

Our current designs are based on a 32 GBaud link with a transceiver energy efficiency of 2.68 pJ/bit, excluding the laser. Custom front-end circuits add 0.3–1 pJ/bit to perform operations including multiply-accumulate, Fourier transforms, and floating-point operations. At that link rate, the design performs the equivalent of 384 INT4 GOPs natively. Our next design target is 112 GBaud, increasing that figure to 1.344 INT4 TOPS. These are engineering targets for the integrated platform, not measurements from the current prototype.

Photonics doesn’t need to replace the GPU to change the architecture of AI infrastructure. GPUs remain well suited to dense computation, while CPUs provide the control and flexibility needed elsewhere in the workload.

But as moving data is becoming such a major part of the problem, simply moving it faster cannot be the whole answer.

Faster memory and interconnects improve the movement itself, but the familiar sequence remains: move the data, perform the computation, then move the result again.

Compute-in-Transit offers another option for the parts of a workload that fit it: transformations can take place while data is already travelling between components, putting the photonic hardware in the data path rather than at the end of it.

Where an operation can be performed on the way, the data does not need to be sent somewhere else simply to perform that operation.

We will continue to explore which dataflows are worth mapping, how they should be represented and scheduled, and what interfaces are needed to connect them efficiently to the rest of the machine.

These are questions for computer architecture and software as much as for photonics. AI infrastructure will continue to need faster processors, memory and interconnects, but novel architectures and capabilities will be foundational to sustain its growth trajectory.

It’s time to recognise photonics as more than connective tissue and explore the transformative potential of photonic Compute-in-Transit.


Figure 5: Detail of the photonic circuit and detection path used in the current Optalysys prototype.


Logo
x