As we inch asymptotically closer to the Shannon limit – or upper bounds – on spectral efficiency (bits/Hz), operators are looking for new ways to increase capacities on their existing fibre pairs. Christian Uremovic, Senior Director, Solution Marketing, Infinera, shares how data centre interconnectivity is affected by growing AI demands.
Data centres play a crucial role in our economy for businesses and consumers. With on-going Digital Transformation and the emergence of Artificial Intelligence (AI), including Generative AI, the critical role of data centres will only increase.
There are already over 10,000 data centre locations worldwide, and they consumed 7.4GW of power in 2023 – a 55% increase from the 4.9GW in 2022. The first half of 2023 saw the largest amount of data centre construction in history in primary markets, and Statista reports that global spending on data centre systems hit a record high of US$259 billion.
While helping to drive our increasingly connected world, data centre expansion also presents key challenges that include:
- Scalability: how do we scale and interconnect data centres more efficiently?
- Security: how do we ensure secure operations and data protection?
- Resiliency: how do we avoid failures and implement protection and restoration mechanisms when they do occur?
- Efficiency: how can we improve power efficiency and reduce power consumption while supporting growth in computation, storage and transport demands?
While each of these challenges deserves thoughtful consideration, we’ll focus on enabling scalable data centre connectivity throughout the remainder of this article.
Data centre connectivity
Data centre connectivity can be grouped into the following categories:
- The intra-data centre network, i.e., the fabric cluster, which interconnects servers over a link distance from 100m to 1,000m within the same room or building
- The metro network, which provides connections between data centres, with a link distance typically less than 80km
- The global backbone network, interconnecting data centres through long-distance transport networks, including subsea cables
All these interconnects require fibre connectivity and optical interfaces to ensure low latency, lower transport power per bit, and the highest capacity at the lowest costs.
Inside the data centre,AI applications and workloads are currently driving growth at an unprecedented rate. Optical connectivity speeds are already gradually developing from 400 Gb/s to 800 Gb/s. Current market demand for high-speed 800+ Gb/s intra-data centre interconnect technology is projected to grow nearly tenfold over the next four years – from about 300,000 units in 2023 to more than 2.5 million units by 2027.
To address this growth, the industry will require innovative high-speed, low-latency and even more power-efficient solutions. Digital signal processor (DSP)-based retimed optics, linear-drive pluggable optics (LPO) and co-packaged optics (CPO) solutions for both serial and parallel fibre applications are currently under development and will help significantly reduce latency and power consumption further.
In metro data centre connectivity, we have a more fibre-rich environment, and despite the many fibre splices and some older higher-loss fibres, many link distances are below 100km. Such distances do not provide a technical challenge for coherent optical transceivers and optical transmission systems.
The cost-effective DWDM technology used here enables parallel transmission of wavelengths in the C-band spectrum of fibre, enabling us to transport ~44 Tb/s of data. There are also limited intermediate sites where amplification is required, and scaling the connectivity in this type of metro network is typically done by lighting up additional fibre pairs and deploying amplifiers at the data centre locations.
In the global backbone network, fibre is rare and expensive, and we need high-performance DWDM technology to realise the lowest cost per bit per kilometre and avoid expensive signal regeneration.
Over the past 15 years, the industry has focused on advances in coherent technology to increase the amount of information that can be packed into a given amount of spectrum, called spectral efficiency. We have increased fibre capacity from 9.6 Tb/s with binary transmission to delivering ~44 Tb/s with 64QAM technology in the extended C-band fibre spectrum.
The next generation of coherent transmission technology remains on 64QAM modulation as higher-order modulation schemes reduce transmission reach with no cost per bit per kilometre improvement. As a result, we are beginning to see diminishing improvements in transceiver spectral efficiency as we reach the Shannon limit, which is the theoretical maximum spectral efficiency that can be achieved, as shown below.
The industry needs to find new ways to cost-effectively increase fibre network capacity. One option is to light up additional fibres. However, in these national and international backbone networks, it is very costly to lease fibres. An easier cost-effective way is to increase the amount of usable spectrum on existing fibres. This can be done by expanding from the commonly used C-band into the L-band. L-band requires additional amplifiers and the use of lower-volume, more expensive L-band transceivers.
A further cost-effective option is to expand the available spectrum within the existing C-band. Optical technology has evolved over the years, enabling expansion of the C-band from 3.2 THz to 4.8 THz. With the latest generation of optical transceiver and amplification technology, we can cost-effectively extend the C-band further, from 4.8 THz to 6.1 THz, while maintaining uniform performance across the spectrum. This increased range is called the Super C-band.
With 27% more capacity, network operators can leverage a single set of amplifiers and higher-volume, lower-cost C-band transponders. Additionally, the solution reduces the power consumption per bit of optical line system transmission by as much as 27% and defers the cost of expanding into the L-band or lighting up new fibre pairs. L-band spectrum can also be expanded in a similar manner as Super C, enabling a total capacity of up to 100 Tb/s on a single fibre pair with Super C+L solutions.
More bands, more fibre types
To support capacity growth further, we will see increased use of transmission outside of the C- and L-bands. It will most likely be the S-band initially, and the combination of S+C+L could almost triple fibre capacity compared to C-band only. To get there, we would need commercial S-band amplifiers based on thulium, rather than erbium, or we could use Raman amplification.
There are also emerging innovations in optical fibre, with hollow-core and multi-core fibres demonstrating viability and improvements in lowering latency, increasing bandwidth and transmission capacity. The manufacturability and operational environment for these fibre technologies will continue to evolve before we see large-scale commercial deployments.
Data centres and their interconnectivity remain critical to our economy. Technology innovations like coherent optical engines, expanded transmission spectrum and new fibre types are required to cost-effectively support growth and improve efficiency.
