Dec 23, 2011

How can data transmission speed in data centers be increased?

A constant bottleneck in data centers is experienced when data is moved between machines. During busy moments, the level of congestion can cause significant issues and slow performance. Short of redesigning the entire architecture, which is much more than I am willing or able to do, what can be done to minimize this bottleneck in a data center?


It may be just around the corner - wireless transmission seems a possible way to go once some still significant problems are ironed out.  IBM was working on some sort of wireless transmission using optical signals as far back as the 1960s, and 50 years later we are still looking at it.  It is going to require moving huge amounts of data across wireless spectrum, with the same requirements for reliability that are expected from current fiber optic cable.  Personally, I think it would make more sense to alter the accepted data center hierarchy, so that there aren't tons of hubs feeding into tons of routers in a pyramid fashion, which is just about perfect for creating the opportunity for bottlenecks.  But that's like saying I think cars should have six wheels - maybe that's a great idea, but it wouldn't be easy to change the paradigm.  

Hi pwarren,

You might want to check out this article that covers memory virtualization and how it increases efficiencies in network resources.

Memory: The Real Data Center Bottleneck

"CIOs and IT managers agree that memory is emerging as a critical resource constraint in the data center for both economic and operational reasons. Regardless of density, memory is not a shareable resource across the data center. In fact, new servers are often purchased to increase memory capacity, rather than to add compute power. While storage capacity and CPU performance have advanced geometrically over time, memory density and storage performance have not kept pace. Data center architects refresh servers every few years, over-provision memory and storage, and are forced to bear the costs of the associated space, power and management overhead. The result of this inefficiency has been high data center costs with marginal performance improvement."
Answer this