Gigabit Ethernet and
Why it is a Very Important Standard

One thing you know as a Networking Guru is that in the next few years you are going to have to provide huge incremental bandwidth to your users. You can see those bandwidth hungry applications coming - things like telephony over IP and video conferencing. You have seen video running over full-duplex 384 Kbps ISDN and it looks terrible. You know that your management will never accept that performance. So you know that you are going to have to use some new technologies. You have looked at switched technologies for the desktop and it looks like that will solve your problems there. Now there remains the question of the backbone.

One of the competing technologies is Gigabit Ethernet. It looks pretty promising. The best thing about it is that your staff is familiar with 10BaseT and 100BaseTX, so you figure that it might be best to go with what you already know and stick with good-old Ethernet. Just for drill, let's run through those aspects of Ethernet you are already familiar with that will help you understand and cope with Gigabit Ethernet LANs.

Cables - First let's consider the cables you are using. If you have an older 10BaseT network you may have some Category 3 or Category 4 cables. They will have to go. Of course, over the last few years you have upgraded all your PCs and they you are now running 100BaseTX everywhere, so maybe you have already solved this problem. If you have Category 5 cables then you will probably be ok. The "probably" is because there are a couple of problems with running 1000BaseT .

First, 1000BaseT requires all four pair of wires in the cable. More than a few installations only terminated 2 pair of wires when installing Ethernet cables. Did you? Second, if the installation was not done exactly to Category 5 standards then the total installation might not work at these higher speeds even though the cable itself is Category 5. If you did terminate all 4 pair, did you certify them all to Category 5 specifications? If not, they might have worked at 10BaseT and maybe even at 100BaseTX, but 1000BaseT is much more aggressive about how it uses the cable. The following is a quote from the Gigabit Ethernet Alliance white paper about 1000BaseT:

  Gigabit Bandwidth over Category 5 Cabling Fast Ethernet (100BASE-TX) achieves 100 Mb/s operation by sending three-level binary encoded symbols across the link at 125 Mbaud. (A 125 Mbaud symbol rate is required because 100BASE-TX uses 4B5B coding.) 100BASE-T uses two pairs (sic): one for transmit, one for receive. The next step up in speed, 1000BASE-T also uses a symbol rate of 125 Mbaud, but it uses all four pairs for the link and a more sophisticated five-level coding scheme. In addition, 1000BASE-T sends and receives simultaneously on each pair. Combining 5-level coding and 4 pairs allows 1000BASE-T to send one byte in parallel at each signal pulse. 125 M symbols/second X 1 Byte (across four pair)/symbol = 1 Gb/s. Of course, it isn't quite this simple. In addition to moving the symbols across the link, 1000BASE-T must also deal with the effects of insertion loss and link-induced interference caused by echo and crosstalk.

To see the entire paper, check the following URL:

http://www.10gea.org/GEA_copper_0999_rev-wp.pdf

Even the media are changed in some cases. For example, there were problems discovered running some fiber cables at gigabit speeds. Special jumper cables were specified to handle those situations. Initially the only copper cable specifications were for twin-axial cable and for shielded-twisted-pair (STP). There is now a Category 5 UTP specification Cat-5e that is required for 1000BaseT cabling. If your network wiring is pre-1999 then it probably will not meet this standard. Of course, we are talking about your backbone, not your entire network, so running a few new cables between servers and switches might not be a killer. Just make sure your installer is certified for this new standard. The point is that 10BaseT or 100BaseT cables and cable installation are not at all the same as with 1000BaseT. You will need to know about cable above category 5.

CSMA/CD - Next let's consider the question of the CSMA/CD protocol. You are doubtless aware that this is the mechanism used in shared Ethernet LANs to control access to the media. This mechanism is very simple, so it saves a lot of money, but it has significant side effects. For one thing, it sets a limit on how long an Ethernet segment (collision domain) can be. For 10BaseT networks that total length is about 2500 meters. 100BaseT runs ten times faster, so the maximum is roughly one-tenth as much, actually about 205 meters. For another thing, CSMA/CD determines both minimum and maximum frame sizes. These are 64 and 1518 bytes, respectively. They are the same for 10 and 100 Mbps. (Interestingly, many sources specify a maximum size of 1516 and some adapters can only run at 1514!)

If the change to Gigabit Ethernet speeds was developed by just running 10 times faster than 100BaseTX, then the maximum network diameter would be 20-25 meters. This was clearly unacceptable. There have been several different proposals about how to solve this problem. The crux of all these solutions solutions is not to use any hubs. There are at least three ways to do this.

The first proposal is to use switches instead of hubs and make the connections directly from the hosts to the switch ports. (Whether this switch is operating at layer two or three makes no difference here.) These ports would then not need to use the CSMA/CD protocol and could run in full-duplex mode. At gigabit speeds, however, data can come at the switch pretty fast, so there has to be lots of RAM for buffers, making such a switch somewhat expensive.

The second alternative is to change the rules about frame sizes. The Gigabit Ethernet standards group has done just that. The minimum transmission block is now 512 bytes instead of sixty-four. Hosts don't have to be aware of that - the NIC or the driver can pad the sixty-four byte frame as needed using a technique called "carrier extension". In addition, however, the committee couldn't stand the thought of all that bandwidth going unused so they have decided to allow multiple packets to be sent in a single transmission using a new technique called "packet bursting". This is a sound idea, but hardly good old 10/100 Ethernet formatting?

The third alternative is to build a new class of device that operates like a switch in that it can read from all ports at the same time but forwards like a hub in that it sends all packets out every port (except the one it came in on) and thus does not have to make any forwarding decisions. Since it can read from all ports at once, it no longer defines a "collision domain". These devices are called "buffered repeaters" or "buffered distributors". But because they are store and forward devices they introduce latency which you will now have to factor into all your network calculations. This is not your father's 10BaseT hub.

Actually, all of these techniques are being tried. In fact, if there is a true 1000BaseT hub on the market or (even plans to develop one) it is certainly well hidden. If one is going to go to the expense of putting in gigabit ports, why would one want to share the bandwidth? The specification allows for such a thing, but it is not clear that there is a market for it. The specifications do say that there can only be a single repeater in a collision domain. This is not new to you if you are already running 100BaseTX with hubs, but it is certainly different from the 5-4-3 rule of old 10BaseT. The bottom line -  forget everything you new about hubs if you are going to Gigabit Ethernet.

Modulation - Most people are probably not aware of the modulation techniques used for 100BaseTX, but then they probably weren't aware of what they were for 10BaseT either. The only reason it is important is that it is a change in the technology so it meant that early products might have interoperability problems and that vendors had to work through the new technology. In the case of 100BaseTX the technique used was not actually a new one. The 4B5B coding was used by FDDI and by 25.6 Mbps ATM, so it was an already well known technology with off-the-shelf parts that could do the job. The 5bit code used by 1000BaseT was completely new. There were likely some surprises from time to time, but they have probably been worked out by now. In any case, the claim that 1000BaseT is just Ethernet is clearly not true in this case either.

Frame Size - The sixty-four byte minimum was not the only frame size parameter that changed. Since many (perhaps all) of the connections in gigabit Ethernet will be full duplex, collision detection is not needed and the limits CSMA/CD placed on frame size are irrelevant. The overhead caused by sending small frames (of about 1500 bytes) was considered by the consortium to be excessive, so, for full-duplex connections at least, the maximum frame size has been increased. There is a thing called a "Jumbo" frame. It is not clear whether there is a maximum frame size anymore. One observed device has a maximum frame size of 9000 bytes. It is not yet clear if this is a de facto standard.

Frame Format - There is one aspect of Gigabit Ethernet that is exactly the same as 10BaseT and 100BaseTX, the MAC layer frame formats. However, the entire purpose of the OSI and TCP/IP models and the protocol stacks in the operating systems is to manage these frame formats and insulate all higher layers, specifically including the applications, from these fields. So why would you care whether the frame format was the same or different? When you install LAN hardware in the servers and workstations you load the drivers that came with the hardware. These drivers take care of the frame formats. This would be true whether you were running Gigabit Ethernet, Gigabit Token Ring, Gigabit ATM, or Gigabit Fred's Friendly Topology. This level of abstraction even extends to network analyzers. If you install a new LAN topology you are going to have to get new analyzer hardware. It will come with its drivers and will incorporate knowledge of the relevant frame formats. OK, the vendors will have to write drivers and decoders - but they have to write new drivers to cope with all the new Ethernet formats as well.

MAC Addresses - O.K., MAC addresses are the same for 10BaseT, 100BaseTX and 1000BaseT. That's two things that didn't change. And the Multicast bit is still the least significant bit of the high order byte. Wonderful! MAC addresses are also something you are not supposed to see at the application level.

Do I care? -  At this point you ought to ask yourself, "Why do I care what the technology is?" At these speeds the connections will mostly be point-to-point or switched connections. If it's point-to-point, it could be Gigabit Gnertz as long as it works, and you wouldn't care. The moral of the story: don't just buy Gigabit Ethernet because it says "Ethernet" and that makes you feel comfortable. It ought to make you uncomfortable that the Ethernet community has changed almost everything about the technology and is still trying to pretend that it's the same old Ethernet. You ought to find out what competing technologies have to offer and make a decision on a more rational basis.

Why would I want something else? - There is one area where Gigabit Ethernet lacks a feature that other technologies can supply. The applications that will require more bandwidth in the future also require varying "quality of service" (QOS) levels. You don't want your server backup to be able to ruin your video presentation. There are basically two ways to provide QOS - overbuild the network so there will always be enough bandwidth for whatever you want to do, or use a technology like ATM that has QOS levels designed into the hardware and can guarantee them. If you have an excess of money to throw at the problem then the first alternative might be easier. If not, then you might want to consider the second.