FDDI

Introduction

Fibre Distributed Data Interface (FDDI) is a ANSI LAN standard which is made from two counter-rotating rings running at 100Mb/s half-duplex. An unbroken FDDI network can run to 100km with nodes being up to 2km apart on multi-mode fibre (62.5/125um), and 10km apart on single-mode fibre. There can be up to 500 nodes on any one ring. If the Primary ring fails and the FDDI wraps, then the network design must be such that the total ring length must not exceed 200km and the number of nodes must not exceed 1000 (see later). Because the secondary ring is there purely for backup, the design of FDDI should be such that 500 dual-attached nodes and 100km maximums are adhered to, no data travels on the secondary ring unless there is a failure in the primary ring.

The maximum packet size is 4.5 Kb (compared with Ethernet's maximum size of 1.5 Kb) which means that FDDI packets must be broken up to go on to Ethernet. The minimum size is 17 Kb.

Data normally travels on the primary ring (anti-clockwise) unless a fault occurs at which point the working stations wrap onto the secondary ring (clockwise) to maintain the network. The upstream neighbour of a particular station is the one that is sending data to this station on the primary ring. The downstream neighbour is the one receiving data from this station on the primary ring.

Copper-stranded Distributed Data Interface (CDDI) is the UTP equivalent that requires stations to be no more than 100m apart. There is also a shielded copper equivalent called Shielded Distributed Data Interface (SDDI).

FDDI frames

Following is a description of the fields:

Preamble - 16 I symbols used to synchronise the individual clocks on the ring since there is no master clock in FDDI.
Starting Delimiter - JK symbol pair which are called Violation Flags (signal phase violation detection) and signal the approach of a frame.
Frame Control field identifies the type of frame using two symbols in the range 41 to 4f. The frame could be one of:
- Token
- SMT
- Beacon
- Claim
- Data (LLC)
Destination address could be a station or group: I/G individual (0), group/multicast (1), U/L universal (0)or locally administered (1). Could be 2 or 6 octets, although 2 is never used.
Source address RII (Routing Information Indicator); 0 = no, 1 = yes. U/L universal (0) or locally administered (1). Could be 2 or 6 octets, although 2 is never used.
Frame Check Sequence contains the 32 bit Cyclic Redundancy Check (CRC).
Ending Delimiter - T symbol (violation flag)
Frame Status has the Address Recognised (A) and Frame Copied (C) and Error (E) indicators which are either Set (S symbol) or Reset (R symbol).

PCM defines the symbol usage (see later on). The Preamble, Starting Delimiter and ending delimiter are the same for both the token and the frame itself.

If a station detects an error in the FCS then it sets the Error bit and as the frame circulates the ring, only that station increments the error counter.

The hardware address, or MAC address is transmitted and stored in Ethernet network devices in Canonical format i.e. Least significant Bit (LSB) first. You may hear the expression Little-Endian to describe the LSB format in which Ethernet is transmitted. Token Ring and FDDI, on the other hand, transmit the MAC address with the Most Significant Bit (MSB) first, or Big-Endian, This is known as Non-Canonical format. Note that this applies on a byte by byte basis i.e. the bytes are transmitted in the same order it is just the bits in each of those bytes that are reversed! The storage of the MAC addresses in Token Ring and FDDI devices however, may sometimes still be in Canonical format so this can sometimes cause confusion. The reference to, the distribution of MAC addresses and the OUI desinations are always carried out in Canonical format. See the document on Token Ring to see an example of converting from canonical to non-canonical format.

FDDI standards

The FDDI model has three sublayers that sit under the 802.2 Logical Link Control (LLC). These are in order of top down:

FDDI MAC
FDDI Physical Layer (PHY)
Either Twisted-pair PMD or Fibre PMD

Along side these layers sits the Station Management (SMT).

In detail:

Physical Medium Dependent (PMD) sublayer is the hardware that provides the link between stations. The standard is ISO 9314-3 PMD. It includes the standards for the MICs, optical levels, bit error rates, signal reception and so on.
Physical Layer Protocol (PHY) - defines the encoding and decoding symbols which are the smallest pieces of information between the MAC and the PMD. The symbol is a chunk of 5 bits and when data is being transmitted one symbol is used for each nibble (4 bits). The PHY also sets up the medium to accept data transfer. The standard is ISO 9314-1 PHY.
Media Access Control (MAC) - manages data transfers, error recovery, network addressing, token passing and station access. The standard is ISO 9314-2 MAC.
Station Management (SMT) - SMT operates across all of the PMD, PHY and MAC sublayers. SMT communicates between stations to maintain and monitor the ring by collecting statistics, detecting, isolating and solving faults. Information on each station is stored in the Management Information Base (MIB), each station is a MIB object. Typical information within the MIB object include:
- Unique station identifier
- SMT version
- Station configuration information
- Available paths
- The number of MACs in the station
- Current port configuration
- Status report frames that are queued
Logical Link Control (LLC) - ensures correct delivery of data to the upper layers via the generation and checking of addresses.

SMT

Connection Management (CMT) manages the physical connection between two ports of neighbouring stations as they transit between OFF, ON, ACTIVE, CONNECT and so on.
Configuration Management (CFM) configures PHY and MAC parts in a station and maintains information on the state of the MAC connection.
SMT Entity Coordination Management (ECM) controls the trace and the optical bypass relays. This oversees the operation of CFM and PCM.
Ring Management (RMT) detects duplicate addresses, stuck beacons and tells the LLC that the MAC layer is ready.
Neighbour Information Frames (NIF) give some information about the station and its upstream neighbour from the Management Information Base (MIB). NIFs are sent every 2 to 30 seconds.
Status Information Frames (SIF) provide some configuration and operation information from the MIB.
Parameter Management Frames (PMF) can read and write to the MIB.
Status Report Frames (SRF) announce status information to management programs and are sent via the SRF multicast address.

Physical Attachment to the Ring

This adheres to the standard X3T9.5 FDDI specification.

Dual Attached Stations (DAS) such as servers or routers attach to both primary and secondary rings and are involved in ring wrap when ring faults occur.
Single Attached Stations (SAS) attach to ports of a concentrator and only on the primary ring.
Dual Attached Concentrators (DAC) provide a reliable connection for stations to the backbone.
Single Attached Concentrators (SAC) are less reliable due to the single connection to the backbone.
Null Attached Concentrators (NAC) are not connected to any backbone.

The diagram below shows various ways of attaching to the FDDI ring.

There is a device known as an Optical Bypass Switch (OBS) which is a purely passive optical relay, dual attached to the FDDI ring which wraps when the device that it is attached to, fails. One thing that needs to be taken into account when using an OBS is the constant optical loss of between 1.5 and 2.5dB that exists when using it. Optical loss between nodes must not exceed 11dB. This is made up of no more than 6dB loss from connectors and 2.5dB per km loss from attenuation.

A station can be connected using Dual Homing which is where a station has two ports A and B which connect either to one concentrator or to two concentrators. The A port takes over if the B port fails.

The following lists the various ports available:

Port A - Primary in/Secondary out, coloured Red.
Port B - Primary out/Secondary in, coloured Blue.
Master port (M) - connects SAS to the primary ring, coloured Green
Slave port (S) - station port for an SAS connecting to a concentrator, no colour.

The keyed Media Interface Coupler (MIC) helps prevent wrong port connections to be made. The keys are colour coded and are attached to cable covers so that you can change the connector to any type by swapping the tiny coloured keys. No key makes the connector a Slave. The following diagram shows the connection keys as you look at the connector end on as if it is about to be plugged into your face.

A and B (Trunk) Ports

Dual-attached concentrators (stations) have a pair of A and B ports that can connect as follows:

The A port can connect to the B port on the neighboring concentrator (station) to create a Primary In (PI) - Secondary Out (SO) attachment.

The B port can connect to the A port on the neighboring concentrator (station) to create a Secondary In (SI) - Primary Out (PO) attachment.

The A or B port can connect to the M port of a higher-level concentrator (station) in a tree topology.

M (Master) Ports

Concentrators have M ports that can be connected as follows:

The M port can connect to the S port on a SAC or a SAS.

The M port can connect to the A and B ports of a dual-homed DAC or DAS.

S (Slave) Ports

The S port of a SAS connects to the M port on the concentrator.

The chart shows the recommended configuration of FDDI ports.

Physical Connection Manager (PCM)

PCM sets up a connection between ports by setting up a mini ring between the ports. The symbols Quiet (Q), Halt (H) and Idle (I) are used to create line states according to the following table:

State	Meaning	Symbols
QLS	Quiet Line State	Q - after 16 or 17 consecutive Q symbols
LSU	Line State Unknown	n/a - no line state at all
NLS	Noise Line State	various - after 16 potential noise events without entering any line state
ALS	Active Line State	JK - after 1 JK pair
MLS	Master Line State	HQ - after 8 or 9 consecutive HQ pairs
HLS	Halt Line State	H - after 16 or 17 consecutive H symbols
ILS	Idle Line State	I - after 4 or 5 consecutive I symbols

As mentioned before, each symbol is 5 bits, but represents 4 bit patterns. The extra bit guarantees an adequate number of signal transitions for clock regeneration to work, and it also acts as a parity check on the symbol. The following table shows some symbols and the way the bit patterns are mapped:

Name	Symbol	Code	Meaning
Line State	Q	00000	Quiet
	I	11111	Idle
	H	00100	Halt
Starting Delimiter	J	11000	First of SD pair
	K	10001	Second of SD pair
Ending Delimiter	T	01101	Terminate data stream
Control Indicators	R	00111	Reset (0)
	S	11001	Set (1)
Data symbols	0	11110	0000 (0x0)
	1	01001	0001 (0x1)
	2	10100	0010 (0x2)
etc.

FDDI Operation

FDDI is different from Token Ring in a number of ways. One difference is that FDDI times its tokens. Another difference is that each station can provide management in a distributed sense rather than have just one management station.

A token travels around the ring and is captured by a particular station that wishes to send data. Frames are sent by the station until all data has been sent or until the Token Holding Timer (THT) expires. Each station on the ring checks each frame for errors and retransmits it. The destination station copies the frame it receives and sets the error indicator, the frame copied indicator and the address recognised indicator. The destination station sends the frame on until the sending station receives it back. At this point, the originating station takes the frame from the ring once it recognises its own address in the source address field. This process of removing its own frame is called Stripping.

As soon as a frame is begun to be received by a station it immediately begins retransmitting it before it has finished receiving it. Because of this, when a station receives back its frame, part of this frame (12 bytes) has already been retransmitted before the station has recognised itself as the original source address. When this happens, a fragment ends up on the ring. FDDI stations have a one byte buffer between reception and transmission so what happens is each station receives the fragment, detects that there is no End Delimiter and forwards the fragment on minus the end byte of the fragment still in the buffer. A 12 byte fragment for instance would be whittled down to 11 bytes by the time it reaches the next downstream neighbour, then 10 bytes etc. until the fragment is cleaned off the ring. This process is called Scrubbing and is continuously operating because it is part of the normal process to see fragments on the FDDI ring.

If a station fails to start or complete sending data frames after grabbing the token, then it can send a frame called a Null Void frame that has a destination MAC of all zeros and a source MAC of itself. It then strips all frames off the ring until it sees its own Null Void frame and resets the transmission timers. A proliferation of Null Void frames indicates that there are station transmission problems.

Timers

The Target Token Rotational Timer (TTRT) is set just at the ring initialisation and is effectively a ring latency parameter that determines the delay on the ring. Each station requests a time for how long it wants the token to take going around the ring. The TTRT is the fastest agreed time it takes for a token to rotate all the way around the ring (see below for ring initialisation). The maximum value that TTRT can have is 165ms.

The Token Rotation Timer (TRT) is the timer which counts down between tokens arriving at a particular station i.e. it times how long it actually takes for the token to go around the ring, therefore the token must be lost if this timer runs out. Once the Claim process (ring initialisation) determines the TTRT, then this is what the TRT is set to initially. Each time the token rotates around the ring its elapsed time is compared with the agreed TTRT. If it is less i.e. healthy, then the TRT is reset to the TTRT. If it is late i.e. there may be a problem, then a Late Counter is incremented and the TRT reset to the TTRT. If it is late again i.e. If TRT is greater than 2 x TTRT then the station tries to reinitialise the ring.

The Token Holding Timer (THT) is the time that a station can transmit frames before releasing the token (This is the Timed-Token Protocol). Whatever time is left in the TRT within the token that arrives back at the station, this value is placed in the THT i.e. THT is set to TTRT - TRT. At the same time, the TRT has its value updated with the TTRT even though the token remains with the station. Whilst the station holds on to the token, it transmits as many frames as it can whilst the THT times out and the TRT is counting down. This is one reason why FDDI is faster than Token Ring because, depending on THT, the station can transmit many frames before it releases the Token, unlike Token Ring.

The Valid Transmission Timer (TVX) times the period between correct frame transmissions, therefore is a check for faults on the ring. If it expires then a new claim process begins. Its value is 2.5ms which is the time it takes for a 4,500 byte frame to travel around a ring that is 200km. This is better than relying on a token failing to go around the ring and waiting for 2 x TTRT which could be 2 x 165ms = 330ms in the worst case.

Other timers are:

Starting Delimiter Time - the maximum delay for the starting delimiter to rotate the ring
Acquisition Time - maximum signal acquisition time
Frame Transmission Time - maximum time required to transmit a 4,500 byte frame
MAC Frame - maximum number of MAC frames allowed on the network
Claim Frame Time - time needed to transmit a Claim frame with its preamble
Setup Length Time - time needed for set up to transmit once a token has been received
Insertion Time - the maximum time needed by a station to be inserted on to the ring

Ring Initialisation

The following steps occur when connecting to the ring:

The PCM shares port type, link quality and connection rule information. This occurs on each point to point physical connection on the ring. Even though FDDI is a ring topology, physically all devices are connected on a point-to-point basis.
The two ports negotiate the duration of the link confidence test.
The test is run.
Results are exchanged.
If successful the station joins the ring, otherwise the steps start again.
Once the new station joins the ring, the ring is initialised by starting the claim process (a fault such as a lost token also starts the claim process!). This is where stations compete to be the one that puts the token on the ring.
Each station continuously submits a time for the TTRT in claim frames called T-Req frames.
Each station compares the TTRT bids from other stations with its own.
If the other stations TTRTs are longer, then it continues to send its own.
If another station's TTRT is shorter than its own then it retransmits the faster TTRT and stops sending its own TTRT.
If the TTRTs happen to be the the same then the station with the highest address wins.
The station which receives its own bid back is the station with the lowest TTRT and so this sets the TTRT for all stations to use and produces the token (the lowest TTRT ensures the fastest possible ring speed).
The Neighbor Notification Protocol (NNP) is used every 30 seconds on the ring whereby a station sends a NIF containing the Upstream Neighbor Address (UNA) and the Downstream Neighbor Address (DNA). The first NIF is called a Next Station Addressing (NSA) request where the UNA and DNA are set to unknown. Stations use NNP to find neighbour addresses and to check for duplicate MAC addresses by examining the A bit to see if it has been set (i.e. a station has recognised its own address).

FDDI can operate two main types of transmission Synchronous and Asynchronous. Synchronous transmission means that bandwidth is reserved for the station that holds the token. This is used for multimedia type applications. The more commonly used Asynchronous transmission does not reserve bandwidth and these frames always have to wait until all Synchronous FDDI frames have been transmitted. There are two forms of Asynchronous transmission:

Non-restricted - has 8 priority levels and operates dynamic bandwidth allocation
Restricted - stations can set the token to restrict the ring for their use only and is used for extended transmissions.

Ring Faults

All stations monitor the ring. If the ring is inactive for too long, or a break exists or there are incorrect frames on the ring then the ring is initialised. If a port breaks then a QLS is detected by the PHY and the port is wrapped without the other stations having to know about it. If the TVX expires then the claim process starts and if that fails then the beacon process starts.

The beacon process is used to detect where the logical fault is on the ring, i.e. a station is connected but not operating correctly. If a station does not receive tokens or data from its upstream neighbour then it sends a broadcast beacon frame which each station in the ring repeats. If this station receives its own beacon then the ring has been fixed and it stops beaconing, otherwise it continues to beacon. After 10 seconds the station declares that there is a stuck beacon and sends directed beacon frames to a management station which then starts a trace.

A trace is sent to the upstream neighbour on the secondary ring. The upstream neighbour acknowledges this and both stations leave the ring and carry out a path test between them both. If the test passes then the stations rejoin, but if one station fails it remains off the ring whilst the other station ring wraps.

FDDI Network Design

There is a limit of 1000 MACs on a network, so there can be 1000 SASs or 500 DASs (since on a ring wrap a DAS will handle all packets twice).

The more stations sit on the dual ring, the more likely it is for ports to wrap. When the primary ring is wrapped on to the secondary ring the ring increases in length thereby increasing the loss on the fibre.

On a heavily used network which includes a number of routers, these routers could end up rebooting due to the load. If a number of routers reboot, then the DAS interfaces also reboot causing the ring to segment into a number of pieces for a period of time as the routers rebuild their tables. Two or more devices wrapping will cause ring segmentation. Concentrators are less likely to go off line than stations.

A topology called the Dual Ring of Trees can be implemented where only the concentrators are connected to the dual ring and critical systems are dual homed. The most critical stations could be dual homed to different concentrators.

Running periodic baseline tests aid in giving a picture of normal errors expected in the network. The following counts should be looked at in particular:

Ring Ops count notes the number of times the ring changed to an operational state.
Link Error Monitor (LEM) count notes the number of invalid line state transitions. Less than 50 a day is ok, 50 - 500 is marginal and over 500 indicates a problem with cable or hardware.
Link Error Rate est (LER est) indicates the rate of errors. 15 means 10^-15. Values 12 - 15 are considered good, 8 - 11 is considered marginal and the port cuts off below 8. i.e. if there are more than 10^-8 errors per second, the port is disabled.
Loss count notes errors that occur in the middle of a frame.
Error count notes FCS errors, normally from communicating with the upstream MAC.
Active and Break counts notes the number of times the PCM has gone into active and break states. It should be the same for each, more breaks than actives means the link retries a number of times before success.

FDDI VLANs

The IEEE 802.10 Interoperable LAN/MAN Security (SLS) standard was designed for MANs and therefore accordingly has provision for security built in. The Secure Data Exchange (SDE) PDU is a MAC layer frame with a 802.10 header inserted between the MAC header and the data field. The following diagram describes the 802.10 frame:

The Protected Header has a copy of the Source Address for validation purposes and the Integrity Check Value (ICV) uses a security algorithm to prevent internal data modification. The Security Management Information Base (SMIB) provides the Security Association Identifier (SAID), which is used as the VLAN ID, and the optional encryption key. The 4 byte SAID allows for 2³² VLANs. The Management Defined Field (MDF) is also optional and carries information to help with PDU processing.

If no security is required then just the Clear Header and the SDE designator are required (7 bytes) giving low processing overhead.

FDDI to Ethernet Operation in Switches

Translational bridging between Ethernet and FDDI can occur by re-ordering the bits of the layer MAC addresses from canonical format to non-canonical format. Whereas with Token Ring only non-routable protocols can make use of translational bridging, with FDDI some layer 3 protocols can be translationally bridged. This is because within the FDDI specification, any MAC addresses that appear in the upper layers MUST be ordered canonically. Protocols that can be bridged between Ethernet and FDDI are IP, OSI and DECnet.

The FDDI Specification states that a station (including a bridge/switch) must wait until it receives a token before it can forward frames coming from an Ethernet station on to the ring. Also, the station is responsible for Stripping its own frame off the ring. In the case of the switch, once it sends a data frame, the switch must follow it with two void frames whilst it still maintains hold of the token. These void frames are called My void frames and are necessary because the switch only forwards frames containing the MAC addresses of the sending and receiving stations rather than its own MAC. The consequence of this is that the switch has no way of recognising any frames that it puts on to the ring, and it needs to strip these off according to the specification. The My void frame is a simple frame with both the destination and source MAC addresses set to that of the switch. Other FDDI nodes refrain from transmitting whilst the switch has the token and the switch uses this time to strip spurious frames off the ring until it sees the void frames return, which it strips off as well.

A potential problem with switches is that a MAC address of an Ethernet station that was previously learned on the Ethernet side of the switch, once a frame is forwarded on to the FDDI ring, this same MAC address could be learned on the FDDI side of the switch and placed in the FDDI forwarding table as if that MAC address came from the FDDI.

If an older FDDI node sends void frames when it does not have the token, then the switch could see one of these void frames and assume that it is one of its own (i.e. a bridged frame from an Ethernet station destined for a FDDI station) and it will stop removing any spurious frames off the ring. If this happens, the switch's own data frame has not yet returned to the switch. When it does return, the switch assumes that the data frame is a new one from a device on the FDDI side rather than the originating Ethernet station! It therefore places the MAC address in the FDDI forwarding table whereas before this MAC address had been learned on the Ethernet side. This means that any frames destined for the Ethernet station from the FDDI ring get dropped now, since the switch believes that the station now sits on the FDDI, and therefore the switch has no need to forward the frame.

In addition, the switch sees that the frame is not from itself (the MAC address being the Ethernet station), assumes that the originating FDDI station will remove the frame once it sees that it has come around again, according to the FDDI specification, and because such a FDDI node does not exist, the frame endlessly travels around the ring taking up bandwidth.

Processes exist such as Cisco's FDDIcheck that are used to discard frames seen on the FDDI ring if the frame's Source address was learned on the Ethernet side thereby preventing the problem of endlessly looping frames described above. In other words, enabling this FDDI Check feature prevents FDDI from relearning MAC addresses learned from an Ethernet interface, however this can have a performance impact on the switch.

Techniques such as Cisco's Automated Packet Recognition and Translation (APaRT) associate Layer 2 frame types with MAC addresses in the FDDI forwarding table and automatically translates between either FDDI_SNAP or FDDI_802.2 and all the Ethernet frame types. This forwarding table can be disabled to improve performance.

References

RFC 1042 describes the SNAP format.

Home

Disclaimer