Interface Troubleshooting
Serial Lines: show interfaces serial Status Line Conditions
| Status Line Condition | Possible Problem | Solution |
|---|---|---|
| Serial x is up, line protocol is up |
--- | This is the proper status line condition. No action required. |
| Serial x is down, line protocol is down (DTE mode) |
Typically indicates that the router is not sensing a CD signal (that is, CD is not active).
|
Step 1
Check the LEDs on the CSU/DSU to see whether CD is
active, or insert a breakout box on the line to check for the CD signal. Step 2 Verify that you are using the proper cable and interface (see your hard- ware installation documentation). Step 3 Insert a breakout box and check all control leads. Step 4 Contact your leased-line or other carrier service to see whether there is a problem. Step 5 Swap faulty parts. Step 6 If you suspect faulty router hardware, change the serial line to another port. If the connection comes up, the previously connected interface has a problem. |
| Serial x is up, line protocol is down (DTE mode) |
|
Step 1
Put the modem, CSU, or DSU in local loopback mode and
use the show interfaces serial command to determine whether the line
protocol comes up. If the line protocol comes up, a telephone company problem or a failed remote router is the likely problem. Step 2 If the problem appears to be on the remote end, repeat Step 1 on the remote modem, CSU, or DSU. Step 3 Verify all cabling. Make certain that the cable is attached to the correct interface, the correct CSU/DSU, and the correct telephone company network termination point. Use the show controllers exec command to determine which cable is attached to which interface. Step 4 Enable the debug serial interface exec command. Caution: Because debugging output is assigned high priority in the CPU process, it can render the system unusable. For this reason, use debug commands only to troubleshoot specific problems or during troubleshooting sessions with Cisco technical support staff. Moreover, it is best to use debug commands during periods of lower network traffic and fewer users. Debugging during these periods decreases the likelihood that increased debug command processing overhead will affect system use. |
| Serial x is
up, line protocol is down (DTE mode) |
Step 5
If the line protocol does not come up in local loopback
mode and if the output of the debug serial interface exec command
shows that the keepalive counter is not incrementing, a router hardware
problem is likely. Swap router interface hardware. Step 6 If the line protocol comes up and the keepalive counter increments, the problem is not in the local router. Step 7 If you suspect faulty router hardware, change the serial line to an unused port. If the connection comes up, the previously connected interface has a problem. |
|
| Serial x is up, line protocol is down (DCE mode) |
|
Step 1
Add the clockrate interface configuration command
on the serial interface. Syntax: clock rate bps Syntax Description: bps---Desired clock rate in bits per second: 1200, 2400, 4800, 9600, 19200, 38400, 56000, 64000, 72000, 125000, 148000, 250000, 500000, 800000, 1000000, 1300000, 2000000, 4000000, or 8000000. Step 2 Set the DTE device to SCTE mode if possible. If your CSU/DSU does not support SCTE, you might have to disable SCTE on the Cisco router interface. Step 3 Verify that the correct cable is being used. Step 4 If the line protocol is still down, there is a possible hardware failure or cabling problem. Insert a breakout box and observe leads. Step 5 Replace faulty parts as necessary. |
| Serial x is up, line protocol is up (looped) |
Loop exists in circuit. The sequence number in the keepalive packet changes to a random number when a loop is initially detected. If the same random number is returned over the link, a loop exists. |
Step 1 Use the show running-config
privileged exec command to look for any loopback interface
configuration command entries. Step 2 If you find a loopback interface configuration command entry, use the no loopback interface configuration command to remove the loop. Step 3 If you do not find the loopback interface configuration command, examine the CSU/DSU to determine whether they are configured in manual loopback mode. If they are, disable manual loopback. Step 4 Reset the CSU or DSU and inspect the line status. If the line protocol comes up, no other action is needed. Step 5 If the CSU or DSU is not configured in manual loopback mode, contact the leased-line or other carrier service for line troubleshooting assistance. |
| Serial x is up, line protocol is down (disabled) |
|
Step 1
Troubleshoot the line with a serial analyzer and
breakout box. Look for toggling CTS7
and DSR8
signals. Step 2 Loop CSU/DSU (DTE loop). If the problem continues, it is likely that there is a hardware problem. If the problem does not continue, it is likely that there is a telephone company problem. Step 3 Swap out bad hardware as required (CSU, DSU, switch, local or remote router). |
| Serial x is administratively down, line protocol is down |
|
Step 1
Check the router configuration for the shutdown
command. Step 2 Use the no shutdown interface configuration command to remove the shutdown command. Step 3 Verify that there are no identical IP addresses using the show running-config privileged exec command or the show interfaces exec command. Step 4 If there are duplicate addresses, resolve the conflict by changing one of the IP addresses. |
Serial Lines: Increasing Output Drops on Serial Link
| Possible Problem | Solution |
|---|---|
| Input rate to serial interface exceeds bandwidth available on serial link | Step 1 Minimize periodic
broadcast traffic such as routing and SAP updates by using access lists or
by other means. For example, to increase the delay between SAP updates, use
the ipx sap-interval interface configuration command. Step 2 Increase the output hold queue size in small increments (for instance, 25 percent), using the hold-queue out interface configuration command. Step 3 On affected interfaces, turn off fast switching for heavily used protocols. For example, to turn off IP fast switching, enter the no ip route-cache interface configuration command. For the command syntax for other protocols, consult the Cisco IOS configuration guides and command references. Step 4 Implement priority queuing on slower serial links by configuring priority lists. For information on config- uring priority lists, see the Cisco IOS configuration guides and command references. Note: Output drops are acceptable under certain conditions. For instance, if a link is known to be overused (with no way to remedy the situation), it is often considered preferable to drop packets than to hold them. This is true for protocols that support flow control and can retransmit data (such as TCP/IP and Novell IPX). However, some protocols, such as DECnet and local-area transport are sensitive to dropped packets and accommodate retransmission poorly, if at all. |
Serial Lines: Increasing Input Drops on Serial Link
| Possible Problem | Solution |
|---|---|
| Input rate exceeds the capacity of the router or input queues exceed the size of output queues | Note: Input drop
problems are typically seen when traffic is being routed between faster
interfaces (such as Ethernet, Token Ring, and FDDI) and serial interfaces.
When traffic is light, there is no problem. As traffic rates increase,
backups start occurring. Routers drop packets during these congested
periods. Step 1 Increase the output queue size on common destination interfaces for the interface that is dropping packets. Use the hold-queue out interface configuration command. Increase these queues by small increments (for instance, 25%) until you no longer see drops in the show interfaces output. The default output hold queue limit is 100 packets. Step 2 Reduce the input queue size, using the hold-queue in interface configuration command, to force input drops to become output drops. Output drops have less impact on the performance of the router than do input drops. The default input hold queue is 75 packets. |
Serial Lines: Increasing Input Errors in Excess of 1% of Total Interface Traffic
| Possible Problem | Solution |
|---|---|
The following problems can result in
this symptom:
|
Note:
Cisco strongly recommends against the use of data
converters when you are connecting a router to a WAN or serial network. Step 1 Use a serial analyzer to isolate the source of the input errors. If you detect errors, it is likely that there is a hardware problem or a clock mismatch in a device that is external to the router. Step 2 Use the loopback and ping tests to isolate the specific problem source. Step 3 Look for patterns. For example, if errors occur at a consistent interval, they could be related to a periodic function such as the sending of routing updates. |
Serial Lines: Troubleshooting Serial Line Input Errors
| Input Error Type (Field Name) |
Possible Problem | Solution |
|---|---|---|
| CRC errors (CRC) |
CRC errors occur when the CRC calculation does not pass (indicating that data is corrupted) for one of the following reasons:
|
Step 1
Ensure that the line is clean enough for transmission
requirements. Shield the cable if necessary. Step 2 Make sure the cable is within the recommended length (no more than 50 feet [15.24 meters], or 25 feet [7.62 meters] for T1 link). Step 3 Ensure that all devices are properly configured for a common line clock. Set SCTE on the local and remote DSU. I Step 4 Make certain that the local and remote CSU/DSU are configured for the same framing and coding scheme as that used by the leased-line or other carrier service (for example, ESF/B8ZS). Step 5 Contact your leased-line or other carrier service and have it perform integrity tests on the line. |
| Framing errors (frame) |
A framing error occurs when a packet does not end on an 8-bit byte boundary for one of the following reasons:
|
Step 1
Ensure that the line is clean enough for transmission
requirements. Shield the cable if necessary. Make certain you are using the
correct cable. Step 2 Make sure the cable is within the recommended length (no more than 50 feet [15.24 meters], or 25 feet [7.62 meters] for T1 link) Step 3 Ensure that all devices are properly configured to use a common line clock. Set SCTE on the local and remote DSU. Step 4 Make certain that the local and remote CSU/DSU is configured for the same framing and coding scheme as that used by the leased-line or other carrier service (for example, ESF/B8ZS). Step 5 Contact your leased-line or other carrier service and have it perform integrity tests on the line. |
| Aborted transmission (abort) |
Aborts indicate an illegal sequence of
one bits (more than seven in a row) The following are possible reasons for this to occur:
|
Step 1 Ensure that all devices
are properly configured to use a common line clock. Set SCTE on the local
and remote DSU. Step 2
Shield the cable if necessary. Make certain the cable is
within the recommended length Step 3 Check the hardware at both ends of the link. Swap faulty equipment as necessary. Step 4 Lower data rates and determine whether aborts decrease. Step 5 Use local and remote loopback tests to determine where aborts are occurring. Step 6 Contact your leased-line or other carrier service and have it perform integrity tests on the line. |
Serial Lines: Increasing Interface Resets on Serial Link
| Possible Problem | Solution |
|---|---|
The following problems can result in
this symptom:
|
When interface resets are occurring,
examine other fields of the show interfaces serial command output to
determine the source of the problem. Assuming that an increase in interface
resets is being recorded, examine the following fields: Step 1 If there is a high number of output drops in the show interfaces serial output, see the section "Serial Lines: Increasing Output Drops on Serial Link" earlier in this chapter. Step 2 Check the carrier transitions field in the show interfaces serial display. If carrier transitions are high while interface resets are being registered, the problem is likely to be a bad link or bad CSU or DSU. Contact your leased-line or carrier service and swap faulty equipment as necessary. Step 3 Examine the input errors field in the show interfaces serial display. If input errors are high while interface resets are increasing, the problem is probably a bad link or bad CSU/DSU. Contact your leased-line or other carrier service and swap faulty equipment as necessary. |
Serial Lines: Increasing Carrier Transitions Count on Serial Link
| Possible Problem | Solution |
|---|---|
The following problems can result in
this symptom:
|
Step 1
Check hardware at both ends of the link (attach a
breakout box or a serial analyzer and test to determine source of problems). Step 2 If an analyzer or breakout box is unable to identify any external problems, check the router hardware. Step 3 Swap faulty equipment as necessary. |
Serial Lines: Clocking Problems and Solutions
| Possible Problem | Solution |
|---|---|
| Incorrect CSU configuration | Step 1 Determine whether the
CSUs at both ends agree on the clock source (local or line). Step 2 If the CSUs do not agree, configure them so that they do (usually the line is the source). Step 3 Check the LBO1 setting on the CSU to ensure that the impedance matches that of the physical line. For information on configuring your CSU, consult your CSU hardware documentation. |
| Incorrect DSU configuration | Step 1 Determine whether the
DSUs at both ends have SCTE mode enabled. Step 2 If SCTE is not enabled on both ends of the connection, enable it. (For any interface that is connected to a line of 128 kbps or faster, SCTE must be enabled. Step 3 Make sure that ones density is maintained. This requires that the DSU use the same framing and coding schemes (for example, ESF and B8ZS) used by the leased-line or other carrier service. Check with your leased-line provider for information on its framing and coding schemes. Step 4 If your carrier service uses AMI coding, either invert the transmit clock on both sides of the link or run the DSU in bit-stuff mode. For information on configuring your DSU, consult your DSU hardware documentation. |
| Cable to router out of specification | If the cable is longer than 50 feet
(15.24 meters), use a shorter cable. If the cable is unshielded, replace it with shielded cable. |
Show Interfaces Serial Field Descriptions
| Field | Description |
| Serial...is {up | down} ...is administratively down |
Indicates whether the interface hardware is currently active (whether carrier detect is present) or whether it has been taken down by an administrator. |
| line protocol is {up | down} |
Indicates whether the software processes that handle the line protocol consider the line usable (that is, whether keepalives are successful) or whether it has been taken down by an administrator. |
| Hardware is | Specifies the hardware type. |
| Internet address is | Specifies the Internet address and subnet mask. |
| MTU | Maximum transmission unit of the interface. |
| BW | Indicates the value of the bandwidth parameter that has been configured for the interface (in kilobits per second). The bandwidth parameter is used to compute IGRP metrics only. If the interface is attached to a serial line with a line speed that does not match the default (1536 or 1544 for T1 and 56 for a standard synchronous serial line), use the bandwidth command to specify the correct line speed for this serial line. |
| DLY | Delay of the interface in microseconds. |
| rely | Reliability of the interface as a fraction of 255 (255/255 is 100% reliability), calculated as an exponential average over five minutes. |
| load | Load on the interface as a fraction of 255 (255/255 is completely saturated), calculated as an exponential average over five minutes. |
| Encapsulation | Encapsulation method assigned to the interface. |
| loopback | Indicates whether loopback is set. |
| keepalive | Indicates whether keepalives are set. |
| Last input | Number of hours, minutes, and seconds since the last packet was successfully received by an interface. Useful for knowing when a dead interface failed. |
| Last output | Number of hours, minutes, and seconds since the last packet was successfully transmitted by an interface. |
| output hang | Number of hours, minutes, and seconds (or never) since the interface was last reset because of a transmission that took too long. When the number of hours in any of the last fields exceeds 24, the number of days and hours is printed. If that field overflows, asterisks are printed. |
| Output queue, drops input queue, drops |
Number of packets in output and input queues. Each number is followed by a slash, the maximum size of the queue, and the number of packets because the queue is full. |
| 5 minute input rate 5 minute output rate |
Average number of bits and packets
transmitted per second in the past five minutes. The five-minute input and output rates should be used only as an approximation of traffic per second during a given five-minute period. These rates are exponentially weighted averages with a time constant of five minutes. A period of four time constants must pass before the average will be within 2% of the instantaneous rate of a uniform stream of traffic over that period. |
| packets input | Total number of error-free packets received by the system. |
| bytes | Total number of bytes, including data and MAC encapsulation, in the error-free packets received by the system. |
| no buffer | Number of received packets discarded because there was no buffer space in the main system. Compare with ignored count. Broadcast storms on Ethernet networks and bursts of noise on serial lines are often responsible for no input buffer events. |
| Received...broadcasts | Total number of broadcast or multicast packets received by the interface. |
| runts | Number of packets that are discarded because they are smaller than the medium's minimum packet size. |
| giants | Number of packets that are discarded because they exceed the medium's maximum packet size. |
| input errors | Total number of no buffer, runts, giants, CRCs, frame, overrun, ignored, and abort counts. Other input-related errors can also increment the count, so this sum might not balance with the other counts. |
| CRC | Cyclic redundancy check generated by the originating station or far-end device does not match the checksum calculated from the data received. On a serial link, CRCs usually indicate noise, gain hits, or other transmission problems on the data link. |
| frame | Number of packets received incorrectly having a CRC error and a noninteger number of octets. On a serial line, this is usually the result of noise or other transmission problems. |
| overrun | Number of times the serial receiver hardware was unable to hand received data to a hardware buffer because the input rate exceeded the receiver's ability to handle the data. |
| ignored | Number of received packets ignored by the interface because the interface hardware ran low on internal buffers. Broadcast storms and bursts of noise can cause the ignored count to be increased. |
| abort | Illegal sequence of one bits on a serial interface. This usually indicates a clocking problem between the serial interface and the data link equipment. |
| carrier transitions | Number of times the carrier detect signal of a serial interface has changed state. For example, if data carrier detect (DCD) goes down and comes up, the carrier transition counter will increment two times. Indicates modem or line problems if the carrier detect line is changing state often. |
| packets output | Total number of messages transmitted by the system. |
| bytes output | Total number of bytes, including data and MAC encapsulation, transmitted by the system. |
| underruns | Number of times that the transmitter has been running faster than the router can handle. This might never be reported on some interfaces. |
| output errors | Sum of all errors that prevented the final transmission of datagrams out of the interface being examined. Note that this might not balance with the sum of the enumerated output errors because some datagrams can have more than one error, and others can have errors that do not fall into any of the specifically tabulated categories. |
| collisions | Number of messages retransmitted due to an Ethernet collision. This usually is the result of an overextended LAN (Ethernet or transceiver cable too long, more than two repeaters between stations, or too many cascaded multiport transceivers). Some collisions are normal. However, if your collision rate climbs to around 4% or 5%, you should consider verifying that there is no faulty equipment on the segment and/or moving some existing stations to a new segment. A packet that collides is counted only once in output packets. |
| interface resets | Number of times an interface has been completely reset. This can happen if packets queued for transmission were not sent within several seconds' time. On a serial line, this can be caused by a malfunctioning modem that is not supplying the transmit clock signal, or by a cable problem. If the system notices that the carrier detect line of a serial interface is up but the line protocol is down, it periodically resets the interface in an effort to restart it. Interface resets can also occur when an interface is looped back or shut down. |
| restarts | Number of times the controller was restarted because of errors. |
| alarm indications, remote alarms, rx LOF, rx LOS | Number of CSU/DSU alarms, and number of occurrences of receive loss of frame and receive loss of signal. |
| BER inactive, NELR inactive, FELR inactive | Status of G.703-E1 counters for bit error rate (BER) alarm, near-end loop remote (NELR), and far-end loop remote (FELR). Note that you cannot set the NELR or FELR. |