Sun Microsystems, Inc.  Sun System Handbook - ISO 3.4 June 2011 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-72-1006491.1
Update Date:2010-09-01
Keywords:

Solution Type  Problem Resolution Sure

Solution  1006491.1 :   Sun Fire[TM] 12K/15K/E20K/E25K: System Controller /var/adm/messages shows many eri "link up" (and sometimes "link down") messages  


Related Items
  • Sun Fire E25K Server
  •  
  • Sun Fire E20K Server
  •  
  • Sun Fire 12K Server
  •  
  • Sun Fire 15K Server
  •  
Related Categories
  • GCS>Sun Microsystems>Servers>High-End Servers
  •  

PreviouslyPublishedAs
209087


Symptoms
On high end Sun Fire[TM] servers, such as the Sun Fire[TM] 15K, many messages indicating that eri interfaces are "up" are noticed in the /var/adm/messages file of the Main SC (System Controller).

Example:

Oct 13 16:03:32 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri7 : 100 Mbps half duplex link up
Oct 13 16:04:14 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri9 : 100 Mbps half duplex link up
Oct 13 16:04:55 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri8 : 100 Mbps half duplex link up
Oct 13 16:05:38 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri7 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri9 : 100 Mbps half duplex link up
Oct 13 16:07:00 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri8 : 100 Mbps half duplex link up
Oct 13 16:07:41 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri7 : 100 Mbps half duplex link up
Oct 13 16:08:22 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri9 : 100 Mbps half duplex link up
Oct 13 16:09:03 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri8 : 100 Mbps half duplex link up


Resolution
This issue can have a number of potential causes, including bad hardware. However, experience has shown that the most likely cause of this issue is in fact a network configuration error.

The System controller's SMS (Systems Management Software) configuration includes a file that holds all of the *expected* IP addresses of the domain's I1 (Internal SC to Domain) network.

On the SC:

$ cat /etc/opt/SUNWSMS/config/MAN.cf
f15ka
C       SC-FLOATER-C1   f15ka-ssc       10.15.2.36
C       SC-TEST-hme0-C1 f15ka-sc1-hme0
C       SC-TEST-eri1-C1 f15ka-sc1-eri1
I2      NM-I2   netmask-i2      255.255.255.252
I2      SC0-I2  f15ka-sc0-i2    30.1.1.1
I2      SC1-I2  f15ka-sc1-i2    30.1.1.2
I1      NM-I1   netmask-i1      255.255.255.224
I1      SC-I1   f15ka-sc-i1      20.1.1.1
I1      DA-I1   f15ka-a 20.1.1.2    <------------------**NOTE!**
I1      DB-I1   f15ka-b 20.1.1.3
I1      DC-I1   f15ka-c 20.1.1.4
I1      DD-I1   f15ka-d 20.1.1.5
<truncated for brevity>

As shown above, it is expected that domain A should use the IP address 20.1.1.2 for its dman0 (I1 network) interface.

Moving to the domain, the configuration can be checked in a few ways:

f15ka-dom-a# ndd -get /dev/dman man_get_hostinfo
manc_magic = 0x4d414e43
manc_version = 01
manc_csum = 0x0
manc_ip_type = AF_INET
manc_dom_ipaddr = 20.1.1.2     <--------------**NOTE**
manc_dom_ip_netmask = 255.255.255.224
manc_dom_ip_netnum = 20.1.1.0
manc_sc_ipaddr = 20.1.1.1
manc_dom_eaddr = 0:0:be:a8:9:aa
manc_sc_eaddr = 8:0:20:f9:f7:40
manc_iob_bitmap = 0x38000       io boards = 15.1, 16.1, 17.1,
manc_golden_iob = 15

This command retrieves information from the IOSRAM (Input Output SRAM) which is the SC's CURRENT configuration for the domain's IP address.

Compare this against the file /etc/hostname.dman0 and the output of running the command ifconfig dman0.

f15ka-dom-a# ifconfig dman0
dman0: flags=1008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu 1500 index 3
inet 20.1.1.3 netmask ffffffe0 broadcast 20.255.255.255
ether 0:0:be:a8:9:aa
f15ka-dom-a# cat /etc/hostname.dman0
20.1.1.3 netmask + broadcast + private up

It can seen above that the configuration does NOT agree. In this case, set the IP address to 20.1.1.2, as configured on the SC. This should be also be changed in the /etc/hostname.dman0 file.

Once the change has been made, reboot to see the change take effect (A good idea to check that the change was correctly made).

Alternatively, changes can be made on the fly with the following command:

f15ka-dom-a# ifconfig dman0 20.1.1.2 netmask 0xffffffe0 broadcast + up

In the event that it is demonstrated that the network configuration is correct, and all files, ndd and ifconfig have matching configuration, the possibility of missing patches and bad hardware should be considered.



Additional Information
Included all variants of eri2 through eri19 in the message to help this document be found in relevant cases.
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri2 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri3 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri4 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri5 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri6 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri7 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri8 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri9 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri10 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri11 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri12 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri13 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri14 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri15 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri16 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri17 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri18 : 100 Mbps half duplex link up
Oct 13 16:06:19 f15ka-dom-a eri: [ID 517527 kern.info] SUNW,eri19 : 100 Mbps half duplex link up

See also:

  •  Symptom Resolution <Document: 1008914.1> : Sun Fire[TM] 15K: Console stuck in IOSRAM mode
  •  Symptom Resolution <Document: 1007730.1> : Sun Fire[TM] 12K/15K/E20K/E25K: showdevices command fails on system controller




Product
Sun Fire E25K Server
Sun Fire E20K Server
Sun Fire 15K Server
Sun Fire 12K Server
Amazon 20/25 Phase 2 Hardware

Internal Comments
This statement is intended for use by Sun IT and Sun IT Partner Engineers only.

Symptom Resolution <Document: 1010314.1> : Sun Fire[TM] 12K/15K: Troubleshooting the I1 MAN Network
15k, f15k, 25k, e25k, 12k, f12k, 20k, e20k, dman, dman0, I1, network, SMS
Previously Published As 82874

Attachments
This solution has no attachment
  Copyright © 2011 Sun Microsystems, Inc.  All rights reserved.
 Feedback