| Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback |
| Sun System Handbook - ISO 3.4 June 2011 Internal/Partner Edition | ||
|
|
||
 |
||||||||||||
Solution Type Problem Resolution Sure Solution 1007782.1 : Dynamic Reconfiguration or hot plug problems on a QGE HBA (4444A).
PreviouslyPublishedAs 210767 Symptoms After unconfiguring a Quad Gigabit Ethernet (QGE) Host Bus Adaptor (HBA) and then configuring it back from within a domain using the cfgadm command, the NIC ports on the QGE HBA do not work anymore (i.e. plumb up) unless the domain is rebooted. Execute the "devfsadm" command does not help, and "cfgadm" command shows everything works fine. However "prtdiag -v" command shows some noticeable differences on the slot when comparing the output of a functioning QGE HBA and a non-functioning QGE HBA after it has been unconfigured/configured. Sample output of the encounter: The QGE HBA is at IO16/Slot 0 and before performing DR: root@dm1c1h # cfgadm |grep pcisch5:e16b1slot0 pcisch5:e16b1slot0 pci-pci/hp connected configured ok root@dm1c1h # prtdiag -v [..] ========================= IO Cards ========================= Bus Max IO Port Bus Freq Bus Dev, Slot ID Type ID Side MHz Freq Func State Name Model ---------- ---- ---- ---- ---- ---- ---- ----- -------------------------------- ---------------------- ...... /IO16/C5V0 PCI 540 B 33 33 1,0 ok pci-pci8086,b154.0/bootbus-contr+ pci-bridge /IO16/C5V0 PCI 540 B 33 33 2,0 ok bootbus-controller-pci108e,c416/+ SUNW,sbbc /IO16/C5V0 PCI 540 B 33 33 3,0 ok pci108e,1100-pci108e,1100.1 /IO16/C5V0 PCI 540 B 33 33 3,1 ok network-pci108e,1101.1 SUNW,pci-eri /IO16/C5V0 PCI 540 B 33 33 3,2 ok firewire-pci108e,1102.1001 /IO16/C5V0 PCI 540 B 33 33 3,3 ok usb-pci108e,1103.1001 /IO16/C3V0 PCI 540 A 66 66 1,0 ok pci-pci8086,b154.0/pci (pci) pci-bridge /IO16 PCI 540 A 66 66 0,0 ok pci-pci8086,b154.0/network (netw+ pci-bridge /IO16 PCI 540 A 66 66 0,0 ok network-pci100b,35.30 SUNW,pci-qge /IO16 PCI 540 A 66 66 1,0 ok network-pci100b,35.30 SUNW,pci-qge /IO16 PCI 540 A 66 66 4,0 ok pci-pci8086,b154.0/network (netw+ pci-bridge /IO16 PCI 540 A 66 66 2,0 ok network-pci100b,35.30 SUNW,pci-qge /IO16 PCI 540 A 66 66 3,0 ok network-pci100b,35.30 SUNW,pci-qge /IO16/C3V2 PCI 541 B 66 66 1,0 ok pci-pci8086,b154.0/network (netw+ pci-bridge /IO16/C3V2 PCI 541 B 66 66 0,0 ok network-pci100b,35.30 SUNW,pci-ce /IO16/C3V2 PCI 541 B 66 66 1,0 ok network-pci100b,35.30 SUNW,pci-ce /IO16/C3V2 PCI 541 B 66 66 2,0 ok scsi-pci1000,b.7/disk (block) /IO16/C3V2 PCI 541 B 66 66 2,1 ok scsi-pci1000,b.7/disk (block) /IO16/C3V1 PCI 541 A 66 66 1,0 ok SUNW,qlc-pci1077,2312.1077.10a.2+ /IO16/C3V1 PCI 541 A 66 66 1,1 ok SUNW,qlc-pci1077,2312.1077.10a.2+ Unconfigure IO16/Slot 0 (C3V0, a QGE HBA): root@dm1c1h # cfgadm -c unconfigure pcisch5:e16b1slot0 root@dm1c1h # cfgadm |grep pcisch5:e16b1slot0 pcisch5:e16b1slot0 unknown connected unconfigured unknown After unconfiguring the QGE HBA, prtdiag does not show C3V0 entry anymore: root@dm1c1h # prtdiag -v [..] ========================= IO Cards ========================= Bus Max IO Port Bus Freq Bus Dev, Slot ID Type ID Side MHz Freq Func State Name Model ---------- ---- ---- ---- ---- ---- ---- ----- -------------------------------- ---------------------- ..... /IO16/C5V0 PCI 540 B 33 33 1,0 ok pci-pci8086,b154.0/bootbus-contr+ pci-bridge /IO16/C5V0 PCI 540 B 33 33 2,0 ok bootbus-controller-pci108e,c416/+ SUNW,sbbc /IO16/C5V0 PCI 540 B 33 33 3,0 ok pci108e,1100-pci108e,1100.1 /IO16/C5V0 PCI 540 B 33 33 3,1 ok network-pci108e,1101.1 SUNW,pci-eri /IO16/C5V0 PCI 540 B 33 33 3,2 ok firewire-pci108e,1102.1001 /IO16/C5V0 PCI 540 B 33 33 3,3 ok usb-pci108e,1103.1001 /IO16/C3V2 PCI 541 B 66 66 1,0 ok pci-pci8086,b154.0/network (netw+ pci-bridge /IO16/C3V2 PCI 541 B 66 66 0,0 ok network-pci100b,35.30 SUNW,pci-ce /IO16/C3V2 PCI 541 B 66 66 1,0 ok network-pci100b,35.30 SUNW,pci-ce /IO16/C3V2 PCI 541 B 66 66 2,0 ok scsi-pci1000,b.7/disk (block) /IO16/C3V2 PCI 541 B 66 66 2,1 ok scsi-pci1000,b.7/disk (block) /IO16/C3V1 PCI 541 A 66 66 1,0 ok SUNW,qlc-pci1077,2312.1077.10a.2+ /IO16/C3V1 PCI 541 A 66 66 1,1 ok SUNW,qlc-pci1077,2312.1077.10a.2+ root@dm1c1h # ifconfig ce11 plumb Sep 15 17:06:16 dm1c1h ip: ip_rput_dlpi(ce11): DL_ERROR_ACK for DL_ATTACH_REQ(11), errno 8, unix 0 ifconfig: SIOCSLIFNAME for ip: ce11: no such interface root@dm1c1h # Sep 15 17:06:16 dm1c1h ip: ip_rput_dlpi(ce11): DL_ERROR_ACK for DL_BIND_REQ(1), errno 3, unix 0 Sep 15 17:06:16 dm1c1h ip: ip_rput_dlpi(ce11): DL_ERROR_ACK for DL_PHYS_ADDR_REQ(49), errno 3, unix 0 Sep 15 17:06:16 dm1c1h ip: ip_rput_dlpi(ce11): DL_ERROR_ACK for DL_UNBIND_REQ(2), errno 3, unix 0 Sep 15 17:06:16 dm1c1h ip: ip_rput_dlpi(ce11): DL_ERROR_ACK for DL_DETACH_REQ(12), errno 3, unix 0 Configure the IO16/Slot 0 QGE HBA back into domain: root@dm1c1h # cfgadm -c configure pcisch5:e16b1slot0 root@dm1c1h # cfgadm |grep pcisch5:e16b1slot0 pcisch5:e16b1slot0 pci-pci/hp connected configured ok prtdiag now shows missing QGE entries for IO16/C3V0: root@dm1c1h # prtdiag -v [..] ========================= IO Cards ========================= Bus Max IO Port Bus Freq Bus Dev, Slot ID Type ID Side MHz Freq Func State Name Model ---------- ---- ---- ---- ---- ---- ---- ----- -------------------------------- ---------------------- ...... /IO16/C5V0 PCI 540 B 33 33 1,0 ok pci-pci8086,b154.0/bootbus-contr+ pci-bridge /IO16/C5V0 PCI 540 B 33 33 2,0 ok bootbus-controller-pci108e,c416/+ SUNW,sbbc /IO16/C5V0 PCI 540 B 33 33 3,0 ok pci108e,1100-pci108e,1100.1 /IO16/C5V0 PCI 540 B 33 33 3,1 ok network-pci108e,1101.1 SUNW,pci-eri /IO16/C5V0 PCI 540 B 33 33 3,2 ok firewire-pci108e,1102.1001 /IO16/C5V0 PCI 540 B 33 33 3,3 ok usb-pci108e,1103.1001 /IO16/C3V0 PCI 540 A 66 66 1,0 ok pci-pci8086,b154/pci (pci) pci-bridge /IO16 PCI 540 A 66 66 0,0 ok pci-pci8086,b154 pci-bridge /IO16 PCI 540 A 66 66 4,0 ok pci-pci8086,b154 pci-bridge /IO16/C3V2 PCI 541 B 66 66 1,0 ok pci-pci8086,b154.0/network (netw+ pci-bridge /IO16/C3V2 PCI 541 B 66 66 0,0 ok network-pci100b,35.30 SUNW,pci-ce /IO16/C3V2 PCI 541 B 66 66 1,0 ok network-pci100b,35.30 SUNW,pci-ce /IO16/C3V2 PCI 541 B 66 66 2,0 ok scsi-pci1000,b.7/disk (block) /IO16/C3V2 PCI 541 B 66 66 2,1 ok scsi-pci1000,b.7/disk (block) /IO16/C3V1 PCI 541 A 66 66 1,0 ok SUNW,qlc-pci1077,2312.1077.10a.2+ /IO16/C3V1 PCI 541 A 66 66 1,1 ok SUNW,qlc-pci1077,2312.1077.10a.2+ root@dm1c1h # ifconfig ce11 plumb Sep 15 17:09:22 dm1c1h ip: ip_rput_dlpi(ce11): DL_ERROR_ACK for DL_ATTACH_REQ(11), errno 8, unix 0 ifconfig: SIOCSLIFNAME for ip: ce11: no such interface root@dm1c1h # Sep 15 17:09:22 dm1c1h ip: ip_rput_dlpi(ce11): DL_ERROR_ACK for DL_BIND_REQ(1), errno 3, unix 0 Sep 15 17:09:22 dm1c1h ip: ip_rput_dlpi(ce11): DL_ERROR_ACK for DL_PHYS_ADDR_REQ(49), errno 3, unix 0 Sep 15 17:09:22 dm1c1h ip: ip_rput_dlpi(ce11): DL_ERROR_ACK for DL_UNBIND_REQ(2), errno 3, unix 0 Sep 15 17:09:22 dm1c1h ip: ip_rput_dlpi(ce11): DL_ERROR_ACK for DL_DETACH_REQ(12), errno 3, unix 0 root@dm1c1h # devfsadm -C Executing "devfsadm" command does not help and the interface "ce11" Resolution The fix for QGE HBA hotplug patches are available on sunsolve: Solaris[TM] 8 Patch ID: 110900-13 -- Post to sunsolve on 5/25/04 Solaris[TM] 9 Patch ID: 112838-13 -- Post to sunsolve on 5/25/04 Additional Information The QGE HBA can not be used with the V1280/Netra[TM] 1280/E2900. This is because the QGE is a long PCI card and will not mechanically fit into the I/O cage for these servers. Product Sun Fire V880 Server Operating Environments Sun Fire E6900 Server Sun Fire E4900 Server Sun Fire 6800 Server Sun Fire 4810 Server Sun Fire 4800 Server Sun Fire 3800 Server Sun Fire E25K Server Sun Fire E20K Server Sun Fire 15K Server Sun Fire 12K Server Internal Comments The following is strictly for Sun employees: The problem is due to a known bug- Bug ID: 4879904 .This bug applies to all High-End servers Sun Fire[TM] 12K, E20K, SF15K and E20K servers regardless of type of I/O assembly. DR, Dynamic Reconfiguration, cfgadm, GQE, 4444A, Quad Gigabit Ethernet UTP Previously Published As 78350 Change History Date: 2004-09-28 User Name: 7058 Action: Approved Comment: Trademarks in place. Acronyms expanded as needed. Checked patches and the patches do not describe the nature of the problem with examples as well as this doc so I think it should be published. Attachments This solution has no attachment |
||||||||||||
|
||||||||||||