Sun Fire[TM] 12K/15K/E20K/E25K: After upgrading hsPCI to hsPCI+ domain fails to boot

Asset ID:	1-72-1006575.1
Update Date:	2011-04-26
Keywords:

Solution Type Problem Resolution Sure

Solution 1006575.1 : Sun Fire[TM] 12K/15K/E20K/E25K: After upgrading hsPCI to hsPCI+ domain fails to boot

Applies to:

Sun Fire 12K Server
Sun Fire 15K Server
Sun Fire E20K Server
Sun Fire E25K Server
All Platforms

Symptoms

This document describes the extra actions you need to take when upgrading or replacing a hsPCI I/O board to a hsPCI+ I/O board: if your bootdisk is attached to slot1, aka C5V0, top right IO-slot, your domain will fail to boot.

Cause

The devicepath to the bootdisk changed as a result of the hardware upgrade.

Solution

Although the hardware seems to be transparent a reboot -r is necessary to finish the upgrade successfully and rebuild the devicetree accordingly.

If indeed the devicepath to your bootdisk is affected by the upgrade, follow the steps in Document 1011952.1 "Rebuild a device tree booted from a Solaris[TM] 8, 9 or 10 Operating Environment cdrom".

Product
hsPCIX for Sun Fire E25K/E20K Servers
Sun Fire E25K Server
Sun Fire E20K Server
Sun Fire 15K Server
Sun Fire 12K Server

Internal Section

Example

After replacing the HSPCI IO-board with a HSPCI+ IO-board we poweron our domain by running setkeyswitch on. After successfully going through post the domain starts booting....

Sun Fire 15000, using IOSRAM based Console

OpenBoot 4.7.6, 8192 MB memory installed, Serial #4456xxxx.

Ethernet address 0:0:be:xx:x:xx, Host ID: 82axxxxx.

Boot device: /pci@15c,700000/pci@1/scsi@2/disk@1,0:a File and args:

SunOS Release 5.9 Version Generic_118558-05 64-bit

Use is subject to license terms.

WARNING: /pci@15c,700000/pci@1/pci@1/scsi@2 (glm2):

Connected command timeout for Target 0.0

WARNING: /pci@15c,700000/pci@1/pci@1/scsi@2 (glm2):

got SCSI bus reset

WARNING: /pci@15c,700000/pci@1/pci@1/scsi@2 (glm2):

Connected command timeout for Target 0.0

WARNING: /pci@15c,700000/pci@1/pci@1/scsi@2 (glm2):

got SCSI bus reset

ifconfig: SIOCSLIFNAME for ip: ce0: no such interface

configuring IPv4 interfaces: dman0.

moving addresses from failed IPv4 interfaces: ce0 (couldn't move, no alternative interface).

moving addresses from failed IPv6 interfaces: ce0 (couldn't move, no alternative interface).

Hostname: v4u-15ka-r-gmp03

/dev/dsk/c0t1d0s1: No such device or address

The / file system (/dev/rdsk/c0t1d0s0) is being checked.

Can't open /dev/rdsk/c0t1d0s0

/dev/rdsk/c0t1d0s0: CAN'T CHECK FILE SYSTEM.

/dev/rdsk/c0t1d0s0: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.

WARNING - Unable to repair the / filesystem. Run fsck

manually (fsck -F ufs /dev/rdsk/c0t1d0s0). Exit the shell when

done to continue the boot process.

Type control-d to proceed with normal startup,

(or give root password for system maintenance):

..and fails because all devices in slot1 now have a different devicepath.

Our boot-device points to /pci@15c,700000/pci@1/scsi@2/disk@1,0:a.

The new HSPCI+ IO-board has an extra PCI bridge-chip, which shows at lines a/b.

Now, that we can fix.

{82} ok show-disks

a) /pci@15c,700000/pci@1/pci@1/scsi@2,1/disk

b) /pci@15c,700000/pci@1/pci@1/scsi@2/disk

q) NO SELECTION

Enter Selection, q to quit: b

/pci@15c,700000/pci@1/pci@1/scsi@2/disk has been selected.

Type ^Y ( Control-Y ) to insert it in the command line.

e.g. ok nvalias mydev ^Y

for creating devalias mydev for /pci@15c,700000/pci@1/pci@1/scsi@2/disk

{82} ok printenv boot-device

boot-device = /pci@15c,700000/pci@1/scsi@2/disk@1,0:a

{82} ok setenv boot-device /pci@15c,700000/pci@1/pci@1/scsi@2/disk@1,0:a

boot-device = /pci@15c,700000/pci@1/pci@1/scsi@2/disk@1,0:a

{82} ok boot

Resetting...

Sun Fire 15000, using IOSRAM based Console

OpenBoot 4.7.6, 8192 MB memory installed, Serial #445xxxxx.

Ethernet address 0:0:be:xx:x:xx, Host ID: 82axxxxx.

Rebooting with command: boot

Boot device: /pci@15c,700000/pci@1/pci@1/scsi@2/disk@1,0:a File and args:

Evaluating:

Can't open boot device

{82} ok The dxs disconnected.

Trying to connect...

Now we are ending up in a failing reboot loop. Bummer. Stop autoboot...

renef@v4u-15ka-sc0-gmp03[/home/renef:3344] showobpparams -d R

auto-boot?=true

diag-switch?=false

fcode-debug?=false

use-nvramrc?=true

security-mode=none

renef@v4u-15ka-sc0-gmp03[/home/renef:3345] setobpparams -d R auto-boot?=false

renef@v4u-15ka-sc0-gmp03[/home/renef:3346] showobpparams -d R

auto-boot?=false

diag-switch?=false

fcode-debug?=false

use-nvramrc?=true

security-mode=none

renef@v4u-15ka-sc0-gmp03[/home/renef:3347]

Now rebuild your device tree following Document 1011952.1

Note that the path to the "golden eri" is different when using an HsPCI board vs an HsPCI+ board.

Ex for HsPCI in slot 9 - SMS 1.6 / OpenBoot 4.21.0 :

{122} ok {122} ok show-nets

a) /pci@13c,700000/network@3,1

...

Ex for HsPCI+ in slot 14 - SMS 1.6 / OpenBoot 4.21.0 :

{180} ok show-nets

...

c) /pci@1dc,700000/pci@1/network@3,1

...

This path to the "golden eri" is used to create the OBP alias "man-net".

Booting on man-net still works even if the alias is not valid.

See Bug 6446922 man-net on HsPCI+ based 15k domain is not valid

Keywords: hsPCI, hsPCI+, upgrade, 12k, 15k, E20k, E25k, boot

Previously Published As 81352

Attachments

This solution has no attachment