How to break out from boot-panic cycle during patching exercise to collect core dump?

Asset ID:	1-75-1010041.1
Update Date:	2009-09-15
Keywords:

Solution Type Troubleshooting Sure

Solution 1010041.1 : How to break out from boot-panic cycle during patching exercise to collect core dump?

Related Items


Sun Ultra 80 Workstation

Related Categories


GCS>Sun Microsystems>Desktops>Workstations

PreviouslyPublishedAs
213790

Description
Sometime the system enter boot-panic cycle immediately after a patching exercise. The fastest way to recover from such situation is to run patchrm after booting from external device. This will restore the system to pre-patch stage, but the cause of the panic may never be known because vmcore cannot be collected.

Steps to Follow
This is a way to collect the vmcore and unix files pair while break out of the boot-panic cycle.

Immediately after a panic, boot the system from external device, eg CD-ROM or from network
use command "dd" to copy the swap device to a file
use "savecore -f <the file> <dest dir>" to generate the vmcore, unix pair

Example

system panicked, dumping to /dev/dsk/c0t1d0s1.
message also shown dump succeeded:

     panic[cpu5]/thread=300010d5400: BAD TRAP: type=31 rp=2a10042b7b0 addr=a80000 mmu_fsr=0 occurred in module "unix" due to an
illegal access to a user address

     nscd: trap type = 0x31
addr=0xa80000
pid=184, pc=0x10031a64, sp=0x2a10042b051, tstate=0x1601, context=0x52
g1-g7: 0, 65, 100d04b8, 240, 10433900, 0, 300010d5400

     000002a10042b3d0 unix:die+80 (31, a80000, 10414f98, 0, 2a10042b7b0, d3f21000)
%l0-3: 0000000000000010 0000000000039d10 0000000000000000 0000000000000000
%l4-7: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
000002a10042b4b0 unix:trap+8b8 (a80000, 1, 5, 0, 2a10042b7b0, 0)
%l0-3: 0000000000000001 0000000000000000 0000030000e87548 0000000000000000
%l4-7: 0000000000000031 0000030001068df8 0000000000010000 0000000000000000
000002a10042b5f0 unix:sfmmu_tsb_miss+640 (104286e0, 0, 300002bdf88, 0, 300002bdf88, 19)
%l0-3: 0000000000a80000 0000000000000004 0000000000000001 000003100003f140
%l4-7: 0000000000a80000 0000000000000000 0000000000000000 0000000000800003
000002a10042b700 unix:prom_rtt+0 (a81e18, 300010d5400, 34, 2a10042ba20, c, 0)
%l0-3: 0000000000000002 0000000000001400 0000000000001601 0000000010018e3c
%l4-7: 000000000000ff00 0000000081000100 0000000000000000 000002a10042b7b0
000002a10042b850 genunix:lookuppn+c4 (a81e18, 0, 0, a81e18, 2a10042bae8, 1)
%l0-3: 0000000000000000 0000030000e87548 000002a10042ba10 0000000000000000
%l4-7: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
000002a10042b910 genunix:lookupname+a4 (0, 0, 1, 0, 2a10042bae8, 1c978)
%l0-3: 0000000000000001 0000000000000000 0000000000000001 00000300010d56c0
%l4-7: 00000000fea03c78 000002a100431af0 0000000000000000 0000000000000000
000002a10042ba30 genunix:stat32+14 (1c978, fd201c28, 0, 0, 0, 0)
%l0-3: 0000000010151e5c 00000300010d56c0 0000000000000000 0000000000000000
%l4-7: 0000000000000000 0000000000000000 0000000000000000 0000000000000000

     syncing file systems... done
dumping to /dev/dsk/c0t1d0s1, offset 54067200
100% done: 7628 pages dumped, compression ratio 5.92, dump succeeded
rebooting...
Resetting...

before system boot from hard disk and enter the boot panic cycle, boot it from external device, eg network or CD-ROM

     {4} ok
{4} ok boot net
Resetting...
Software Power ON
...

go to command prompt

     # id
uid=0(root) gid=0(root)
# pwd
/tmp/root

mount a spare disk to be used to collect the result of "dd"

     # format
Searching for disks...done

     AVAILABLE DISK SELECTIONS:
0. c0t1d0 <SUN9.0G cyl 4924 alt 2 hd 27 sec 133>
/sbus@3,0/SUNW,fas@3,8800000/sd@1,0
1. c1t12d0 <SUN4.2G cyl 3880 alt 2 hd 16 sec 135>
/pci@7,4000/SUNW,isptwo@3/sd@c,0
2. c1t13d0 <SUN4.2G cyl 3880 alt 2 hd 16 sec 135>
/pci@7,4000/SUNW,isptwo@3/sd@d,0
Specify disk (enter its number): ^D

     # mount /dev/dsk/c0t1d0s0 /a
# cd /a/var/crash/*
# ls
bounds
# pwd
/a/var/crash/v4u-6500a

dd the swap device to a file

     # dd if=/dev/rdsk/c0t1d0s1 of=/a/var/crash/v4u-6500a/dump.out.dd
527877+0 records in
527877+0 records out

convert the dd out file to the dump pair

     # ls -al
total 528182
drwx------   2 root     root         512 Jan  2 14:34 .
drwxr-xr-x   3 root     root         512 Sep 27 11:47 ..
-rw-r--r--   1 root     root           2 Dec  5 08:59 bounds
-rw-r--r--   1 root     root     270273024 Jan  2 14:41 dump.out.dd
# cat bounds
6
# savecore -f ./dump.out.dd /a/var/crash/v4u-6500a
System dump time: Wed Jan  2 13:43:54 2008
Constructing namelist /a/var/crash/v4u-6500a/unix.6
Constructing corefile /a/var/crash/v4u-6500a/vmcore.6
100% done: 7628 of 7628 pages saved
# ls -al
total 653446
drwx------   2 root     root         512 Jan  2 14:49 .
drwxr-xr-x   3 root     root         512 Sep 27 11:47 ..
-rw-r--r--   1 root     root           2 Jan  2 14:49 bounds
-rw-r--r--   1 root     root     270273024 Jan  2 14:49 dump.out.dd
-rw-r--r--   1 root     root      590863 Jan  2 14:49 unix.6
-rw-r--r--   1 root     root     63488000 Jan  2 14:49 vmcore.6

send for analysis

     # ftp cores.central

Product
Ultra 80 Workstation

boot-panic cycle, patching exercise, collect coredump
Previously Published As
91630

Change History
Date: 2008-01-03
User Name: 97961
Action: Approved
Comment: - Audience changed to "Contract" per FvF http://kmo.central/howto/FvF.html
Also refer to: http://kmo.central/howto/content/voyager-contributor-standards.html
- Converted to STM formatting for better readability
- Made simple sentence/grammatical corrections
Version: 3
Date: 2008-01-03
User Name: 97961
Action: Accept
Comment:
Version: 0
Date: 2008-01-03
User Name: 38638
Action: Approved
Comment: Contents reviewed, looked great.

Attachments

This solution has no attachment