Goal

This document details how to initiate a Service Action Plan to investigate whether a hardware component should be replaced as implicated by the Predictive Self-Healing Diagnosis Engine (FMA DE) on a Sun SPARC Enterprise Mx000 system.

Solution

• An error event caused an automated recovery action to take place on an OPL system (panic/reboot/errors/etc).
  
• The FMA DE determined that a FRU(s) is Faulty and may have automatically disabled or deconfigured the suspect FRU(s).
• The FMA DE produced a FMA Event Code which when looked up in My Oracle Support recommends replacing a FRU and may refer to this document.

1. Collect the FMA Fault Message.

The output can be displayed using fmdump -m on the XSCF console. Example output is as follows:

     MSG-ID: SCF-8001-4X, TYPE: Fault, VER: 1, SEVERITY: Major
     EVENT-TIME: Tue Mar 20 21:23:54 UTC 2007
     PLATFORM: SPARC-Enterprise, CSN: 7860000772, HOSTNAME: genericff2
     SOURCE: sde, REV: 1.12
     EVENT-ID: d93a7654-f414-46aa-aaf5-21f70a5af931
     DESC: The number of uncorrectable and correctable errors on single DIMM exceeds an
     acceptable threshold. This fault is detected while running POST.
      Refer to http://www.sun.com/msg/SCF-8001-4X for more information.
     AUTO-RESPONSE: The memory associated with the memory bank containing the errors is deconfigured.
     IMPACT: POST is restarted after the memory associated with the memory bank has been deconfigured.
     REC-ACTION: Schedule a repair action to replace the affected Field Replaceable Unit (FRU),
     the identity of which can be determined using fmdump -v -u EVENT_ID.
     Please consult the detail section of the knowledge article for additional information.

2. Collect the "fmdump -v -u " output relating to the fault event.

Example (uses the same event as Step 1's example - note the event ID is in bold ):

   xscf> fmdump -v -u d93a7654-f414-46aa-aaf5-21f70a5af931
         TIME                 UUID                                 MSG-ID
         Mar 20 21:23:54.0192 d93a7654-f414-46aa-aaf5-21f70a5af931 SCF-8001-4X
           100%  fault.chassis.SPARC-Enterprise.memory.bank.err
                 Problem in: hc:///chassis=0/cmu=0/mem=0
                    Affects: hc:///chassis=0/cmu=0/mem=0
                        FRU: hc://:product-id=SPARC-Enterprise:chassis-id=7860000772:
                             server-id=san-ff2-21-0:serial=04126711:
                             part=72T128000HR3.7A:revision=252b/component=/MBU_B/MEMB#0/MEM#3A

3. Collect the fault information to prepare to log a Service Request:

• The "FMA Fault Message" from step 1.
• The "fmdump" output (from step 2).
• Specify whether the first FRU in fmdump output has been recently serviced, replaced, or errored.
• Mention this document was referenced in opening the request.
• Specify your contact information so the Oracle Support Services engineer can contact you to schedule the service.

4. Contact Oracle Support Services or your local service representative and open a "Service Request".

5. Review FRU Replacement Methods information to prepare your configuration for the FRU replacement.

• Components can be replaced using three different FRU Replacement Methods depending on which platform is involved, the specific FRU in question, and whether it is redundantly configured.
• It is recommended to review <Document: 1003993.1> Sun SPARC Enterprise Mx000: Field Replaceable Unit (FRU) Replacement Methods to be aware of these replacement methods and prepare for the service.

6. A Oracle Support Services Engineer may need additional data to be collected. If so they will specify the data to collect.

Please assist in capturing requested data so Oracle can resolve your issue with as little delay as possible. The most likely data requested will be:

• XSCF Snapshot data.
• Domain Explorer

Reference <Document: 1008229.1> Running explorer on Sun SPARC Enterprise[TM] Mx000 (OPL) Servers if required for help with either Explorer or Snapshot data requests.

Confirm the fault event message, fmdump output, and all data are from the same date and implicate the same FRU component.
Confirm whether the implicated FRU(s) resources have been disabled or deconfigured.
Use the Predictive Self-Healing ( PSH ) Knowledge Article to confirm the event requires FRU replacement.

The list of FRUs is displayed in the order in which they are intended to be replaced (percentage of likelihood).
Replacements should proceed in the order of likeliest FRU to least likely.
Prepare to have the Primary FRU replaced (proceed to the next step).

The customer may have specified a desired method to use, so verify if it the method desired is possible.
Reference:  <Document 1003993.1> Sun SPARC Enterprise Mx000: Field Replaceable Unit (FRU) Replacement Methods. 

Use the Action Plan Creator Tool to create the Service Action Plan.
Provide the Customer/End User with the summary of the action plan, including any changes to the FRU which will be replaced and/or the method in which the replacement will proceed.

Reference the Service Manual for the Platform type and FRU in question if needed:
   M3000 collection    M4000 collection    M5000 collection
   M8000 collection    M9000 collection

Confirm that the FRU replacement resolved the issue and no errors have repeated for at least 24 hours.
Confirm all resources are re-enabled and configured into the domain(s) properly.

If the same error persists and all FRUs in the list have been replaced or you are unsure of the next steps, collaborate with the next level of support for further investigation.
See Troubleshooting References for Collaboration below for details.

This solution has no attachment