Sun Microsystems, Inc.  Sun System Handbook - ISO 3.4 June 2011 Internal/Partner Edition
   Home | Current Systems | Former STK Products | EOL Systems | Components | General Info | Search | Feedback

Asset ID: 1-71-1003310.1
Update Date:2010-06-07
Keywords:

Solution Type  Technical Instruction Sure

Solution  1003310.1 :   KERNEL:How to: E-cache scrubber  


Related Items
  • Sun Enterprise 4500 Server
  •  
  • Sun Enterprise 5500 Server
  •  
  • Sun Enterprise 450 Server
  •  
  • Sun Ultra 450 Workstation
  •  
  • Sun Ultra 80 Workstation
  •  
  • Sun Enterprise 3500 Server
  •  
  • Sun Enterprise 6500 Server
  •  
  • Sun Enterprise 220R Server
  •  
  • Sun Enterprise 250 Server
  •  
  • Solaris SPARC Operating System
  •  
  • Sun Ultra 60 Workstation
  •  
  • Sun Enterprise 420R Server
  •  
Related Categories
  • GCS>Sun Microsystems>Desktops>Workstations
  •  

PreviouslyPublishedAs
204591


Description
Many Ultra Sparc II (USII) processors need certain patches and kernel parameters set to avoid unnecessary system panics and HW replacements.


Steps to Follow
Many older USII systems require an e-cache scrubber to avoid certain kinds of panics associated with e-cache parity errors.

This does not eliminate panics of this nature but it does dramatically reduce their likelihood. Affected systems include all volume class servers and desktop systems with USII processors. Early models of the Enterprise (E) class systems are also affected unless they contain mirrored cache modules. If you are unsure if your E class server has mirrored e-cache modules, please contact Sun Services and reference this InfoDoc.

The first step is to make sure you have the minimum kernel patch installed on your system so that the scrubber is available:

Solaris Version

Minimum Patch Requirement

2.5.1

103640-34

2.6

105181-23

7

106541-13

8

108528-04

All versions of Solaris more recent than 8 have the scrubber already installed.

Next, to maximize the effectiveness of the scrubber the following lines need to be added to /etc/system:

set ecache_scrub_enable=1
set ecache_scan_rate=1000
set ecache_calls_a_sec=100

These settings will have a marginal performance impact (less then 1% on CPUs which run at 400mhz or more).



Product
Solaris
Sun Enterprise 6500 Server
Sun Enterprise 5500 Server
Sun Enterprise 4500 Server
Sun Enterprise 450 Server
Sun Enterprise 3500 Server
Sun Enterprise 420R Server
Sun Enterprise 220R Server
Sun Enterprise 250 Server
Ultra 80 Workstation
Ultra 60 Workstation
Ultra 450 Workstation

Internal Comments
The E-cache scrubber has two roles.

1. The E-cache scrubber avoids lines
being resident in cache long enough to "decay" leading to parity
errors.


2. The E-cache scrubber tries to find and displace bad clean
lines before something else does ,avoiding E-cache errors that
could lead to a system downtime.


The E-cache scrubber wakes up and inspects a number of cache
lines 100 times a second on each CPU. It inspects the entire cache
in 10 seconds.


Upon line tag inspection (Modified), parity (AFSRs) and
idle/busy flag, the scrubber decides whether to flush the cache
line. The cache line is flushed if the tag is invalid. If the cache
line is not in use, it is skipped.
















































Modify



Parity



Busy



Idle



Clean



Good



*



Flush



Clean



Bad



Flush



Flush



Dirty



Good



*



Flush



Dirty



Bad



-



-




* On mirrored E$ cpu modules, we additionally flush clean_good_busy and dirty_good_busy lines.

Modify : EC_state indication of clean/dirty (non-modified/modified)
Parity : Parity error found in E$ data
Idle : CPU currently running idle thread

If flushing, a physical displacement flush using PA (Physical
Address), constructed from tag, is performed.


If ecache_scrub_verbose is set ( the default on Solaris 8
108528-16 or above ), the kernel logs the E-cache lines of the
following states:


clean_bad_idle : CBI event


clean_bad_busy : CBB event


dirty_bad_idle and : DBI event


dirty_bad_busy : DBB event


One can use "kstat -n ecache_kstat"
to confirm the scrubber is active.


If ecache_scrub_panic is set, the kernel panics in addition to
logging (not recommended)


Standard E-Cache Best Practices apply to CBI CBB DBI and DBB
events - if only a single error is observered the CPU should not be
replaced ( unless it has mirrored e-cache ). See <Document: 1009200.1> for further details.


Note: In the case of DBI and DBB events the bad data cannot be
flushed from the cache, and as a result DBI/DBB events are often
immediately followed by a WP/EDP/CP event as a thread attempts to
actually use the bad data. In those cases the DBI/DBB and WP/EDP/CP
events should be considered as a single E-Cache event.

See < Solution: 201640 > for additional details regarding the scrubber.


See <Document: 1018753.1> to identify mirrored e-cache
modules.


E-cache, scrubber, clean, dirty, bad, busy, idle, CBI, CBB, DBI, DBB, event, USII, Ultra Sparc II
Previously Published As
47385

Change History
Date: 2007-08-13
User Name: 7058
Action: Approved
Comment: Updates OK to publish
Version: 6
Date: 2007-08-10
User Name: 7058
Action: Accept
Comment:
Version: 0

Date: 2007-08-10
User Name: 26074
Action: Approved
Comment: The changes are good.
Version: 0
Product_uuid
3285bfa4-224e-11d6-8eb3-843d3a923213|Solaris
29ccbffa-0a18-11d6-944e-9f2643ef1cd7|Sun Enterprise 6500 Server
29c606ce-0a18-11d6-98c8-de563f7f1f97|Sun Enterprise 5500 Server
29bdb91a-0a18-11d6-9d2c-c8c4d893e741|Sun Enterprise 4500 Server
29656238-0a18-11d6-91d9-a4e449afd809|Sun Enterprise 450 Server
29b720d2-0a18-11d6-9865-e60ab6457390|Sun Enterprise 3500 Server
296beb4e-0a18-11d6-8ee5-bfb7a4eb3177|Sun Enterprise 420R Server
2968a092-0a18-11d6-8c14-d5add976b103|Sun Enterprise 220R Server
2962179a-0a18-11d6-845c-9c8399e8b5f2|Sun Enterprise 250 Server
29ff195a-0a18-11d6-8369-dc30c95555de|Ultra 80 Workstation
29fbc43a-0a18-11d6-8111-b340515b0ffc|Ultra 60 Workstation
2a025a3e-0a18-11d6-9e8e-9fc98998d49d|Ultra 450 Workstation

Attachments
This solution has no attachment
  Copyright © 2011 Sun Microsystems, Inc.  All rights reserved.
 Feedback