---------------------------------------------------------------------
- Sun Proprietary/Confidential: Internal Use Only -
---------------------------------------------------------------------  
                            FIELD INFORMATION NOTICE
                  (For Authorized Distribution by SunService)

FIN #: I0541-2

Create Date: Aug/02/00

Keywords: 

Restrictions on multi-initiator config using FC100/P card on A5x00

SunAlert: No

Top FIN/FCO Report: No

Products Reference: Multi hosted PCI FCAL failures

Product Category: Storage / Service

Product Affected: 

Mkt_ID   Platform   Model   Description          Serial Number
------   --------   -----   -----------          -------------
Systems Affected
----------------
   -       A20       ALL    Ultra 450                  -
   -       A25       ALL    Ultra Enterprise 450       -
   -       A26       ALL    Enterprise 250             -


NOTE: All Sun SPARC platforms using PCI FC100/P card.  This may
      include any Sun SPARC platform fitted with PCI expansion bus.

X-Options Affected
------------------
X6729A         -      -    FC-AL PCI Card (FC100/P)                   -
X2652A       E3500    -    FC-AL Interface Board                      -
X6731A         -      -    FC-AL GBIC Module 100MB/s                  -
SG-XARY51%   A5000    -    Sun StorEdge A5000 14-Drive FC-AL Arrays   -
SG-XARY01%   A5000    -    Sun StorEdge A5000 14-Drive FC-AL Arrays   -
SG-XARY53%   A5100    -    Sun StorEdge A5100 14-Drive FC-AL Arrays   -
SG-XARY52%   A5200    -    Sun StorEdge A5200 22-Drive FC-AL Arrays   -

Parts Affected: 

Part Number   Description       Model
-----------   -----------       -----
375-0040-04   FC-AL PCI Card      -

References: 

BugId:  4243157 - Multi-hosted PCI fcal A5000 breaks mirrors when one 
                  node is rebooted.
ESC:    520843 
        523486

Issue Description: 

-----------------
|FROM FIN I0541-1:|
 -----------------
A problem may occur with FCAL loop comprising A5x00 storage array and
at least two (2) host connections to the same A5x00 (i.e.
Multi-initiator) where one of the host connections is through the PCI
FC100/P initiator card and no intervening hub or switch is involved,
and where both host-connections are on the same loop.

The symptoms appear when the PCI FC100/P card equipped host is either
halted, powered down, or during reboot.
	
During the reboot of the PCI FC100/P equipped host, the other host with
FCAL initiator-card on the loop may experience the loop going offline
permanently, mirrors or metadisks being broken.  Messages files may be
filled with LIP messages.

Sample error messages:

Dec  8 05:18:17 name unix: transport rejected (-2)
Dec  8 05:18:17 name unix:
Dec  8 05:18:17 name unix: WARNING: /pci@4,4000/SUNW,
                           ifp@4/ssd@w2100002037141659,0 (ssd32)
Dec  8 05:18:17 name unix: transport rejected (-2)
Dec  8 05:18:17 name unix:
Dec  8 05:18:18 name unix: ifp0:    LIP occured
Dec  8 05:18:18 name unix: ifp0:    Loop up
Dec  8 05:18:18 name unix: ifp0:    Loop reconfigure in progress
Dec  8 05:18:18 name unix: ifp0:    Loop reconfigure done
Dec  8 08:17:41 name unix: ifp0:    Loop down
Dec  8 08:17:46 name unix: ifp0:    LIP reset occured
Dec  8 08:17:46 name last message repeated 1 time
Dec  8 08:17:46 name unix: ifp0:    LIP occured
Dec  8 08:17:46 name unix: ifp0:    Loop up
Dec  8 08:17:46 name unix: ifp0:    Loop reconfigure in progress
Dec  8 08:17:46 name unix: ifp0:    Loop reconfigure done
Dec  8 08:18:23 name unix: ifp0:    LIP reset occured
Dec  8 08:18:23 name unix: ifp0:    LIP occured
Dec  8 08:18:24 name unix: ifp0:    LIP reset occured
Dec  8 08:18:24 name unix: ifp0:    LIP occured
Dec  8 08:18:24 name unix: ifp0:    Loop reconfigure in progress
Dec  8 08:18:24 name unix: ifp0:    Loop reconfigure done

NOTE: The A5X00 Front-Panel-Module may display the loop-connection
     "lightning bolt" icon as having a flickering or unsteady status.

NOTE: The A5X00 Front-Panel-Module's "node-count" display, may show
      0 (zero) nodes on the affected loop, during the bootup.

The root-cause of this issue is an incompatibility between the PCI
FC100/P card, and the IB-card on the A5X00 Storage Array.  The FC100/P 
card, and the IB-card, have conflicting concept of "loop up" status.
This may cause the IB-card to conclude that a loop connection to the 
FC100/P card is up and in healthy state, when in fact the FC100/P card 
is down or being rebooted or firmware reloaded and not passing on
Fibre-Channel packets across the network.

All currently shipping FC100/P cards, and all currently shipping
StorEdge A5X00 IB-cards, are affected.

 -----------------------
|UPDATE FOR FIN I0541-2:|
 -----------------------

The Corrective Action on FIN I0541-1 provides two different options 
or solutions to the above mentioned problems as follows:

      OPTION-1: The FC100/P card is the only Initiator on any one loop       
                where Sun A5x00 Storage Array are direct connected.

      OPTION-2: A Sun FCAL hub is placed between the FC100/P card(s), 
                and the A5X00 Storage Array.  In this case, more than 
                one host-adapter on the same loop, may be used. 

.  Due to resistance from the Field over the cost of implementation,
   OPTION-2 has been removed from the Corrective Action.  This option
   would have required the customer to purchase and connect a hub to
   the A5x00 Storage Array.  The Corrective Action now recommends only 
   OPTION-1 for avoiding this FC100/P card problem.

This restriction is advantageous from high-availability/RAS point of
view since having each host on a separate loop removes a single point
Of failure (SPOF) in Multi Initiator configurations.

Implementation: 

---
        |   |   MANDATORY (Fully Pro-Active)
         ---


         ---
        |   |   CONTROLLED PRO-ACTIVE (per Sun Geo Plan)
         ---


         ---
        | X |   REACTIVE (As Required)
         ---

Corrective Action: 

Enterprise Customers and Authorized Enterprise Field Service
Representatives that use or install multi-initiator configurations as
described in BugId 4243157, should note the following recommendations
in order to avoid the encountering above mentioned problem:

A new restriction is placed on the use of Sun PCI FC100/P
FCAL-initiator host adapter card (option 6729A) with Sun A5X00 Storage
Array (see option list above) in multi-initiator configurations where
no hub or switch is used.    

EXAMPLE:

  If an FC100/P card is used on the A-loop of a particular direct-connect
  A5x00 StorEdge Array, then another host and/or initiator card may
  be connected to the B-loop only of that A5x00 StorEdge Array - the
  second host may not share the A-loop with the FC100/P card.

Comments: 

None

--------------------------------------------------------------------------

Implementation Footnote: 

i)   In case of MANDATORY FINs, Enterprise Services will attempt to
     contact all affected customers to recommend implementation of
     the FIN.

ii)  For CONTROLLED PROACTIVE FINs, Enterprise Services mission critical
     support teams will recommend implementation of the FIN  (to their
     respective accounts), at the convenience of the customer.

iii) For REACTIVE FINs, Enterprise Services will implement the FIN as the
     need arises.
----------------------------------------------------------------------------

All released FINs and FCOs can be accessed using your favorite network
browser as follows:

SunWeb Access:
--------------
* Access the top level URL of http://sdpsweb.ebay/FIN_FCO/

* From there, select the appropriate link to query or browse the FIN and
  FCO Homepage collections.

SunSolve Online Access:
-----------------------
* Access the SunSolve Online URL at http://sunsolve.Corp/

* From there, select the appropriate link to browse the FIN or FCO index.

Supporting Documents:
---------------------
* Supporting documents for FIN/FCOs can be found on Edist.  Edist can be
  accessed internally at the following URL: http://edist.corp/.

* From there, follow the hyperlink path of "Enterprise Services Documenta-
  tion" and click on "FIN & FCO attachments", then choose the appropriate
  folder, FIN or FCO.  This will display supporting directories/files for
  FINs or FCOs.

Internet Access:
----------------
* Access the top level URL of https://infoserver.Sun.COM
--------------------------------------------------------------------------
General:
--------
* Send questions or comments to [email protected]
--------------------------------------------------------------------------