Asset ID:	1-72-1011087.1
Update Date:	2010-10-14
Keywords:

---

Solution Type  Problem Resolution Sure

Solution  1011087.1 :   Unplug/plug fibre to Sun StorEdge[TM] 3510FC path sometimes fails to recover  


PreviouslyPublishedAs
215258

Symptoms
Unplug and plug fiber cable on host port directly attached to a Sun StorEdge[TM] 3510FC, the port sometimes remain "NOT CONNECTED". The path would remain unusable as if the cable has not been plugged in. Consequently multipathing software, such as STMS or VxDMP, would not be able to re-enable corresponding affected path.
The problem does not occur consistently. Lab reproduction show that most of the time the path would resume operation as per expectation, but upon multiple attempts of unplug/plug, sometimes the path did not resume.

This problem has been observed with:

- StorEdge[TM] 3510FC dual pathed, direct attached, loop mode, with firmware 3.27R
- SAN Foundation version from 4.4 through 4.4.4
- Sun Qlogic HBA and drivers

Upon replugging the fiber cable, the FC port was shown to be ONLINE by qlc driver as logged in /var/adm/messages, but "luxadm -e port" would show the port to be "NOT CONNECTED".

Resolution
Unplug/plug fibre to Sun StorEdge[TM] 3510FC path sometimes fails to recover.

CR 6329824 was pending resolution as of this writing. A simple workaround is available. The sole objective of this article is to document this workaround.

Relief/Workaround
Alternatively, the command "luxadm -e forcelip" on the affected port would also resume the path's operation.

Symptoms to look out for:

• Unplug fibre cable, /var/adm/messages would show messages similar to the following:

  qlc: [ID 686697 kern.info] NOTICE: Qlogic qlc(2): Loop OFFLINE
:
:
fctl: [ID 517869 kern.warning] WARNING: 162=>fp(2)::OFFLINE timeout
scsi: [ID 107833 kern.warning] WARNING: /pci@8,700000/SUNW,qlc@5,1/fp@0,0/ssd@w226000c0ff901ef4,0 (ssd12):
transport rejected (-2)
scsi: [ID 243001 kern.info] /pci@8,700000/SUNW,qlc@5,1/fp@0,0 (fcp2):
offlining lun=0 (trace=0), target=a6 (trace=2800004)

• If i/o is present, and depending on multipathing software in used, more messages from the multipathing driver would be logged in /var/adm/messages indicating the corresponding path has been disabled. (No sample message is shown here to avoid confusion, since those messages are merely consequences of the path being unplugged.)

• Upon replugging the fibre cable, message is logged in /var/adm/messages showing:

  qlc: [ID 686697 kern.info] NOTICE: Qlogic qlc(2): Loop ONLINE

• MOST OF THE TIME the path would resume operation where "luxadm -e port" command would indicate the port is now "CONNECTED"

  Found path to 3 HBA ports
/devices/pci@8,700000/SUNW,qlc@5/fp@0,0:devctl                     CONNECTED
/devices/pci@8,700000/SUNW,qlc@5,1/fp@0,0:devctl                   CONNECTED <---
/devices/pci@8,600000/SUNW,qlc@2/fp@0,0:devctl                     CONNECTED

• If multipathing software is used, messages from the multipathing drivers would be logged indicating the path is being re-enabled. Depending on driver, this may take as much as 5 minutes for the path to be re-probed and re-enabled.

• HOWEVER SOMETIMES "luxadm -e port" would show the corresponding port remaining "NOT CONNECTED" even though the HBA driver shows that the port has became "ONLINE".

  Found path to 3 HBA ports
/devices/pci@8,700000/SUNW,qlc@5/fp@0,0:devctl                     CONNECTED
/devices/pci@8,700000/SUNW,qlc@5,1/fp@0,0:devctl                   NOT CONNECTED <---
/devices/pci@8,600000/SUNW,qlc@2/fp@0,0:devctl                     CONNECTED

• In this case, the path will remain unusable, and as well, multipathing driver would not be able to use the path, and no further messages can be observed.

Additional Information
CR 6329824 host port to SE3510FC sometimes "NOT CONNECTED" when fibre is unplugged and replugged.


Product
Sun StorageTek SAN 4.4.4 Software
Sun StorageTek SAN 4.4.3 Software
Sun StorageTek SAN 4.4.2 Software
Sun StorageTek SAN 4.4.1 Software
Sun StorageTek SAN 4.4 Software
Sun StorageTek 3510 FC Array



Internal Comments
For internal Sun use only.

Service Request ID: 10717406

Escalation ID: 1-10227455

Solution ID: 1-12732039

See Also:  CR 6208790, CR 6260549, CR 6267754

CR6208790 "STMS path does not back to online when the cable is reconnected on SE3510" documented the problem to be
specific to the STMS. No conclusion has been made for this CR as of this writing.
CR6206907 "Luxadm shows the state as "Not Connected" in a 3510 even after the fibre is plugged back to the HBA" further explored
the same problem but investigation was stalled as Incomplete/Need More Info. The corresponding escalation has since been closed
as not reproducible.
CR6329824 "host port to SE3510FC sometimes "NOT CONNECTED" when fibre is unplugged and replugged" was filed again on the
same problem with a reproducible case, while demonstrated that vxdmp would also yield the same symptoms, therefore eliminated
multipathing drivers as part of the problem.
To avoid confusing the customer, Doc 79467 was removed from SunSolve and archived since evidences presented in the doc were
inconclusive.<
This doc is created for the sole purpose of documenting the WORKAROUND. It is not the intention of this doc to discuss the possible
causes of the bug, see the various bug reports for details of investigation.
When the true cause for the numerous bugs logged for this issue is found, a review/consolidation between Doc 79467 and this doc
should occur.
SE3510FC, minnow, SAN Foundation Software, SFS, leadville, NOT CONNECTED
Previously Published As
83021

Change History
Date: 2005-10-27
User Name: 95826
Action: Approved
Comment: - verified metadata
- changed review date to 2006-10-26
- checked for TM - added 'Sun' into 'Sun StorEdge'
- checked audience : contract
Publishing
Version: 3
Date: 2005-10-27
User Name: 95826
Action: Accept
Comment:
Version: 0

Attachments

This solution has no attachment