dropdown menu

Adapter

Connector can be PCI:

-FC HBA - Fibre Channel Host Bus Adapter
    fcsx - hardware adapter,
    fscsix - fc-scsi protocol device (logical device) (scsi protocol over the fc protocol)
     
   fcstat fcsX                 fiber channel statistics report


-SCSI HBA - SCSI Host Bus Adapter

    sisscsiaX - SCSI ADAPTER (internal disks are connected to this)

    lsdev -C | grep sisscsia   list the scsi adapters
    lsdev -Cs scsi -H          lists the scsi devices with location code

----------------------------------------
A host bus adapter is a physical Fibre Channel adapter that can be assigned to a logical partition.
----------------------------------------

fcstat:

# fcstat fcs0
Port Speed (running):   8 GBIT        <--speed of adapter
Port FC ID: 0x6df640                  <--information about the switch in hexa (first 2 digits after x will show switch id, here 6d)
Port Type: Fabric
Attention Type:   Link Up             <--link status
Topology:  Point to Point or Fabric


Port FC ID in detail:

Port FC ID: 0x6df640

six hexa digits:
1st 2 digits after x: domain id of the SAN switch, we can call "switch id" (here 6d)
2nd 2 digits after x: port ID (but could be some virtualized interpretation, here f6),
3rd 2 digits after x: loop id if in loop mode (00)

Checking "switch id", will show if ports of an FC adapter are connected to different fabrics (switches) or not. Keep in mind, that there may be more switches in a Fabric, so multiple "switch ids" are not guarantee for multiple Fabrics.

If we check a 4-port adapter, and if the first 2 hexa digits are the same, we can say that we are connected to the same switch.
fcs0: Port FC ID: 0xd1e6c0                  <--Fabric 1 (switch id: d1)
fcs1: Port FC ID: 0xd1e7c0                  <--Fabric 1 (switch id: d1)
fcs2: Port FC ID: 0x6de6c0                  <--Fabric 2 (switch id: 6d)
fcs3: Port FC ID: 0x6de7c0                  <--Fabric 2 (switch id: 6d)

----------------------------------------

ADAPTER SETTINGS:

root@aix21: / # lsattr -El fscsi0
attach       switch       How this adapter is CONNECTED         False   <-- shows actual connection protocol (None,switch,al lehet)
dyntrk       no           Dynamic Tracking of FC Devices        True    <-- dynamically follows SAN changes
fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True    <-- 3. delayed_fail or fast_fail
scsi_id      0x133300     Adapter SCSI ID                       False
sw_fc_class  3            FC Class for Fabric                   True

root@aix21: / # lsattr -El fcs0
bus_intr_lvl  305        Bus interrupt level                                False
bus_io_addr   0xffc00    Bus I/O address                                    False
bus_mem_addr  0xfffbf000 Bus memory address                                 False
init_link     al         INIT Link flags                                    True    <--al or pt2pt
intr_priority 3          Interrupt priority                                 False
lg_term_dma   0x800000   Long term DMA                                      True    <--deafault 0x200000, max. 0x8000000
max_xfer_size 0x100000   Maximum Transfer Size                              True    <--default is 100000, max 1000000
num_cmd_elems 200        Maximum number of COMMANDS to queue to the adapter True    <--default 200, max is 1024 or 2048
pref_alpa     0x1        Preferred AL_PA                                    True
sw_fc_class   2          FC Class for Fabric                                True


attach:(sets the Fiber Channel topology)
    none: if no cable has been plugged in
    al: all devices are in a loop or ring, similar to token ring networking. The failure of one device causes a break in the ring.
    switch: all devices or loops of devices are connected to Fibre Channel switches, similar conceptually to modern Ethernet implementations  

dyntrk:
    http://publib.boulder.ibm.com/infocenter/systems/index.jsp?topic=/com.ibm.aix.prftungd/doc/prftungd/fast_fail_dynamic_interaction.htm
    If it is on yes, it enables dynamic changing of fibre channel cable connections on switch ports or storage ports.
    We have changed few StoragePorts to another Switch last saturday ==> that means, that the HardwarePath changed for these servers,
    where the Option "dyntrk" is set to no. --> reconfigure your disks

fc_err_recov:
    http://publib.boulder.ibm.com/infocenter/systems/index.jsp?topic=/com.ibm.aix.prftungd/doc/prftungd/fast_fail_dynamic_interaction.htm

    delayed_fail: waits some time before acting.
    fast_fail is useful where multipathing software is used. This would support faster failover to alternate paths

init_link:
    from online help: "Do not change this attribute unless directed by IBM support."
    al: arbitrared loop : first tries al, then pt2pt, if not succes then link remains down
    pt2pt: tries pt2pt, if not succes then link remains down

lg_term_dma:
    it controls the DMA memory resource that an adapter driver can use.
    If you have a dual port adapter the maximum value is divided between the adapter ports.

max_xfer_size:
    controls the maximum transfer size. You can increase this value to improve performance.
    Different storages might need different values to utilize the performance

num_cmd_elems:
    Maximum number of commands to be queued to the adapter.
    When a large number of supported storage devices are configured, you can increase this to improve performance.

    if queue_depth=16  (hdisk or power path) and we have 10 disks, num_cmd_element should be  minimum 160
-----------------------

WWPN for an FC Adapter:

It can be obtained with "lscfg":
# lscfg -vpl fcsX | grep Netw
        Network Address.............21000025FF8338A0


Storage Admins likes in this format:
(with colons ':' and with small letters)
# lscfg -vpl fcsX | grep Netw | cut -c 37-54 | tr '[A-Z]' '[a-z]' | sed 's/../&:/g;s/:$//'
21:00:00:25:ff:83:38:a0

-----------------------

HOT PLUG adapter replacement:
0. lsslot -c pci  (check if the adapter hot pluggable)

1. paths are better in offline (C) state: dlnkmgr offline -hba 01.01

-------
it can be done with "smitty mpio" as well:
> MPIO Management > MPIO Path Management > Remove Paths  > Disable Paths under a Parent
-------

2. rmdev -Rl fcs1 <--(-R: unconfigures the device and its children, -l: specifies the logical device)

3. adapter replacement:
diag --> task selection  --> hot plug task ---> PCI Hot Plug Manager ---> Replace/Remove...

After starting it, adapter will blink.
During replacement we should leave this menu point in open,after replacement we can close this menu point in diag (enter)
(If this menu option is in open state, it will lock the adapter and configuring it is not possible, so should be closed.)

4. cfgmgr, the new device will come up with new WWN (lscfg -vpl fcsX, lsattr -El fscsiX)

5. mail to storage team, for mapping the storage to the new WWN

6. Another cfgmgr will bring up the new disks

7. put adapter to online state: dlnkmgr online -hba 01.01

------------------------

Adapter replacement with SDD dynamically (hot plug):

datapath query adapter                        <--identify the adapter to be replaced
datapath remove adapter X                     <--X is the adapter number to be removed ("select counts" shows I/O still running on other adapt.)
rmdev -dl fcsX -R                             <--remove fcs0 and all of its child devices (lsdev -Cc disk, should not show any device)
diag --> task selection  --> hot plug task ---> PCI Hot Plug Manager ---> Replace/Remove...
(from command line: lscfg -vpl fcs0| grep Adapter -> drslot -R -c pci -s P1-I8 <--P1-I8 is the slot location)
go to the ESS specialist to update the WWN of the new adapter (the zone information of the fabric switch must be updated)
cfgmgr                                        <--check with lsdev -Cc adapter, lsdev -Cc disk
addpaths                                      <--configure the new adapter and its child devices to SDD

------------------------


Adapter parameter change with SDD:

lsattr -El fcsX, lsattr -El fscsiX            <--checking the needed settings (num_cmd_elements, fc_err_recov,...)
rmdev -dl dpo -R                              <--remove SDD vpath devices
rmdev -l fcsX -R                              <--put fc adapter and its children devices to Defined state
chdev -l fcsX -a num_cmd_elems=1024
chdev -l fscsiX -a fc_err_recov=fast_fail
cfgmgr -l fcsX                                <--configiure the fc adapter, children devices and hdisks
cfallvpath                                    <--configure SDD vpath devices (if they were removed)

------------------------

PCIDMA (DMA) error:

If there are too many FC adapters and too many LUNs attached to the adapter, this will lead to issues configuring the LUNs. Errors will look like:

LABEL: DMA_ERR
IDENTIFIER: 00530EA6

Resource Name: PCIDMA
Resource Class: NONE
Resource Type: NONE

So if you get these errors, you'll need to change the max_xfer_size back to the default value. Also note that if you are booting from SAN, if you encounter this error, you won't be able to boot, so be sure to have a back out plan if you plan to change this and are booting from SAN.

------------------------

31 comments:

  1. This is a terrific page! Very helpful, thank you!

    ReplyDelete
  2. Well done. Thanks

    ReplyDelete
  3. very good link please continue the with latest updates

    ReplyDelete
  4. OK...and I'm happy I could help you :)

    ReplyDelete
  5. for a newbie AIX system admin, your blog is such a relief, helping specially in Sev1s.. :)

    ReplyDelete
  6. Hey there! Someone in my Myspace group shared this site with us
    so I came to check it out. I'm definitely enjoying the information. I'm book-marking and will be tweeting this to my followers! Exceptional blog and fantastic style and design.
    My page: Central London

    ReplyDelete
    Replies
    1. Thanks! I am appreciating your contribution to this blog!
      (When I travel to London and I need a personal trainer, I know where to go :))

      Delete
    2. Very good explantion. Thanks for the post.

      Delete
  7. AWESOME !

    This site itself is having bunch of knowledge !!!

    Thanks a LOT for preserving/upating this.

    bookmarked this on ALL browsers I'm using :)

    Santosh Bhalerao

    ReplyDelete
  8. This is really very helpful.. Thanks a lot.

    ReplyDelete
  9. what is the fiber port replacement process in aix p740 server??

    ReplyDelete
    Replies
    1. it is written above at adapter replacement section...

      Delete
  10. Hi,

    How i can change init_link if i can see "false" at the end of the line ?

    Many thanks

    ReplyDelete
    Replies
    1. Hi, if it is false, it means user cannot change that line. I would get in contact with IBM in this case.

      Delete
  11. Hi
    How to check the Multipath status?how to check whether multipath software is installed or not?Please explain me some EMC powerpath important commands.if the hdisk is coming from SAN,How to check its path,Multipath status,Lun ID.

    Sorry for asking so many Q;)

    ReplyDelete
    Replies
    1. Description about EMC can be found here: http://aix4admins.blogspot.hu/2011/05/emc-emc-comes-with-default-settings.html

      Delete
  12. Nice post! Didn't know about the lsslot. Really love the detail here

    I didn't see "lsdev -Cc adapter" explicitly mentioned above.. Just to add, I like this just for seeing on the spot if I have a dual port or single port

    e.g.
    bash-3.2# lsdev -Cc adapter
    fcs0 Available 03-00 4Gb FC PCI Express Adapter (df1000fe)
    fcs1 Available 03-01 4Gb FC PCI Express Adapter (df1000fe)
    fcs2 Available 04-00 4Gb FC PCI Express Adapter (df1000fe)
    fcs3 Available 04-01 4Gb FC PCI Express Adapter (df1000fe)
    fcs4 Available 06-00 4Gb FC PCI Express Adapter (df1000fe)
    fcs5 Available 06-01 4Gb FC PCI Express Adapter (df1000fe)
    bash-3.2#

    I have three dual ports

    Just wanted to add!

    ReplyDelete
  13. Replies
    1. Can you answer me about Adapter in AIX? what's the difference between PCI, IDE and SCSI in AIX? I have search on the internet but I can't understand?

      Please .. ! Thanks

      Delete
  14. You saved my life :)

    ReplyDelete
  15. could you please help on one of my common problem is. scenario is: my /tmp is 2% now. but after one minute i get alert /tmp is 100% full. but when i see top usage files. I donot find any big files.. then i use #fuser -kuxc /tmp and sudden i find that again /tmp is 2 %.. please suggest me . how can i find a process which reserves all the /tmp space.

    ReplyDelete
  16. Hi, thanks for the very useful link.
    Have a question:
    There are options we can use for fcstat (-d; -e; -z etc).

    http://www-01.ibm.com/support/knowledgecenter/ssw_aix_71/com.ibm.aix.cmds2/fcstat.htm

    However, the fcstat -d did not work.
    Anyone can shred a light?

    ReplyDelete
  17. Hi ... I think the answer is yes to these 2 questions that I have but I need to verify please. Would appreciate.

    1) Can I reboot my HMC without interuptions to my attached host?
    2) Can I upgrade my HMC without interruptions to my attached host?

    Thank you.

    ReplyDelete
    Replies
    1. Yes, you can do both of them without interruption in the service on the running LPARs.

      Delete
  18. I really self learned AIX stuff with your blogs .. Its awesome work by you ....!

    ReplyDelete