The fpin options that are developed do have a whole plethora of options and do not mainly trigger paths being in a marginal state. Th mpio layer could utilise the various triggers like congestion and latency and not just use a marginal state as a decisive point. If a path is somewhat congested the amount of io's dispersed over these paths could just be reduced by a flexible margin depending on how often and which fpins are actually received. If for instance and fpin is recieved that an upstream port is throwing physical errors you may exclude is entirely from queueing IO's to it. If it is a latency related problem where credit shortages come in play you may just need to queue very small IO's to it. The scsi CDB will tell the size of the IO. Congestion notifications may just be used for potentially adding an artificial delay to reduce the workload on these paths and schedule them on another.
Not really sure what the possibilities are from a DM-Multipath viewpoint, but I feel if the OS options are not properly aligned with what the FC protocol and HBA drivers are able to provide we may miss a good opportunity to optimize the dispersion of IO's and improve overall performance.
Hi Benjamin,
My replies are below
On Tue, Mar 23, 2021 at 05:52:33PM +1000, Erwin van Londen wrote:
Hello All,
Just wondering if there were any plans to incorporate FPIN
congestion/latency notifications in dm-multipath to disperse IO over
non-affected paths.
For whats worth, general support in Kernel for a new path state in answer
to existing FPIN notifications was added earlier this year:
But this only adds a new port-state and support of it for one particular
driver (lpfc). Not aware of any other driver supporting this new state
yet, but I might have missed it. Also, the port-state is not set in
kernel, but has to be set by something external, unlike with RSCNs, where
we set the >state in the kernel.
We had a discussion with Marvel and they are adding the support in
their(qlaxx) driver.
What it does, once a path is set into 'Marginal' state, is to not retry
commands on the same shaky path, once it already failed one time already.
Yes
As far as dm-multipath is concerned, I asked that as well when this patch
series was developed:
Hannes answered that in the thread:
use.de/
Not sure what happened in between, didn't see anything on the mpath topic
yet.
As Hannes mentioned in his reply we have an external daemon called fctxpd
which acts on fpin-li events and sets the path to marginal path group as
well as set the port state to marginal.
This daemon is part of epel8.
Below is the path for the same where we have changes
The above code is reviewed by the Benjamin Marzinski from redhat .
Note:The latest release will be available on the epel8 where we have the
support to set the port state to marginal in a week time
As we have all the support in the kernel for fpin registration,
notifications and also setting the port_state to marginal
We had a initial discussion with Hannes adding the fpin based native
support in dm multipathd for FPIN Congestion/Latency notifications .
I will take the initiative and start the discussion with Benjamin
Marzinski and get this work done with the help of Hannes.
Regards,
Muneendra.
