On 08/12/2014 04:52 PM, Nicholas A. Bellinger wrote:
So I've been thinking about this recently, and wondering what use cases could end up being problematic with a 'BCL' approach, especially if done below the targetcli (end-user) level that end up preventing the inherent parallelism provided by existing /sys/kernel/config/target/* logic. One case that comes to mind is where a large number (say 256) of backends + fabric endpoints + LUNs are failing over from one physical host to another. Having to sequentially preform these options under the guise of a single BCL can significantly slow down the total time to recreate backends + fabric endpoints + LUNs, potentially resulting in forward facing I/O timeouts and other types of unpleasantness.
I don't follow - if userspace is doing a large want-to-be atomic transition like this, wouldn't grabbing the BCL allow it to do its thing without disruption? Every time a finer-grained lock is released is a chance for another entity to get in and change things unexpectedly, so to be safe the initial entity would have to re-read configfs under the new lock instead of carrying knowledge of the state across the entire transaction.
This can be compounded by the amount of time it takes for hardware to re-initialize when transitioning into target mode. One example that comes to mind is with qla2xxx, where enabling target mode ends up taking 5-10 seconds per WWPN.
This isn't the best example, because afaik qla2xxx must be set into target mode at boot. As a counterexample I would say in general configfs doesn't wait for hardware, it allows the setting to be set and doesn't block. iscsi lets you configure portals for IPs that have not been configured, and tcm_fc also lets you configure WWPNs before they are present, as examples.
So that said, I think there is some value in a BCL approach at the targetcli level to help avoid simple beginner mistakes for new users, but such an approach at rtslib level really limits what the kernel design was intended to do; Allowing for many outstanding operations to be executed in parallel from different processes within separate configfs group contexts. So please, let's avoid putting training wheels on rtslib that limit the underlying parallelism already built into target/configfs.
Taking a step back, I think we all agree that contention will be extremely low -- this is really about a safeguard.
I think there's still room for an entity grabbing the lock to parallelize its own configfs accesses while holding the BCL.
Given that, and that we'd want to evangelize this to other people writing LIO tools in different languages, simplicity is key. A single BCL enables easier parallelism within an accessing entity, and the downside of not enabling cross-entity parallelism is moot if contention is low.
BTW we also could prototype some different designs and try them out, to help reach a consensus, too.
Regards -- Andy -- To unsubscribe from this list: send the line "unsubscribe target-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
