Mike Christie wrote:
There is still memory and scatterlist allocations. If we are not
going to allocate all the memory for a command buffer and request
with GFP_ATOMIC (and can then run from the the HW interrupt or soft
irq) we have to pass that on to a thread. I guess there is
disagreement whether that part is a feature or bad use of GFP_ATOMIC
though so... But I just mean to say there could be a little more to do.
Actually, there is the way to allocate sg vectors with buffers in SIRQ
and not with GFP_ATOMIC. This is the second major improvement, which
is pending in scst. I called it sgv_pool. This is a new allocator in
the kernel similar to mem_pool, but it contains *complete* sg-vectors
of some size with data buffers (pages). Initiator sends data requests
usually with some fixed size, like 128K. After a data command
completed, its sg vector will not be immediately freed, but will be
kept in
We considered this, but what did you decide is the upper limit size for
the pool? Is it dynmaic? We also wanted something that the SCSI ULDs
could use for their allocations which could go up to 6 MB.
Why do you think it needs any upper limit size? Would you like also any
upper limits on sizes of the page or slab caches? I don't see any
difference between them and sgv_pool. Let it allocate and keep unused as
much memory as possible until it would be notified about memory pressure
and ask to free some. Similarly as slab does.
BTW, from our experience, initiators tend to send commands without limit
and in case if actual SCSI device isn't so fast to serve all of them
with the rate, which they arrive, incoming queued commands very quickly
eat all system memory with obvious consequences. So, there must be some
kind of IO-throttling, when after some watermark the initiators start
receiving TASK QUEUE FULL, which calm them down. I implemented very
stupid one: don't let more than 32 commands per session. This let scst
survive under extreme loads. You can think about something smarter :-)
Vlad
-
: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
