On Thu, Sep 05, 2019 at 09:40:20PM -0400, Daniel Jordan wrote: > v2 -> v3: > - Rebase onto cryptodev. > > v1 -> v2: > - Updated patch 8 to avoid queueing the reorder work if the next object > by sequence number isn't ready yet (Herbert) > - Added Steffen's ack to all but patch 8 since that one changed. > > RFC -> v1: > - Included Tejun's acks. > - Added testing section to cover letter. > > Padata binds the parallel part of a job to a single CPU and round-robins > over all CPUs in the system for each successive job. Though the serial > parts rely on per-CPU queues for correct ordering, they're not necessary > for parallel work, and it improves performance to run the job locally on > NUMA machines and let the scheduler pick the CPU within a node on a busy > system. > > This series makes parallel padata jobs run on unbound workqueues. > > Patch Description > ----- ----------- > > 1 Make a padata instance allocate its workqueue internally. > > 2 Unconfine some recently-confined workqueue interfaces. > > 3-6 Address recursive CPU hotplug locking issue. > > padata_alloc* requires its callers to hold this lock, but allocating > an unbound workqueue and calling apply_workqueue_attrs also take it. > Fix by removing the requirement for callers of padata_alloc*. > > 7-8 Add a second workqueue for each padata instance that's dedicated to > parallel jobs. > > 9 Small cleanup. > > Performance > ----------- > > Measurements are from a 2-socket, 20-core, 40-CPU Xeon server. > > For repeatability, modprobe was bound to a CPU and the serial cpumasks > for both pencrypt and pdecrypt were also restricted to a CPU different > from modprobe's. > > # modprobe tcrypt alg="pcrypt(rfc4106(gcm(aes)))" type=3 > # modprobe tcrypt mode=211 sec=1 > # modprobe tcrypt mode=215 sec=1 > > Busy system (tcrypt run while 10 stress-ng tasks were burning 100% CPU) > > base test > ---------------- --------------- > speedup key_sz blk_sz ops/sec stdev ops/sec stdev > > (pcrypt(rfc4106-gcm-aesni)) encryption (tcrypt mode=211) > > 117.2x 160 16 960 30 112555 24775 > 135.1x 160 64 845 246 114145 25124 > 113.2x 160 256 993 17 112395 24714 > 111.3x 160 512 1000 0 111252 23755 > 110.0x 160 1024 983 16 108153 22374 > 104.2x 160 2048 985 22 102563 20530 > 98.5x 160 4096 998 3 98346 18777 > 86.2x 160 8192 1000 0 86173 14480 > > (pcrypt(rfc4106-gcm-aesni)) decryption (tcrypt mode=211) > > 127.2x 160 16 997 5 126834 24244 > 128.4x 160 64 1000 0 128438 23261 > 127.6x 160 256 992 7 126627 23493 > 124.0x 160 512 1000 0 123958 22746 > 122.8x 160 1024 989 20 121372 22632 > 112.8x 160 2048 998 3 112602 18287 > 106.9x 160 4096 994 10 106255 16111 > 91.7x 160 8192 1000 0 91742 11670 > > multibuffer (pcrypt(rfc4106-gcm-aesni)) encryption (tcrypt mode=215) > > 242.2x 160 16 2363 141 572189 16846 > 242.1x 160 64 2397 151 580424 11923 > 231.1x 160 256 2472 21 571387 16364 > 237.6x 160 512 2429 24 577264 8692 > 238.3x 160 1024 2384 97 568155 6621 > 216.3x 160 2048 2453 74 530627 3480 > 209.2x 160 4096 2381 206 498192 19177 > 176.5x 160 8192 2323 157 410013 9903 > > multibuffer (pcrypt(rfc4106-gcm-aesni)) decryption (tcrypt mode=215) > > 220.3x 160 16 2341 228 515733 91317 > 216.6x 160 64 2467 33 534381 101262 > 217.7x 160 256 2451 45 533443 85418 > 213.8x 160 512 2485 26 531293 83767 > 211.0x 160 1024 2472 28 521677 80339 > 200.8x 160 2048 2459 67 493808 63587 > 188.8x 160 4096 2491 9 470325 58055 > 159.9x 160 8192 2459 51 393147 25756 > > Idle system (tcrypt run by itself) > > base test > ---------------- --------------- > speedup key_sz blk_sz ops/sec stdev ops/sec stdev > > (pcrypt(rfc4106-gcm-aesni)) encryption (tcrypt mode=211) > > 2.5x 160 16 63412 43075 161615 1034 > 4.1x 160 64 39554 24006 161653 981 > 6.0x 160 256 26504 1436 160110 1158 > 6.2x 160 512 25500 40 157018 951 > 5.9x 160 1024 25777 1094 151852 915 > 5.8x 160 2048 24653 218 143756 508 > 5.6x 160 4096 24333 20 136752 548 > 5.0x 160 8192 23310 15 117660 481 > > (pcrypt(rfc4106-gcm-aesni)) decryption (tcrypt mode=211) > > 2.4x 160 16 53471 48279 128047 31328 > 3.4x 160 64 37712 20855 128187 31074 > 4.5x 160 256 27911 4378 126430 31084 > 4.9x 160 512 25346 175 123870 29099 > 3.1x 160 1024 38452 23118 120817 26846 > 4.7x 160 2048 24612 187 115036 23942 > 4.5x 160 4096 24217 114 109583 21559 > 4.2x 160 8192 23144 108 96850 16686 > > multibuffer (pcrypt(rfc4106-gcm-aesni)) encryption (tcrypt mode=215) > > 1.0x 160 16 412157 3855 426973 1591 > 1.0x 160 64 412600 4410 431920 4224 > 1.1x 160 256 410352 3254 453691 17831 > 1.2x 160 512 406293 4948 473491 39818 > 1.2x 160 1024 395123 7804 478539 27660 > 1.2x 160 2048 385144 7601 453720 17579 > 1.2x 160 4096 371989 3631 449923 15331 > 1.2x 160 8192 346723 1617 399824 18559 > > multibuffer (pcrypt(rfc4106-gcm-aesni)) decryption (tcrypt mode=215) > > 1.1x 160 16 407317 1487 452619 14404 > 1.1x 160 64 411821 4261 464059 23541 > 1.2x 160 256 408941 4945 477483 36576 > 1.2x 160 512 406451 611 472661 11038 > 1.2x 160 1024 394813 2667 456357 11452 > 1.2x 160 2048 390291 4175 448928 8957 > 1.2x 160 4096 371904 1068 449344 14225 > 1.2x 160 8192 344227 1973 404397 19540 > > Testing > ------- > > In addition to the bare metal performance runs above, this series was > tested in a kvm guest with the tcrypt module (mode=215). All > combinations of CPUs among parallel_cpumask, serial_cpumask, and CPU > hotplug online/offline were run with 3 possible CPUs, and over 2000 > random combinations of these were run with 8 possible CPUs. Workqueue > events were used throughout to verify that all parallel and serial > workers executed on only the CPUs allowed by the cpumask sysfs files. > > Finally, tcrypt mode=215 was run at each patch in the series when built > with and without CONFIG_PADATA/CONFIG_CRYPTO_PCRYPT. > > v2: https://lore.kernel.org/linux-crypto/20190829173038.21040-1-daniel.m.jordan@xxxxxxxxxx/ > v1: https://lore.kernel.org/linux-crypto/20190813005224.30779-1-daniel.m.jordan@xxxxxxxxxx/ > RFC: https://lore.kernel.org/lkml/20190725212505.15055-1-daniel.m.jordan@xxxxxxxxxx/ > > Daniel Jordan (9): > padata: allocate workqueue internally > workqueue: unconfine alloc/apply/free_workqueue_attrs() > workqueue: require CPU hotplug read exclusion for > apply_workqueue_attrs > padata: make padata_do_parallel find alternate callback CPU > pcrypt: remove padata cpumask notifier > padata, pcrypt: take CPU hotplug lock internally in > padata_alloc_possible > padata: use separate workqueues for parallel and serial work > padata: unbind parallel jobs from specific CPUs > padata: remove cpu_index from the parallel_queue > > Documentation/padata.txt | 12 +-- > crypto/pcrypt.c | 167 ++++--------------------------- > include/linux/padata.h | 16 +-- > include/linux/workqueue.h | 4 + > kernel/padata.c | 201 ++++++++++++++++++++++---------------- > kernel/workqueue.c | 25 +++-- > 6 files changed, 170 insertions(+), 255 deletions(-) All applied. Thanks. -- Email: Herbert Xu <herbert@xxxxxxxxxxxxxxxxxxx> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
