This is a note to let you know that I've just added the patch titled random: do not use input pool from hard IRQs to the 4.14-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary The filename of the patch is: random-do-not-use-input-pool-from-hard-irqs.patch and it can be found in the queue-4.14 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable@xxxxxxxxxxxxxxx> know about it. >From foo@baz Thu Jun 16 07:08:33 PM CEST 2022 From: "Jason A. Donenfeld" <Jason@xxxxxxxxx> Date: Fri, 6 May 2022 18:30:51 +0200 Subject: random: do not use input pool from hard IRQs From: "Jason A. Donenfeld" <Jason@xxxxxxxxx> commit e3e33fc2ea7fcefd0d761db9d6219f83b4248f5c upstream. Years ago, a separate fast pool was added for interrupts, so that the cost associated with taking the input pool spinlocks and mixing into it would be avoided in places where latency is critical. However, one oversight was that add_input_randomness() and add_disk_randomness() still sometimes are called directly from the interrupt handler, rather than being deferred to a thread. This means that some unlucky interrupts will be caught doing a blake2s_compress() call and potentially spinning on input_pool.lock, which can also be taken by unprivileged users by writing into /dev/urandom. In order to fix this, add_timer_randomness() now checks whether it is being called from a hard IRQ and if so, just mixes into the per-cpu IRQ fast pool using fast_mix(), which is much faster and can be done lock-free. A nice consequence of this, as well, is that it means hard IRQ context FPU support is likely no longer useful. The entropy estimation algorithm used by add_timer_randomness() is also somewhat different than the one used for add_interrupt_randomness(). The former looks at deltas of deltas of deltas, while the latter just waits for 64 interrupts for one bit or for one second since the last bit. In order to bridge these, and since add_interrupt_randomness() runs after an add_timer_randomness() that's called from hard IRQ, we add to the fast pool credit the related amount, and then subtract one to account for add_interrupt_randomness()'s contribution. A downside of this, however, is that the num argument is potentially attacker controlled, which puts a bit more pressure on the fast_mix() sponge to do more than it's really intended to do. As a mitigating factor, the first 96 bits of input aren't attacker controlled (a cycle counter followed by zeros), which means it's essentially two rounds of siphash rather than one, which is somewhat better. It's also not that much different from add_interrupt_randomness()'s use of the irq stack instruction pointer register. Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx> Cc: Filipe Manana <fdmanana@xxxxxxxx> Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx> Cc: Borislav Petkov <bp@xxxxxxxxx> Cc: Theodore Ts'o <tytso@xxxxxxx> Signed-off-by: Jason A. Donenfeld <Jason@xxxxxxxxx> Signed-off-by: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx> --- drivers/char/random.c | 51 +++++++++++++++++++++++++++++++++++--------------- 1 file changed, 36 insertions(+), 15 deletions(-) --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -1081,6 +1081,7 @@ static void mix_interrupt_randomness(str * we don't wind up "losing" some. */ unsigned long pool[2]; + unsigned int count; /* Check to see if we're running on the wrong CPU due to hotplug. */ local_irq_disable(); @@ -1094,12 +1095,13 @@ static void mix_interrupt_randomness(str * consistent view, before we reenable irqs again. */ memcpy(pool, fast_pool->pool, sizeof(pool)); + count = fast_pool->count; fast_pool->count = 0; fast_pool->last = jiffies; local_irq_enable(); mix_pool_bytes(pool, sizeof(pool)); - credit_init_bits(1); + credit_init_bits(max(1u, (count & U16_MAX) / 64)); memzero_explicit(pool, sizeof(pool)); } @@ -1139,22 +1141,30 @@ struct timer_rand_state { /* * This function adds entropy to the entropy "pool" by using timing - * delays. It uses the timer_rand_state structure to make an estimate - * of how many bits of entropy this call has added to the pool. - * - * The number "num" is also added to the pool - it should somehow describe - * the type of event which just happened. This is currently 0-255 for - * keyboard scan codes, and 256 upwards for interrupts. + * delays. It uses the timer_rand_state structure to make an estimate + * of how many bits of entropy this call has added to the pool. The + * value "num" is also added to the pool; it should somehow describe + * the type of event that just happened. */ static void add_timer_randomness(struct timer_rand_state *state, unsigned int num) { unsigned long entropy = random_get_entropy(), now = jiffies, flags; long delta, delta2, delta3; + unsigned int bits; - spin_lock_irqsave(&input_pool.lock, flags); - _mix_pool_bytes(&entropy, sizeof(entropy)); - _mix_pool_bytes(&num, sizeof(num)); - spin_unlock_irqrestore(&input_pool.lock, flags); + /* + * If we're in a hard IRQ, add_interrupt_randomness() will be called + * sometime after, so mix into the fast pool. + */ + if (in_irq()) { + fast_mix(this_cpu_ptr(&irq_randomness)->pool, + (unsigned long[2]){ entropy, num }); + } else { + spin_lock_irqsave(&input_pool.lock, flags); + _mix_pool_bytes(&entropy, sizeof(entropy)); + _mix_pool_bytes(&num, sizeof(num)); + spin_unlock_irqrestore(&input_pool.lock, flags); + } if (crng_ready()) return; @@ -1185,11 +1195,22 @@ static void add_timer_randomness(struct delta = delta3; /* - * delta is now minimum absolute delta. - * Round down by 1 bit on general principles, - * and limit entropy estimate to 12 bits. + * delta is now minimum absolute delta. Round down by 1 bit + * on general principles, and limit entropy estimate to 11 bits. + */ + bits = min(fls(delta >> 1), 11); + + /* + * As mentioned above, if we're in a hard IRQ, add_interrupt_randomness() + * will run after this, which uses a different crediting scheme of 1 bit + * per every 64 interrupts. In order to let that function do accounting + * close to the one in this function, we credit a full 64/64 bit per bit, + * and then subtract one to account for the extra one added. */ - credit_init_bits(min_t(unsigned int, fls(delta >> 1), 11)); + if (in_irq()) + this_cpu_ptr(&irq_randomness)->count += max(1u, bits * 64) - 1; + else + credit_init_bits(bits); } void add_input_randomness(unsigned int type, unsigned int code, Patches currently in stable-queue which might be from Jason@xxxxxxxxx are queue-4.14/random-do-not-take-pool-spinlock-at-boot.patch queue-4.14/random-remove-kernel.random.read_wakeup_threshold.patch queue-4.14/random-simplify-arithmetic-function-flow-in-account.patch queue-4.14/random-order-timer-entropy-functions-below-interrupt-functions.patch queue-4.14/random-introduce-drain_entropy-helper-to-declutter-crng_reseed.patch queue-4.14/random-fix-locking-in-crng_fast_load.patch queue-4.14/random-cleanup-uuid-handling.patch queue-4.14/random-group-userspace-read-write-functions.patch queue-4.14/random-make-credit_entropy_bits-always-safe.patch queue-4.14/latent_entropy-avoid-build-error-when-plugin-cflags-are-not-set.patch queue-4.14/revert-hwrng-core-freeze-khwrng-thread-during-suspend.patch queue-4.14/drivers-char-random.c-make-primary_crng-static.patch queue-4.14/random-do-not-re-init-if-crng_reseed-completes-before-primary-init.patch queue-4.14/random-always-fill-buffer-in-get_random_bytes_wait.patch queue-4.14/random-use-proper-jiffies-comparison-macro.patch queue-4.14/init-call-time_init-before-rand_initialize.patch queue-4.14/random-split-primary-secondary-crng-init-paths.patch queue-4.14/random-pull-add_hwgenerator_randomness-declaration-into-random.h.patch queue-4.14/random-unify-early-init-crng-load-accounting.patch queue-4.14/drivers-char-random.c-remove-unused-stuct-poolinfo-poolbits.patch queue-4.14/random-use-blake2s-instead-of-sha1-in-extraction.patch queue-4.14/random-make-cpu-trust-a-boot-parameter.patch queue-4.14/random-convert-to-using-fops-write_iter.patch queue-4.14/random-initialize-chacha20-constants-with-correct-endianness.patch queue-4.14/random-remove-incomplete-last_data-logic.patch queue-4.14/random-group-entropy-extraction-functions.patch queue-4.14/random-optimize-add_interrupt_randomness.patch queue-4.14/random-add-proper-spdx-header.patch queue-4.14/linux-random.h-remove-arch_has_random-arch_has_random_seed.patch queue-4.14/random-rewrite-header-introductory-comment.patch queue-4.14/random-make-dev-random-be-almost-like-dev-urandom.patch queue-4.14/random-remove-ifdef-d-out-interrupt-bench.patch queue-4.14/um-use-fallback-for-random_get_entropy-instead-of-zero.patch queue-4.14/lib-crypto-sha1-re-roll-loops-to-reduce-code-size.patch queue-4.14/random-tie-batched-entropy-generation-to-base_crng-generation.patch queue-4.14/sparc-use-fallback-for-random_get_entropy-instead-of-zero.patch queue-4.14/random-use-linear-min-entropy-accumulation-crediting.patch queue-4.14/random-remove-batched-entropy-locking.patch queue-4.14/xtensa-use-fallback-for-random_get_entropy-instead-of-zero.patch queue-4.14/powerpc-remove-arch_has_random-arch_has_random_seed.patch queue-4.14/fdt-add-support-for-rng-seed.patch queue-4.14/random-continually-use-hwgenerator-randomness.patch queue-4.14/random-access-input_pool_data-directly-rather-than-through-pointer.patch queue-4.14/random-add-arch_get_random_-long_early.patch queue-4.14/random-inline-leaves-of-rand_initialize.patch queue-4.14/random-cleanup-poolinfo-abstraction.patch queue-4.14/random-wire-up-fops-splice_-read-write-_iter.patch queue-4.14/random-handle-latent-entropy-and-command-line-from-random_init.patch queue-4.14/random-remove-use_input_pool-parameter-from-crng_reseed.patch queue-4.14/random-credit-architectural-init-the-exact-amount.patch queue-4.14/ia64-define-get_cycles-macro-for-arch-override.patch queue-4.14/random-replace-custom-notifier-chain-with-standard-one.patch queue-4.14/random-support-freezable-kthreads-in-add_hwgenerator_randomness.patch queue-4.14/random-document-get_random_int-family.patch queue-4.14/random-remove-the-blocking-pool.patch queue-4.14/random-avoid-initializing-twice-in-credit-race.patch queue-4.14/random-avoid-warnings-for-config_numa-builds.patch queue-4.14/crypto-drbg-add-fips-140-2-ctrng-for-noise-source.patch queue-4.14/random-mark-bootloader-randomness-code-as-__init.patch queue-4.14/random-zero-buffer-after-reading-entropy-from-userspace.patch queue-4.14/random-remove-whitespace-and-reorder-includes.patch queue-4.14/random-ignore-grnd_random-in-getentropy-2.patch queue-4.14/random-clear-fast-pool-crng-and-batches-in-cpuhp-bring-up.patch queue-4.14/random-document-add_hwgenerator_randomness-with-other-input-functions.patch queue-4.14/random-fix-typo-in-add_timer_randomness.patch queue-4.14/random-do-crng-pre-init-loading-in-worker-rather-than-irq.patch queue-4.14/powerpc-use-bool-in-archrandom.h.patch queue-4.14/random-do-not-split-fast-init-input-in-add_hwgenerator_randomness.patch queue-4.14/timekeeping-add-raw-clock-fallback-for-random_get_entropy.patch queue-4.14/random-early-initialization-of-chacha-constants.patch queue-4.14/crypto-drbg-prepare-for-more-fine-grained-tracking-of-seeding-state.patch queue-4.14/random-delete-code-to-pull-data-into-pools.patch queue-4.14/crypto-drbg-always-try-to-free-jitter-rng-instance.patch queue-4.14/random-simplify-entropy-debiting.patch queue-4.14/random-don-t-reset-crng_init_cnt-on-urandom_read.patch queue-4.14/random-skip-fast_init-if-hwrng-provides-large-chunk-of-entropy.patch queue-4.14/random-use-siphash-as-interrupt-entropy-accumulator.patch queue-4.14/random-avoid-checking-crng_ready-twice-in-random_init.patch queue-4.14/random-fix-soft-lockup-when-trying-to-read-from-an-uninitialized-blocking-pool.patch queue-4.14/random-group-sysctl-functions.patch queue-4.14/random-don-t-let-644-read-only-sysctls-be-written-to.patch queue-4.14/random-document-crng_fast_key_erasure-destination-possibility.patch queue-4.14/random-only-wake-up-writers-after-zap-if-threshold-was-passed.patch queue-4.14/random-use-wait_event_freezable-in-add_hwgenerator_randomness.patch queue-4.14/random-check-for-signal-and-try-earlier-when-generating-entropy.patch queue-4.14/random-check-for-signals-every-page_size-chunk-of-dev-random.patch queue-4.14/arm-use-fallback-for-random_get_entropy-instead-of-zero.patch queue-4.14/random-absorb-fast-pool-into-input-pool-after-fast-load.patch queue-4.14/random-give-sysctl_random_min_urandom_seed-a-more-sensible-value.patch queue-4.14/crypto-blake2s-generic-c-library-implementation-and-selftest.patch queue-4.14/random-cleanup-fractional-entropy-shift-constants.patch queue-4.14/random-use-rdseed-instead-of-rdrand-in-entropy-extraction.patch queue-4.14/random-move-rand_initialize-earlier.patch queue-4.14/random-don-t-wake-crng_init_wait-when-crng_init-1.patch queue-4.14/random-add-a-urandom_read_nowait-for-random-apis-that-don-t-warn.patch queue-4.14/random-do-not-sign-extend-bytes-for-rotation-when-mixing.patch queue-4.14/random-move-initialization-functions-out-of-hot-pages.patch queue-4.14/random-remove-dead-code-left-over-from-blocking-pool.patch queue-4.14/drivers-char-random.c-constify-poolinfo_table.patch queue-4.14/crypto-drbg-track-whether-drbg-was-seeded-with-rng_is_initialized.patch queue-4.14/random-use-computational-hash-for-entropy-extraction.patch queue-4.14/random-add-and-use-pr_fmt.patch queue-4.14/random-round-robin-registers-as-ulong-not-u32.patch queue-4.14/random-always-wake-up-entropy-writers-after-extraction.patch queue-4.14/s390-remove-arch_has_random-arch_has_random_seed.patch queue-4.14/random-do-not-xor-rdrand-when-writing-into-dev-random.patch queue-4.14/crypto-drbg-make-reseeding-from-get_random_bytes-synchronous.patch queue-4.14/random-convert-to-entropy_bits-for-better-code-readability.patch queue-4.14/char-random-add-a-newline-at-the-end-of-the-file.patch queue-4.14/random-move-randomize_page-into-mm-where-it-belongs.patch queue-4.14/random-only-call-crng_finalize_init-for-primary_crng.patch queue-4.14/random-cleanup-integer-types.patch queue-4.14/random-re-add-removed-comment-about-get_random_-u32-u64-reseeding.patch queue-4.14/random-unify-cycles_t-and-jiffies-usage-and-types.patch queue-4.14/random-insist-on-random_get_entropy-existing-in-order-to-simplify.patch queue-4.14/random-group-initialization-wait-functions.patch queue-4.14/linux-random.h-mark-config_arch_random-functions-__must_check.patch queue-4.14/random-remove-unused-extract_entropy-reserved-argument.patch queue-4.14/random-check-for-signal_pending-outside-of-need_resched-check.patch queue-4.14/random-access-primary_pool-directly-rather-than-through-pointer.patch queue-4.14/random-fix-sysctl-documentation-nits.patch queue-4.14/random-remove-unused-tracepoints.patch queue-4.14/random-only-read-from-dev-random-after-its-pool-has-received-128-bits.patch queue-4.14/nios2-use-fallback-for-random_get_entropy-instead-of-zero.patch queue-4.14/random-treat-bootloader-trust-toggle-the-same-way-as-cpu-trust-toggle.patch queue-4.14/random-make-consistent-usage-of-crng_ready.patch queue-4.14/lib-crypto-blake2s-move-hmac-construction-into-wireguard.patch queue-4.14/parisc-define-get_cycles-macro-for-arch-override.patch queue-4.14/x86-tsc-use-fallback-for-random_get_entropy-instead-of-zero.patch queue-4.14/crypto-chacha20-fix-keystream-alignment-for-chacha20_block.patch queue-4.14/random-add-grnd_insecure-to-return-best-effort-non-cryptographic-bytes.patch queue-4.14/random-fix-whitespace-pre-random-bytes-work.patch queue-4.14/crypto-drbg-move-dynamic-reseed_threshold-adjustments-to-__drbg_seed.patch queue-4.14/random-check-for-signals-after-page-of-pool-writes.patch queue-4.14/random-make-random_get_entropy-return-an-unsigned-long.patch queue-4.14/random-check-for-crng_init-0-in-add_device_randomness.patch queue-4.14/random-add-a-config-option-to-trust-the-cpu-s-hwrng.patch queue-4.14/random-remove-unnecessary-unlikely.patch queue-4.14/random-defer-fast-pool-mixing-to-worker.patch queue-4.14/random-harmonize-crng-init-done-messages.patch queue-4.14/crypto-blake2s-include-linux-bug.h-instead-of-asm-bug.h.patch queue-4.14/random-use-static-branch-for-crng_ready.patch queue-4.14/random-rather-than-entropy_store-abstraction-use-global.patch queue-4.14/drivers-char-random.c-remove-unused-dont_count_entropy.patch queue-4.14/random-remove-extern-from-functions-in-header.patch queue-4.14/siphash-use-one-source-of-truth-for-siphash-permutations.patch queue-4.14/random-group-entropy-collection-functions.patch queue-4.14/random-de-duplicate-input_pool-constants.patch queue-4.14/random-mix-build-time-latent-entropy-into-pool-at-init.patch queue-4.14/random-remove-useless-header-comment.patch queue-4.14/linux-random.h-use-false-with-bool.patch queue-4.14/maintainers-co-maintain-random.c.patch queue-4.14/random-remove-outdated-int_max-6-check-in-urandom_read.patch queue-4.14/m68k-use-fallback-for-random_get_entropy-instead-of-zero.patch queue-4.14/alpha-define-get_cycles-macro-for-arch-override.patch queue-4.14/random-mix-bootloader-randomness-into-pool.patch queue-4.14/random-remove-some-dead-code-of-poolinfo.patch queue-4.14/random-do-not-use-batches-when-crng_ready.patch queue-4.14/crypto-drbg-always-seeded-with-sp800-90b-compliant-noise-source.patch queue-4.14/s390-define-get_cycles-macro-for-arch-override.patch queue-4.14/random-do-not-pretend-to-handle-premature-next-security-model.patch queue-4.14/random-avoid-arch_get_random_seed_long-when-collecting-irq-randomness.patch queue-4.14/random-use-is_enabled-config_numa-instead-of-ifdefs.patch queue-4.14/random-make-crng-state-queryable.patch queue-4.14/random-avoid-superfluous-call-to-rdrand-in-crng-extraction.patch queue-4.14/random-use-symbolic-constants-for-crng_init-states.patch queue-4.14/random-reseed-more-often-immediately-after-booting.patch queue-4.14/random-ensure-early-rdseed-goes-through-mixer-on-init.patch queue-4.14/random-deobfuscate-irq-u32-u64-contributions.patch queue-4.14/random-do-not-use-input-pool-from-hard-irqs.patch queue-4.14/random-help-compiler-out-with-fast_mix-by-using-simpler-arguments.patch queue-4.14/revert-random-use-static-branch-for-crng_ready.patch queue-4.14/random-fix-crash-on-multiple-early-calls-to-add_bootloader_randomness.patch queue-4.14/random-return-nbytes-filled-from-hw-rng.patch queue-4.14/random-account-for-arch-randomness-in-bits.patch queue-4.14/crypto-blake2s-adjust-include-guard-naming.patch queue-4.14/random-do-not-allow-user-to-keep-crng-key-around-on-stack.patch queue-4.14/x86-remove-arch_has_random-arch_has_random_seed.patch queue-4.14/crypto-deduplicate-le32_to_cpu_array-and-cpu_to_le32_array.patch queue-4.14/random-remove-ratelimiting-for-in-kernel-unseeded-randomness.patch queue-4.14/random-remove-unused-irq_flags-argument-from-add_interrupt_randomness.patch queue-4.14/random-prepend-remaining-pool-constants-with-pool_.patch queue-4.14/powerpc-define-get_cycles-macro-for-arch-override.patch queue-4.14/random-remove-unused-output_pool-constants.patch queue-4.14/mips-use-fallback-for-random_get_entropy-instead-of-just-c0-random.patch queue-4.14/random-use-hash-function-for-crng_slow_load.patch queue-4.14/random-fix-typo-in-comments.patch queue-4.14/random-remove-preempt-disabled-region.patch queue-4.14/random-use-proper-return-types-on-get_random_-int-long-_wait.patch