Re: [PATCH-cgroup v2] cgroup: Move rcu_head up near the top of cgroup_root

Waiman Long <longman@xxxxxxxxxx> · Thu, 7 Dec 2023 12:40:33 -0500

On 12/7/23 11:46, Michal Koutný wrote:
On Thu, Dec 07, 2023 at 08:46:14AM -0500, Waiman Long <longman@xxxxxxxxxx> wrote:
Commit 77070eeb8821 ("cgroup: Avoid false cacheline sharing of read
mostly rstat_cpu") happens to be the last straw that breaks it.
FTR, when I build kernel from that commit, I can see

struct cgroup_root {
	struct kernfs_root *       kf_root;              /*     0     8 */
	unsigned int               subsys_mask;          /*     8     4 */
	int                        hierarchy_id;         /*    12     4 */

	/* XXX 48 bytes hole, try to pack */

	/* --- cacheline 1 boundary (64 bytes) --- */
	struct cgroup              cgrp __attribute__((__aligned__(64))); /*    64  2368 */

	/* XXX last struct has 8 bytes of padding */

	/* --- cacheline 38 boundary (2432 bytes) --- */
	struct cgroup *            cgrp_ancestor_storage; /*  2432     8 */
	atomic_t                   nr_cgrps;             /*  2440     4 */

	/* XXX 4 bytes hole, try to pack */

	struct list_head           root_list;            /*  2448    16 */
	struct callback_head       rcu __attribute__((__aligned__(8))); /*  2464    16 */
	unsigned int               flags;                /*  2480     4 */
	char                       release_agent_path[4096]; /*  2484  4096 */
	/* --- cacheline 102 boundary (6528 bytes) was 52 bytes ago --- */
	char                       name[64];             /*  6580    64 */

	/* size: 6656, cachelines: 104, members: 11 */
	/* sum members: 6592, holes: 2, sum holes: 52 */
	/* padding: 12 */
	/* paddings: 1, sum paddings: 8 */
	/* forced alignments: 2, forced holes: 1, sum forced holes: 48 */
} __attribute__((__aligned__(64)));
2480 has still quite a reserve below 4096. (I can't see an CONFIG_*
affecting this.)

Perhaps, I missed something from the linux-next merging thread?

CONFIG_LOCKDEP and some other debug configs are enabled with 
allmodconfig. This can greatly increase the size of some of the 
structures. I am not able to use pahole due to missing BTF info so I 
don't the exact size. However, I can reproduce the build failure and the 
patch is  able to fix it.

Cheers,
Longman