Am 19.06.24 um 13:09 schrieb Julian Sikorski:
Am 13.02.24 um 18:52 schrieb Julian Sikorski:
Am 13.02.24 um 16:16 schrieb Gary Buhrmaster:
On Tue, Feb 13, 2024 at 9:52 AM Miroslav Suchý <msuchy@xxxxxxxxxx>
wrote:
Dne 13. 02. 24 v 9:08 Julian Sikorski napsal(a):
Could this be the reason for ccache not working?
I wonder whether it is Mock problem, Ccache issue or problem in
packaging? Does the ccache speadup the build when you
run it with plain rpmbuild and ccache on host?
I have lost track of the details (and the
version of ccache when the issue was
addressed/patched), but ccache at one
time included SOURCE_DATE_EPOCH
in the default hash, resulting in no
cache hits.
This appears to have been fixed in ccache-4.2. F36, obsolete as it is,
shipped with 4.5.1.
Having said that, with current kernel there is this line in the log:
+ perl -p -i -e
's/^CONFIG_BUILD_SALT.*/CONFIG_BUILD_SALT="6.7.4-200.lacie03.fc39.x86_64"/' .config
This would change every build.
Best regards,
Julian
I had to bisect a kernel regression once again so I revisited this. I
have modified my mock with PR showing ccache stats [1].
Interestingly enough, rebuilding the exact same srpm does result in a
significant speedup and almost 100% cache hits. Build took ~12 minutes
(as opposed to ~25 when building without cache) and cache stats looks as
follows:
DEBUG util.py:463: Cacheable calls: 21570 / 21954 (98.25%)
DEBUG util.py:463: Hits: 21554 / 21570 (99.93%)
DEBUG util.py:463: Direct: 21458 / 21554 (99.55%)
DEBUG util.py:463: Preprocessed: 96 / 21554 ( 0.45%)
DEBUG util.py:463: Misses: 16 / 21570 ( 0.07%)
DEBUG util.py:463: Uncacheable calls: 384 / 21954 ( 1.75%)
Next bisect step, couple hundred commits away (from fe46a7dd189e to
eaf0e7a3d271 in case this is relevant), is a completely different story.
Build took ~45 minutes and there are almost no cache hits:
DEBUG util.py:463: Cacheable calls: 21554 / 21938 (98.25%)
DEBUG util.py:463: Hits: 1421 / 21554 ( 6.59%)
DEBUG util.py:463: Direct: 1365 / 1421 (96.06%)
DEBUG util.py:463: Preprocessed: 56 / 1421 ( 3.94%)
DEBUG util.py:463: Misses: 20133 / 21554 (93.41%)
DEBUG util.py:463: Uncacheable calls: 384 / 21938 ( 1.75%)
Further down the bisect, despite differences from build to build getting
smaller, there is almost no difference. From 902861e34c40 to
e5eb28f6d1af, build took around ~45 minutes too, and almost no cache
hits were found either:
DEBUG util.py:463: Cacheable calls: 21571 / 21955 (98.25%)
DEBUG util.py:463: Hits: 1395 / 21571 ( 6.47%)
DEBUG util.py:463: Direct: 1373 / 1395 (98.42%)
DEBUG util.py:463: Preprocessed: 22 / 1395 ( 1.58%)
DEBUG util.py:463: Misses: 20176 / 21571 (93.53%)
DEBUG util.py:463: Uncacheable calls: 384 / 21955 ( 1.75%)
Can the cache files be inspected with something to see what is getting
cached?
Best regards,
Julian
[1] https://github.com/rpm-software-management/mock/pull/1299
I decided to investigate this further and enabled CCACHE_DEBUG and
CCACHE_DEBUGDIR for mock. Patch attached in case anyone is interested.
I then started building consecutive 6.9.x kernels, paying attention to
files which have not changed in a really long time.
Exhibit 1: ipc/compat.c, hasn't changed in six years.
Compared ccache-input-text of 6.9.5 and 6.9.6 contains the attached
difference. Different cwd obviously and two %P to %c changes in assembly.
Differences from 6.9.6 to 6.9.7 are minimal too.
I then decided to change just one file, lib/sort.c and see what happens.
The only remaining difference is the path (in two places), but still no
cache hits.
Does anyone know what this # 1 " entry is? Looking at the docs, setting
base_dir might help here but getting the right value could be tricky.
Best regards,
Julian
From 43ea7be73b257db0eea2597ae961d520186d45dc Mon Sep 17 00:00:00 2001
From: rpm-build <rpm-build>
Date: Sat, 29 Jun 2024 14:25:01 +0200
Subject: [PATCH] Enable ccache debugging
---
py/mockbuild/plugins/ccache.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/py/mockbuild/plugins/ccache.py b/py/mockbuild/plugins/ccache.py
index 1a20846..e1058f2 100644
--- a/py/mockbuild/plugins/ccache.py
+++ b/py/mockbuild/plugins/ccache.py
@@ -57,7 +57,7 @@ class CCache(object):
@traceLog()
def _ccachePreInitHook(self):
getLog().info("enabled ccache")
- envupd = {"CCACHE_DIR": "/var/tmp/ccache", "CCACHE_UMASK": "002"}
+ envupd = {"CCACHE_DIR": "/var/tmp/ccache", "CCACHE_UMASK": "002", "CCACHE_DEBUG": "1", "CCACHE_DEBUGDIR": "/var/tmp/ccache"}
if self.ccache_opts.get('compress') is not None:
envupd["CCACHE_COMPRESS"] = str(self.ccache_opts['compress'])
self.buildroot.env.update(envupd)
--
2.45.2
--- kernel-6.9.5/linux-6.9.5-200.ds01.fc40.x86_64/ipc/compat.o.20240629_211025_389664.ccache-input-text 2024-06-29 21:10:26.527865814 +0200
+++ kernel-6.9.6/linux-6.9.6-200.ds01.fc40.x86_64/ipc/compat.o.20240629_205637_186348.ccache-input-text 2024-06-29 20:56:38.148863406 +0200
@@ -10,7 +10,7 @@
### LANG
C.UTF-8
### cwd
-/builddir/build/BUILD/kernel-6.9.5/linux-6.9.5-200.ds01.fc40.x86_64
+/builddir/build/BUILD/kernel-6.9.6/linux-6.9.6-200.ds01.fc40.x86_64
=== DIRECT MODE ===
### cache entry version
1
@@ -438,7 +438,7 @@
ipc/compat.c
"
# 1 "
-/builddir/build/BUILD/kernel-6.9.5/linux-6.9.5-200.ds01.fc40.x86_64
+/builddir/build/BUILD/kernel-6.9.6/linux-6.9.6-200.ds01.fc40.x86_64
"
# 0 "
<built-in>
@@ -16770,7 +16770,7 @@
kmsan_unpoison_memory(page, ((1UL) << 12));
- asm __inline volatile ("# ALT: oldinstr2\n" "661:\n\t" "call %P[old]" "\n662:\n" "# ALT: padding2\n" ".skip -((" "((" "665""1""f-""664""1""f" ") ^ (((" "665""1""f-""664""1""f" ") ^ (" "665""2""f-""664""2""f" ")) & -(-((" "665""1""f-""664""1""f" ") < (" "665""2""f-""664""2""f" ")))))" " - (" "662b-661b" ")) > 0) * " "(" "((" "665""1""f-""664""1""f" ") ^ (((" "665""1""f-""664""1""f" ") ^ (" "665""2""f-""664""2""f" ")) & -(-((" "665""1""f-""664""1""f" ") < (" "665""2""f-""664""2""f" ")))))" " - (" "662b-661b" ")), 0x90\n" "663" ":\n" ".pushsection .altinstructions,\"a\"\n" " .long 661b - .\n" " .long " "664""1""f - .\n" " .4byte " "( 3*32+16)" "\n" " .byte " "663""b-661b" "\n" " .byte " "665""1""f-""664""1""f" "\n" " .long 661b - .\n" " .long " "664""2""f - .\n" " .4byte " "( 9*32+ 9)" "\n" " .byte " "663""b-661b" "\n" " .byte " "665""2""f-""664""2""f" "\n" ".popsection\n" ".pushsection .altinstr_replacement, \"ax\"\n" "# ALT: replacement " "1" "\n" "664""1"":\n\t" "call %P[new1]" "\n" "665""1" ":\n" "# ALT: replacement " "2" "\n" "664""2"":\n\t" "call %P[new2]" "\n" "665""2" ":\n" ".popsection\n" : "=D" (page), "+r" (current_stack_pointer) : [old] "i" (clear_page_orig), [new1] "i" (clear_page_rep), [new2] "i" (clear_page_erms), "0" (page) : "cc", "memory", "rax", "rcx")
+ asm __inline volatile ("# ALT: oldinstr2\n" "661:\n\t" "call %c[old]" "\n662:\n" "# ALT: padding2\n" ".skip -((" "((" "665""1""f-""664""1""f" ") ^ (((" "665""1""f-""664""1""f" ") ^ (" "665""2""f-""664""2""f" ")) & -(-((" "665""1""f-""664""1""f" ") < (" "665""2""f-""664""2""f" ")))))" " - (" "662b-661b" ")) > 0) * " "(" "((" "665""1""f-""664""1""f" ") ^ (((" "665""1""f-""664""1""f" ") ^ (" "665""2""f-""664""2""f" ")) & -(-((" "665""1""f-""664""1""f" ") < (" "665""2""f-""664""2""f" ")))))" " - (" "662b-661b" ")), 0x90\n" "663" ":\n" ".pushsection .altinstructions,\"a\"\n" " .long 661b - .\n" " .long " "664""1""f - .\n" " .4byte " "( 3*32+16)" "\n" " .byte " "663""b-661b" "\n" " .byte " "665""1""f-""664""1""f" "\n" " .long 661b - .\n" " .long " "664""2""f - .\n" " .4byte " "( 9*32+ 9)" "\n" " .byte " "663""b-661b" "\n" " .byte " "665""2""f-""664""2""f" "\n" ".popsection\n" ".pushsection .altinstr_replacement, \"ax\"\n" "# ALT: replacement " "1" "\n" "664""1"":\n\t" "call %c[new1]" "\n" "665""1" ":\n" "# ALT: replacement " "2" "\n" "664""2"":\n\t" "call %c[new2]" "\n" "665""2" ":\n" ".popsection\n" : "=D" (page), "+r" (current_stack_pointer) : [old] "i" (clear_page_orig), [new1] "i" (clear_page_rep), [new2] "i" (clear_page_erms), "0" (page) : "cc", "memory", "rax", "rcx")
@@ -18403,7 +18403,7 @@
static inline __attribute__((__gnu_inline__)) __attribute__((__unused__)) __attribute__((no_instrument_function)) __attribute__((__always_inline__)) bool _static_cpu_has(u16 bit)
{
asm goto(
- "# ALT: oldinstr2\n" "661:\n\t" "jmp 6f" "\n662:\n" "# ALT: padding2\n" ".skip -((" "((" "665""1""f-""664""1""f" ") ^ (((" "665""1""f-""664""1""f" ") ^ (" "665""2""f-""664""2""f" ")) & -(-((" "665""1""f-""664""1""f" ") < (" "665""2""f-""664""2""f" ")))))" " - (" "662b-661b" ")) > 0) * " "(" "((" "665""1""f-""664""1""f" ") ^ (((" "665""1""f-""664""1""f" ") ^ (" "665""2""f-""664""2""f" ")) & -(-((" "665""1""f-""664""1""f" ") < (" "665""2""f-""664""2""f" ")))))" " - (" "662b-661b" ")), 0x90\n" "663" ":\n" ".pushsection .altinstructions,\"a\"\n" " .long 661b - .\n" " .long " "664""1""f - .\n" " .4byte " "( 3*32+21)" "\n" " .byte " "663""b-661b" "\n" " .byte " "665""1""f-""664""1""f" "\n" " .long 661b - .\n" " .long " "664""2""f - .\n" " .4byte " "%P[feature]" "\n" " .byte " "663""b-661b" "\n" " .byte " "665""2""f-""664""2""f" "\n" ".popsection\n" ".pushsection .altinstr_replacement, \"ax\"\n" "# ALT: replacement " "1" "\n" "664""1"":\n\t" "jmp %l[t_no]" "\n" "665""1" ":\n" "# ALT: replacement " "2" "\n" "664""2"":\n\t" "" "\n" "665""2" ":\n" ".popsection\n"
+ "# ALT: oldinstr2\n" "661:\n\t" "jmp 6f" "\n662:\n" "# ALT: padding2\n" ".skip -((" "((" "665""1""f-""664""1""f" ") ^ (((" "665""1""f-""664""1""f" ") ^ (" "665""2""f-""664""2""f" ")) & -(-((" "665""1""f-""664""1""f" ") < (" "665""2""f-""664""2""f" ")))))" " - (" "662b-661b" ")) > 0) * " "(" "((" "665""1""f-""664""1""f" ") ^ (((" "665""1""f-""664""1""f" ") ^ (" "665""2""f-""664""2""f" ")) & -(-((" "665""1""f-""664""1""f" ") < (" "665""2""f-""664""2""f" ")))))" " - (" "662b-661b" ")), 0x90\n" "663" ":\n" ".pushsection .altinstructions,\"a\"\n" " .long 661b - .\n" " .long " "664""1""f - .\n" " .4byte " "( 3*32+21)" "\n" " .byte " "663""b-661b" "\n" " .byte " "665""1""f-""664""1""f" "\n" " .long 661b - .\n" " .long " "664""2""f - .\n" " .4byte " "%c[feature]" "\n" " .byte " "663""b-661b" "\n" " .byte " "665""2""f-""664""2""f" "\n" ".popsection\n" ".pushsection .altinstr_replacement, \"ax\"\n" "# ALT: replacement " "1" "\n" "664""1"":\n\t" "jmp %l[t_no]" "\n" "665""1" ":\n" "# ALT: replacement " "2" "\n" "664""2"":\n\t" "" "\n" "665""2" ":\n" ".popsection\n"
".pushsection .altinstr_aux,\"ax\"\n"
"6:\n"
" testb %[bitnum]," "%P[cap_byte] (%% rip)" "\n"
--- kernel-6.9.6/linux-6.9.6-200.ds01.fc40.x86_64/ipc/compat.o.20240629_205637_186348.ccache-input-text 2024-06-29 20:56:38.148863406 +0200
+++ kernel-6.9.7/linux-6.9.7-200.ds01.fc40.x86_64/ipc/compat.o.20240629_212409_743515.ccache-input-text 2024-06-29 21:24:10.699574068 +0200
@@ -10,7 +10,7 @@
### LANG
C.UTF-8
### cwd
-/builddir/build/BUILD/kernel-6.9.6/linux-6.9.6-200.ds01.fc40.x86_64
+/builddir/build/BUILD/kernel-6.9.7/linux-6.9.7-200.ds01.fc40.x86_64
=== DIRECT MODE ===
### cache entry version
1
@@ -438,7 +438,7 @@
ipc/compat.c
"
# 1 "
-/builddir/build/BUILD/kernel-6.9.6/linux-6.9.6-200.ds01.fc40.x86_64
+/builddir/build/BUILD/kernel-6.9.7/linux-6.9.7-200.ds01.fc40.x86_64
"
# 0 "
<built-in>
@@ -81727,11 +81727,12 @@
# 2056 "
./include/linux/security.h
"
-int security_audit_rule_init(u32 field, u32 op, char *rulestr, void **lsmrule);
+int security_audit_rule_init(u32 field, u32 op, char *rulestr, void **lsmrule,
+ gfp_t gfp);
int security_audit_rule_known(struct audit_krule *krule);
int security_audit_rule_match(u32 secid, u32 field, u32 op, void *lsmrule);
void security_audit_rule_free(void *lsmrule);
-# 2088 "
+# 2089 "
./include/linux/security.h
"
extern struct dentry *securityfs_create_file(const char *name, umode_t mode,
@@ -81743,7 +81744,7 @@
const char *target,
const struct inode_operations *iops);
extern void securityfs_remove(struct dentry *dentry);
-# 2129 "
+# 2130 "
./include/linux/security.h
"
union bpf_attr;
@@ -81765,7 +81766,7 @@
extern void security_bpf_token_free(struct bpf_token *token);
extern int security_bpf_token_cmd(const struct bpf_token *token, enum bpf_cmd cmd);
extern int security_bpf_token_capable(const struct bpf_token *token, int cap);
-# 2205 "
+# 2206 "
./include/linux/security.h
"
struct perf_event_attr;
@@ -81777,7 +81778,7 @@
extern void security_perf_event_free(struct perf_event *event);
extern int security_perf_event_read(struct perf_event *event);
extern int security_perf_event_write(struct perf_event *event);
-# 2244 "
+# 2245 "
./include/linux/security.h
"
extern int security_uring_override_creds(const struct cred *new);
--- kernel-6.9.7/linux-6.9.7-200.ds01.fc40.x86_64/ipc/compat.o.20240629_212409_743515.ccache-input-text 2024-06-29 21:24:10.699574068 +0200
+++ kernel-6.9.7/linux-6.9.7-200.test.fc40.x86_64/ipc/compat.o.20240629_213856_347763.ccache-input-text 2024-06-29 21:38:57.472891436 +0200
@@ -10,7 +10,7 @@
### LANG
C.UTF-8
### cwd
-/builddir/build/BUILD/kernel-6.9.7/linux-6.9.7-200.ds01.fc40.x86_64
+/builddir/build/BUILD/kernel-6.9.7/linux-6.9.7-200.test.fc40.x86_64
=== DIRECT MODE ===
### cache entry version
1
@@ -438,7 +438,7 @@
ipc/compat.c
"
# 1 "
-/builddir/build/BUILD/kernel-6.9.7/linux-6.9.7-200.ds01.fc40.x86_64
+/builddir/build/BUILD/kernel-6.9.7/linux-6.9.7-200.test.fc40.x86_64
"
# 0 "
<built-in>
--
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
