Am 29.06.24 um 22:02 schrieb Julian Sikorski:
Am 19.06.24 um 13:09 schrieb Julian Sikorski:
Am 13.02.24 um 18:52 schrieb Julian Sikorski:
Am 13.02.24 um 16:16 schrieb Gary Buhrmaster:
On Tue, Feb 13, 2024 at 9:52 AM Miroslav Suchý <msuchy@xxxxxxxxxx>
wrote:
Dne 13. 02. 24 v 9:08 Julian Sikorski napsal(a):
Could this be the reason for ccache not working?
I wonder whether it is Mock problem, Ccache issue or problem in
packaging? Does the ccache speadup the build when you
run it with plain rpmbuild and ccache on host?
I have lost track of the details (and the
version of ccache when the issue was
addressed/patched), but ccache at one
time included SOURCE_DATE_EPOCH
in the default hash, resulting in no
cache hits.
This appears to have been fixed in ccache-4.2. F36, obsolete as it
is, shipped with 4.5.1.
Having said that, with current kernel there is this line in the log:
+ perl -p -i -e
's/^CONFIG_BUILD_SALT.*/CONFIG_BUILD_SALT="6.7.4-200.lacie03.fc39.x86_64"/' .config
This would change every build.
Best regards,
Julian
I had to bisect a kernel regression once again so I revisited this. I
have modified my mock with PR showing ccache stats [1].
Interestingly enough, rebuilding the exact same srpm does result in a
significant speedup and almost 100% cache hits. Build took ~12 minutes
(as opposed to ~25 when building without cache) and cache stats looks
as follows:
DEBUG util.py:463: Cacheable calls: 21570 / 21954 (98.25%)
DEBUG util.py:463: Hits: 21554 / 21570 (99.93%)
DEBUG util.py:463: Direct: 21458 / 21554 (99.55%)
DEBUG util.py:463: Preprocessed: 96 / 21554 ( 0.45%)
DEBUG util.py:463: Misses: 16 / 21570 ( 0.07%)
DEBUG util.py:463: Uncacheable calls: 384 / 21954 ( 1.75%)
Next bisect step, couple hundred commits away (from fe46a7dd189e to
eaf0e7a3d271 in case this is relevant), is a completely different
story. Build took ~45 minutes and there are almost no cache hits:
DEBUG util.py:463: Cacheable calls: 21554 / 21938 (98.25%)
DEBUG util.py:463: Hits: 1421 / 21554 ( 6.59%)
DEBUG util.py:463: Direct: 1365 / 1421 (96.06%)
DEBUG util.py:463: Preprocessed: 56 / 1421 ( 3.94%)
DEBUG util.py:463: Misses: 20133 / 21554 (93.41%)
DEBUG util.py:463: Uncacheable calls: 384 / 21938 ( 1.75%)
Further down the bisect, despite differences from build to build
getting smaller, there is almost no difference. From 902861e34c40 to
e5eb28f6d1af, build took around ~45 minutes too, and almost no cache
hits were found either:
DEBUG util.py:463: Cacheable calls: 21571 / 21955 (98.25%)
DEBUG util.py:463: Hits: 1395 / 21571 ( 6.47%)
DEBUG util.py:463: Direct: 1373 / 1395 (98.42%)
DEBUG util.py:463: Preprocessed: 22 / 1395 ( 1.58%)
DEBUG util.py:463: Misses: 20176 / 21571 (93.53%)
DEBUG util.py:463: Uncacheable calls: 384 / 21955 ( 1.75%)
Can the cache files be inspected with something to see what is getting
cached?
Best regards,
Julian
[1] https://github.com/rpm-software-management/mock/pull/1299
I decided to investigate this further and enabled CCACHE_DEBUG and
CCACHE_DEBUGDIR for mock. Patch attached in case anyone is interested.
I then started building consecutive 6.9.x kernels, paying attention to
files which have not changed in a really long time.
Exhibit 1: ipc/compat.c, hasn't changed in six years.
Compared ccache-input-text of 6.9.5 and 6.9.6 contains the attached
difference. Different cwd obviously and two %P to %c changes in assembly.
Differences from 6.9.6 to 6.9.7 are minimal too.
I then decided to change just one file, lib/sort.c and see what happens.
The only remaining difference is the path (in two places), but still no
cache hits.
Does anyone know what this # 1 " entry is? Looking at the docs, setting
base_dir might help here but getting the right value could be tricky.
Best regards,
Julian
I tried experimenting with base_dir and hash_dir with the attached patch
and managed to get caching across multiple builds working.
CCACHE_NOHASHDIR takes care of the cwd in the input text, whereas
CCACHE_BASEDIR takes care of the # 1 ".
I have now filed an issue with mock in order to document my findings:
https://github.com/rpm-software-management/mock/issues/1395
Best regards,
Julian
From 43ea7be73b257db0eea2597ae961d520186d45dc Mon Sep 17 00:00:00 2001
From: rpm-build <rpm-build>
Date: Sat, 29 Jun 2024 14:25:01 +0200
Subject: [PATCH] Enable ccache debugging
---
py/mockbuild/plugins/ccache.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/py/mockbuild/plugins/ccache.py b/py/mockbuild/plugins/ccache.py
index 1a20846..e1058f2 100644
--- a/py/mockbuild/plugins/ccache.py
+++ b/py/mockbuild/plugins/ccache.py
@@ -57,7 +57,7 @@ class CCache(object):
@traceLog()
def _ccachePreInitHook(self):
getLog().info("enabled ccache")
- envupd = {"CCACHE_DIR": "/var/tmp/ccache", "CCACHE_UMASK": "002"}
+ envupd = {"CCACHE_DIR": "/var/tmp/ccache", "CCACHE_UMASK": "002", "CCACHE_DEBUG": "1", "CCACHE_DEBUGDIR": "/var/tmp/ccache", "CCACHE_BASEDIR": "/builddir/build/BUILD/kernel-6.9.7/linux-6.9.7-200.ds01.fc40.x86_64", "CCACHE_NOHASHDIR": "1"}
if self.ccache_opts.get('compress') is not None:
envupd["CCACHE_COMPRESS"] = str(self.ccache_opts['compress'])
self.buildroot.env.update(envupd)
--
2.45.2
--
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
