Re: [PATCH] Please make the output of cache files reproducible

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hey Akira,

> not sure what it should be named but the curly brackets is supposed to
> be started at a new line

Thanks for the quick turnaround. They are, of course, called
"squigglies". ;-)

I've attached an updated patch that addresses all the issues you
mention, as well as fixing a rather embarrassing typo in the first
line of the commit message. :)


Best wishes,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@xxxxxxxxxx / chris-lamb.co.uk
       `-
From c31e9092fe499ee6d775ca6180def7b5f826de0f Mon Sep 17 00:00:00 2001
From: Chris Lamb <chris@xxxxxxxxxxxxxxxx>
Date: Sat, 28 Apr 2018 12:21:24 -0700
Subject: [PATCH] Ensure cache checksums are deterministic

Whilst working on the Reproducible Builds[0] effort, we noticed that
fontconfig generates unreproducible cache files.

This is due to fc-cache uses the modification timestamps of each
directory in the "checksum" and "checksum_nano" members of the _FcCache
struct. This is so that it can identify which cache files are valid
and/or require regeneration.

This patch changes the behaviour of the checksum calculations to prefer
the value of the SOURCE_DATE_EPOCH[1] environment variable over the
directory's own mtime. This variable can then be exported by build
systems to ensure reproducible output.

If SOURCE_DATE_EPOCH is not set or is newer than the mtime of the
directory, the existing behaviour is unchanged.

This work was sponsored by Tails[2].

 [0] https://reproducible-builds.org/
 [1] https://reproducible-builds.org/specs/source-date-epoch/
 [2] https://tails.boum.org/
---
 src/fccache.c | 58 +++++++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 52 insertions(+), 6 deletions(-)

diff --git a/src/fccache.c b/src/fccache.c
index 7abb750..6318135 100644
--- a/src/fccache.c
+++ b/src/fccache.c
@@ -989,6 +989,54 @@ FcDirCacheLoadFile (const FcChar8 *cache_file, struct stat *file_stat)
     return cache;
 }
 
+static int
+FcDirChecksum (struct stat *statb)
+{
+    int			ret = (int) statb->st_mtime;
+    char		*endptr;
+    char		*source_date_epoch;
+    unsigned long long	epoch;
+
+    source_date_epoch = getenv("SOURCE_DATE_EPOCH");
+    if (source_date_epoch) {
+	epoch = strtoull(source_date_epoch, &endptr, 10);
+
+	if (endptr == source_date_epoch)
+	    fprintf (stderr,
+		     "Fontconfig: SOURCE_DATE_EPOCH invalid\n");
+	else if ((errno == ERANGE && (epoch == ULLONG_MAX || epoch == 0))
+		|| (errno != 0 && epoch == 0))
+	    fprintf (stderr,
+		     "Fontconfig: SOURCE_DATE_EPOCH: strtoull: %s: %llu\n",
+		     strerror(errno), epoch);
+	else if (*endptr != '\0')
+	    fprintf (stderr,
+		     "Fontconfig: SOURCE_DATE_EPOCHh has trailing garbage\n");
+	else if (epoch > ULONG_MAX)
+	    fprintf (stderr,
+		     "Fontconfig: SOURCE_DATE_EPOCH must be <= %lu but saw: %llu\n",
+		     ULONG_MAX, epoch);
+	else if (epoch < ret)
+	    /* Only override if directory is newer */
+	    ret = (int) epoch;
+    }
+
+    return ret;
+}
+
+static int64_t
+FcDirChecksumNano (struct stat *statb)
+{
+#ifdef HAVE_STRUCT_STAT_ST_MTIM
+    /* No nanosecond component to parse */
+    if (getenv("SOURCE_DATE_EPOCH"))
+	return 0;
+    return statb->st_mtim.tv_nsec;
+#else
+    return 0;
+#endif
+}
+
 /*
  * Validate a cache file by reading the header and checking
  * the magic number and the size field
@@ -1007,10 +1055,10 @@ FcDirCacheValidateHelper (FcConfig *config, int fd, struct stat *fd_stat, struct
 	ret = FcFalse;
     else if (fd_stat->st_size != c.size)
 	ret = FcFalse;
-    else if (c.checksum != (int) dir_stat->st_mtime)
+    else if (c.checksum != FcDirChecksum (dir_stat))
 	ret = FcFalse;
 #ifdef HAVE_STRUCT_STAT_ST_MTIM
-    else if (c.checksum_nano != dir_stat->st_mtim.tv_nsec)
+    else if (c.checksum_nano != FcDirChecksumNano (dir_stat))
 	ret = FcFalse;
 #endif
     return ret;
@@ -1086,10 +1134,8 @@ FcDirCacheBuild (FcFontSet *set, const FcChar8 *dir, struct stat *dir_stat, FcSt
     cache->magic = FC_CACHE_MAGIC_ALLOC;
     cache->version = FC_CACHE_VERSION_NUMBER;
     cache->size = serialize->size;
-    cache->checksum = (int) dir_stat->st_mtime;
-#ifdef HAVE_STRUCT_STAT_ST_MTIM
-    cache->checksum_nano = dir_stat->st_mtim.tv_nsec;
-#endif
+    cache->checksum = FcDirChecksum (dir_stat);
+    cache->checksum_nano = FcDirChecksumNano (dir_stat);
 
     /*
      * Serialize directory name
-- 
2.17.0

_______________________________________________
Fontconfig mailing list
Fontconfig@xxxxxxxxxxxxxxxxxxxxx
https://lists.freedesktop.org/mailman/listinfo/fontconfig

[Index of Archives]     [Fedora Fonts]     [Fedora Users]     [Fedora Cloud]     [Kernel]     [Fedora Packaging]     [Fedora Desktop]     [PAM]     [Gimp Graphics Editor]     [Yosemite News]

  Powered by Linux