On Fri, Dec 18, 2020 at 02:29:45AM +0000, brian m. carlson wrote:
> > And from that argument, I think the obvious question becomes: is it
> > worth using a real one-way function, as opposed to just obscuring the
> > raw bytes (which Ævar went into in more detail). I don't have a strong
> > opinion either way (the obvious one in favor is that it's less expensive
> > to do so; and something like "git log" will have to either compute a lot
> > of these hashes, or cache the hash computations internally).
> [...]
> So I think I'm firmly in favor of hashing. If that means my patch needs
> to implement caching, then I'll reroll with that change. I think by
> switching to a hash table I may be able to actually improve total
> performance overall, at least in some cases.
OK. I agree it raises the bar a little bit. Whether that matters or not
depends on your threat model (e.g., casual spammers versus dedicated
information seekers). I don't have a particularly strong opinion on
what's realistic, but I don't mind erring on the side of caution here.
It might be worth making a short argument along those lines in the
commit message.
As far as caching goes, my main concern is mostly that people who are
not using the feature do not pay a performance penalty. So:
- if the feature is not used in the repository's mailmap, it should
have zero cost (i.e., we do not bother hashing lookup entries if
there are no hashed entries in the map)
- as soon as there is one hashed entry, we need to hash the key for
every lookup in the map. I'm not sure what the overhead is like. It
might be negligible. But I think we should confirm that before
proceeding.
> And as someone who had to download all 21 GB of the Chromium repository
> for testing purposes recently, I can tell you that absent a very
> compelling use case, nobody's going to want to download that entire
> repository just to extract some personal information, especially since
> the git index-pack operation is essentially guaranteed to take at least
> 7 minutes at maximum speed. So by hashing, we've guaranteed significant
> inconvenience unless you have the repository, whereas that's not the
> case for base64. And making abuse even slightly harder can often deter
> a surprising amount of it[0].
They just need the objects that have ident lines in them, so:
$ time git clone --bare --filter=tree:0 https://github.com/chromium/chromium
Cloning into bare repository 'chromium.git'...
remote: Enumerating objects: 202, done.
remote: Counting objects: 100% (202/202), done.
remote: Compressing objects: 100% (161/161), done.
remote: Total 1105453 (delta 49), reused 194 (delta 41), pack-reused 1105251
Receiving objects: 100% (1105453/1105453), 462.14 MiB | 11.13 MiB/s, done.
Resolving deltas: 100% (99790/99790), done.
real 0m49.304s
user 0m21.330s
sys 0m4.727s
gets you there much quicker. I don't think that negates your point about
raising the bar, but my guess is that the threat model of "casual
spammer" would probably be deterred, but "troll who wants to annoy
specific person" would probably not be.
-Peff
