Search squid archive

Re: Compute digest as content is written to cache

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 11/08/12 11:37 PM, Amos Jeffries wrote:
On 11/08/2012 10:21 p.m., Jack Bates wrote:
On 11/08/12 12:30 AM, Amos Jeffries wrote:
On 11/08/2012 7:22 p.m., Jack Bates wrote:
I am interested in intercepting content as it is written to the cache,
and computing a digest from the content. Do you know if this can be
done in some kind of add on, or would it require a change to the core?

What type of digest and to what purpose?

I was thinking of using OpenSSL
SHA256_Init()/SHA256_Update()/SHA256_Final(). The purpose I have in
mind is to detect identical content at different URLs

Given a response with a "Location: ..." header and a "Digest:
SHA-256=..." header (such as from MirrorBrain), if the URL in the
"Location: ..." header is not already cached but the "Digest:
SHA-256=..." header matches the content at some other URL that is
already cached, then I want to update the "Location: ..." header with
the cached URL. I think this should redirect clients to mirrors that
are already cached

Small problem there. The digest is not calculated/known until the object
is finished arriving. By then it is too late to attach new headers. And
way too late to decide whether to ask that source for it.

Thanks Amos, but I want to calculate the digest of different objects than the ones that I want to rewrite the "Location: ..." header of

As objects are cached, I want to compute the digest and record it (possibly in some external database). Then when a response with a "Location: ..." header and a "Digest: SHA-256=..." header is received (such as from MirrorBrain), if the URL in the "Location: ..." header is not already cached, then I want to compare the digest in the "Digest: SHA-256=..." header to the previously recorded digests. If a match is found, than I want to rewrite the "Location: ..." header with the URL of the match


[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux