Caching Content from Dynamic URLs (very hacky solution)

["Volker-Yoblick, Adam" <avolker@xxxxxx>]

Greetings everyone,

Like many others, we have the need to cache data from URLs that are dynamic, even if the data is the same. I know about the storeurl_rewrite command, but that's only available in squid 2.x, and for some reason, that version didn't play well with our proprietary tools.

I wanted to share my solution (which is a super ugly hack) since it seems to be working for us, in case anyone else needs to make a similar change in the future. All changes were done to a 3.1.9 release. The changes take about 5 minutes total.

-----

1. In src/protos.h, add the following declaration:

	SQUIDCEXTERN char *getRelativePathFromUrl(char *);


2. In src/store_key_md5.cc (or in any file, really), add the following:

char*
getRelativePathFromUrl(char *url)
{
	char *relativePath = ...
	return relativePath;
}

This function should map a dynamic path to a relative one. The structure of all our URLs is http://********/foo/<GUID>/path/to/file.txt, so this function, for us, would return path/to/file.txt
I'm not pasting the body of this function, since it's VERY specific to our tools, but you can hack in whatever URL translation you need here.


3. In src/store_key_md5.cc, add the following line:

	char *relativeUrl = getRelativePathFromUrl((char *)url);

To the following functions:

	storeKeyPrivate
	storeKeyPublic
	storeKeyPublicByRequestMethod

Then, replace all references to the const char *url with relativeUrl.


4. In src/StoreMetaUrl.cc, add the following lines:

    char *relativeUrl = getRelativePathFromUrl(e->mem_obj->url);
    char *relativeValue = getRelativePathFromUrl((char *)value);

Replace all references to e->mem_obj->url with relativeUrl
Replace all references to (char *)value with relativeValue

-----

That's all I did, and the caching seems to be working. If I deploy two sets of the exact same files, with different GUIDs, the second set is all TCP_HITs.

I know this solution is not a good one, just wanted to share in case anyone else was in a similar bind like me.

Thanks for your time,

-Adam