Search squid archive

Fwd: Using Squid to redirect Steam CDNs using storeID_rewrite

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello all,
Apologies for the possibly incorrect format/posting of this query; I am new to this mode of discussion in relation to software.

I am attempting to use Squid, in combination with storeID rewrite, to redirect Steam CDN requests allowing multiple CDN requests to be served from the single Squid cache entry.
Here's a breakdown of my current versions/configurations:

OS: IPFire 2.19 (x86_64) - core103
Kernel: Linux ipfire 3.14.65-ipfire #1 SMP Tue Jun 14 06:21:39 GMT 2016 x86_64 GNU/Linux
Squid: 3.5.19

/var/ipfire/proxy/advanced/acls/include.acl
#squid.conf
acl cacheDomain dstdomain .steampowered.com .edgesuite.net .steamstatic.com .steamcontent.com
cache deny !cacheDomain
store_id_program /usr/lib/squid/storeid_file_rewrite /etc/squid/storeid_rewrite.conf
store_id_children 10 startup=3 idle=1 concurrency=0

/etc/squid/storeid_rewrite.conf
^http.*steam.*\.com\/(.*) http://steamupdates.squid.internal/$1

The issue appears to be stemming from how squid and storeID_rewrite interact; currently if I test the storeid_rewrite.conf with the following command:
echo http://valve314.steamcontent.com/depot/255711/chunk/a3f17a1be9c7861cbc56d1098b8ede146e114391? | /usr/lib/squid/storeid_file_rewrite /etc/squid/storeid_rewrite.conf
I get in return:
OK store-id=http://steamupdates.squid.internal/depot/255711/chunk/a3f17a1be9c7861cbc56d1098b8ede146e114391?

This indicates that storeID rewrite is functioning and using my RegEx command to rewrite the URL into one that only contains the unique game and chunk IDs from the URL. The issue is when I test the entire system using the following process I see multiple entries into the squid cache for the specific game chunk ID:

  • Remove /var/log/squid/access.log to ensure no previous attempts will appear in test
  • Clear the cache through ipFire webUI
  • Restart Squid cache service through ipFire webUI
  • Download game through Steam interface
  • Verify Squid cached the download chunks by grepping through both /var/log/squid/access.log and Squid cache for specific game chunk IDs (this is a spot check at best)
  • Change Steam CDN location through Steam UI
  • Delete Steam game local content
  • Re-download Steam game
  • Verify Squid cache using game chunk ID

The command used to grep against the Squid cache and it's results are as follows:
squidclient -h 192.168.1.1 cache_object://192.168.1.1 mgr:objects | grep f37f9405e2f38417a226eac378ac3982223d2966?
GET http://valve608.steamcontent.com/depot/26502/chunk/f37f9405e2f38417a226eac378ac3982223d2966?
GET http://valve313.steamcontent.com/depot/26502/chunk/f37f9405e2f38417a226eac378ac3982223d2966?

This indicates that at some point in the process, Squid is generating a KEY entry for the chunk based off the original Steam CDN URL and NOT the RegEx'd URL storeID_rewrite is supposedly generating.
I have attempted to determine how Squid is generating it's KEY entries for the chunks it is storing, but have had no luck (basing attempts off this white paper entry)
At this point I've exhausted my limited knowledge of how Squid and storeID rewrite function and any assistance would be quite welcome; please let me know if there's any further info needed to try and crack this walnut open!


_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux