Hey Omid,
Indeed my preference is that if you can ask ask and I will try to give you couple more details on the service and the subject.
Windows updates are considered very static since they have Last-Modified and Date headers.
Else then that, they support to validate and invalidate requests based on these times.
This what makes it so simple to store them.
Now the main issue(which you probably already know) is that clients are requesting partial content from a full static object.
The full file or object is a resource and when you have the full object most web services can serve the partial content.
Technically if the client software uses static ranges when accessing the a resource it would be very simple to "map" a range request into a specific object "ID" but the issue is that there are scenarios which the client ask for multiple ranges in the same request and everything get a bit complicated.
From a cache(squid) point of view when a client runs a "fetch" operation he also populate the cache.
This is the most "preferred" way of handling cache population since it relies on a need which is somehow considered as required.
Now when you look at it, in many cases it's a bit silly and can be considered in a way simple minded when you are talking about GB's of static content.
When I look at MS updates I see lots of "Cache-Control: public,max-age=172800" in responses and it might be based on the assumption that the object is predicted to be a part of an update "torrent" of about 48 hours.
The real world is far from this header and caches needs to be smarter in order to avoid re-population and re-deletion of existing content.
Now since MS updates would be probably used over and over again by real clients it's sometimes would be good to just store them for a period of time.
For example there aren’t many Windows XP out there under paid support but if clients are still updating then it's right to have them.
Now what I did was simply wrote a simple web-service which is also a forward proxy that is based on another file system rather compared to the standard ones.
You have a storedata directory which can be changed in the command line.
You have a listening port, and you also have some level of debug info.
The store data have three important sub directories:
request/v1/
header/v1/
response/v1/
Since it's a simple web service that relies on a specific file system structure it doesn't have the TOS and QOS features that are in a much lower level services have.
Since you have full control on the web-service and the response headers are reliable you can safely use some kind of Internal response headers and be sure that MS and their CDN network will not use these and will "harm" your statistics.
You will just need to use the concept which was mentioned in the HIT-MISS thread from 2012:
acl mshit rep_header X-Company-Ms-Cache HIT
clientside_tos 0x30 mshit
And you can get wild with the name to verify that it will be 100% unique and will not collide with the CDN headers.
Also you can use another term then HIT for example "INTERNAL-CACHE-SERVER" would probably not be coming from the up-stream CDN.
Or even add a unique ID(#uuidgen) for this service that should never be mimicked.
Since it's a web-service with static header files you will just need to use the perl script which I sent you in the last email to inject these headers into the response files.
If the store-service is only piping the connection to the up-stream services the response will not include your customized embedded headers.
The response headers are at:
/var/storedata/header/v1/*
The bodies are at:
/var/storedata/response/v1/*
Just as a side note: this store service was designed for MS updates only and using it for other services is prohibited in the code level.
In the mean while I will look at the TOS\QOS options if at all.
Eliezer
----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer@xxxxxxxxxxxx
-----Original Message-----
From: squid-users [mailto:squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx] On Behalf Of Omid Kosari
Sent: Sunday, July 17, 2016 9:34 AM
To: squid-users@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: Windows Updates a Caching Stub zone, A windows updates store.
Dear Eliezer,
Thanks for reply .
1. I am trying to understand but with your description it would be simpler
2. I already use logical volumes . Silly question
3. I don't want just a hit in log . I try to describe my need . Currently we
have exclude the cache hits (based on TOS value) from our customers reserved
bandwidth . For example you have 150Mbps internet link from our company and
we have limitation for you on our QOS routers . But we have excluded cache
hits from your 150M and you may have more than that if you are downloading
from our cache hits .
qos_flows local-hit=0x90
qos_flows sibling-hit=0x90
qos_flows parent-hit=0x90
But the hits from your code could not be counted . Even you may help me to
do that with linux iptables+squid it would be fine .
Thanks
--
View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Windows-Updates-a-Caching-Stub-zone-A-windows-updates-store-tp4678454p4678530.html
Sent from the Squid - Users mailing list archive at Nabble.com.
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
