Search squid archive

Re: Squid full request logging

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hey Niels,

Take a peek at:
https://github.com/andybalholm/redwood

I am using it in production and it was written because of squid limitations.
squid is great but take a peek and see how it works for you.
I have 2 servers in ha cluster which works great.

An example I wrote to filter youtube traffic is at:
https://github.com/elico/yt-classification-service-example

let me know if it helps you or gives you any direction.

בתאריך יום ה׳, 4 במרץ 2021, 23:33, מאת Niels Hofmans ‏<hello@xxxxxxxxxxx>:
Hi Alex,

Thanks for the feedback. Although I am not proficient in C for writing an ecap service, is there some binding available online for Go?
This was the reason I originally opted for an ICAP service since I can abstract Go behind the HTTP ICAP layer.
Now I understand this has its limitations, but AFAIK a preview cap at 100kb would be sufficient per request.
But this will slow down my current setup greatly, as I’m currently sending -only- the headers.

Would you think that a) using Go for the ecap adapter or b) using two ICAP services.
One would validate the headers and return OK or NOT (bypass=0), while the other only pushes the 1kb request/response to a queue.
Ideally those two would be contacted simultaneously while only the first one is blocking.
..just thinking aloud tough.

Regards,
Niels Hofmans

SITE   https://ironpeak.be
BTW   BE0694785660
BANK BE76068909740795

On 4 Mar 2021, at 22:23, Alex Rousskov <rousskov@xxxxxxxxxxxxxxxxxxxxxxx> wrote:

On 3/4/21 2:52 PM, Niels Hofmans wrote:

is it possible to do full request/response logging?

Squid can log HTTP headers with %>h and %<h logformat codes.

Squid cannot log HTTP message bodies.


I do not see the appropriate log_format directive in the docs.
I was hoping not having to do this in my ICAP service since this slows
down approval of the HTTP request. (Empty preview v.s. a request capped
at 1MB that needs to be sent over every time)

FWIW, an ICAP or eCAP service can start responding to the request
_before_ the service receives the entire HTTP message body. To get
things going, all the service needs is HTTP headers (and even that is,
technically, optional in some cases).

Using an adaptation service is still an overhead, of course, but, very
few legitimate Squid use cases involve logging message bodies, so there
is no built-in mechanism optimized for that specific rare purpose
(yet?). The fastest option available today is probably a dedicated eCAP
service that refuses to adapt the message bit continues to receive (and
log) the message body.


HTH,

Alex.
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux