On 9/16/19 10:37 AM, Felipe Arturo Polanco wrote: > We would like to add some logic to our custom made ICAP server, one of > these logics is to analyze up to 10MB of data of a given file and if the > file is larger than that, squid should not keep sending it to icap, > basically, a 204 message should be returned. > squid has a cap of 64K of preview size that limit us in the aspect. True. > Is there a way to extend this limit? Modifying the source code perhaps? Yes, of course. > If so, what are the disadvantages of doing so? Simply increasing BodyPipe::MaxCapacity might open you up to difficult-to-find vulnerabilities where some old Squid code still assumes that the buffers are limited by 64KB and crashes/asserts when that assumption becomes false. FWIW, I have not audited all BodyPipe-related code and do not know whether such bad code exists today. Increasing BodyPipe::MaxCapacity will also increase Squid RAM requirements if your Squid receives requests (and/or ICAP REQMOD-generated HTTP responses) that exceed 64KB body limit. I am not sure about regular responses -- their accumulation is limited by other/independent restrictions. > Would it require more RAM per response [You should worry about "per message" accumulation -- requests will use the same limit, even if you do not have REQMOD services.] Yes, but only for messages that exceed the limit. > or is RAM dynamically allocated as the file is being received? Yes, the RAM in question is allocated dynamically on "as needed" basis. YMMV, but Squid may slow down a lot while dealing with the associated reallocations because of the too-small 64KB increment used for each such reallocation. An alternative solution to consider here is extending ICAP features to allow for splicing of the 10MB prefix (analyzed _and_ returned back by the ICAP server) and the remaining virgin response suffix (paused without excessive accumulation on the Squid side). This would be similar to the existing ICAP 206 use-original-body="10MB" feature[1], but would require additional negotiation and changes to disable storage of the 10MB prefix on the Squid side. This alternative can be implemented without worrying about hidden 64KB body buffer assumptions, but the implementation will not be trivial. [1] http://www.icap-forum.org/documents/specification/draft-icap-ext-partial-content-07.txt HTH, Alex. _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
