On 1/18/21 6:45 AM, Moti Berger wrote: > If the ICAP server sets 'Preview: 0' in the OPTIONS it means that when > the ICAP client sends a request, it should not contain the body. The above summary may mislead many readers. I would describe the protocol operation differently: * Preview in an OPTIONS response indicates that the server supports Preview in general and specifies the maximum Preview size the client should use (e.g., Preview:0 limits Preview to HTTP headers). * The Preview mode for a specific REQMOD or RESPMOD transaction is signaled in the corresponding REQMOD or RESPMOD request (not a previous OPTIONS response) by adding a Preview:N ICAP request header (Preview:0 specifies a headers-only Preview for the current transaction). * The REQMOD or RESPMOD transaction with a Preview:0 request header is split into two phases. During the first phase, the client must not send the virgin body. During the second phase, if any, the client must send the virgin body. Both phases comprise a single ICAP transaction, with a single ICAP request and a single ICAP response. Thus, one cannot say that this transaction (as a whole) "should not contain a body". > This is the REQMOD request: > > F..n...DREQMOD icap://censor-req.proxy:14590/request ICAP/1.0 > Host: censor-req.proxy:14590 > Date: Mon, 18 Jan 2021 11:34:54 GMT > Encapsulated: req-hdr=0, req-body=222 > Preview: 0 > Allow: 204, trailers > X-custom-header: data > > POST http://www.dst-server.com:22222/v1/test HTTP/1.1 > User-Agent: python-requests/2.25.1 > Accept-Encoding: gzip, deflate > Accept: */* > Content-Length: 10 > Content-Type: application/json > Host: www.dst-server.com:22222 <http://www.dst-server.com:22222> > The ICAP 'Encapsulated' header has a req-body even though no 'body' > should be in this request. Not exactly. The request may not be over at this point. Please see my third bullet above for details. > The ICAP server has no way to encapsulate the HTTP request body if it > didn't get it. To get the request body in Preview:0 mode, the ICAP server must respond with ICAP 100 (Continue). > I want to avoid sending the body because the adaptation is body agnostic. Yes, I know, but you have to work within the ICAP protocol boundaries. ICAP simply does not optimize your use case! After you have the basics working well, you can invest in implementing a use-original-body ICAP extension[1] that, in _some_ cases, can prevent the body exchange while adapting HTTP headers. Alternatively, you can use an existing (extendible) ICAP server to do the legwork for you [2]. Many individuals and companies have learned the hard way that implementing an ICAP service correctly from scratch is very difficult and often prohibitively expensive. [1] http://www.icap-forum.org/documents/specification/draft-icap-ext-partial-content-07.txt [2] https://wiki.squid-cache.org/Features/ICAP#ICAP_Servers HTH, Alex. > On Sun, Jan 17, 2021 at 11:34 PM Alex Rousskov wrote: > > On 1/17/21 3:08 PM, Moti Berger wrote: > > What should the ICAP response look like? > > The vast majority off ICAP responses containing an HTTP POST message > will look like ICAP header + HTTP header + HTTP body. Please see RFC > 3507 and its errata for examples of and discussion about those three > components. It should help avoid guessing and developing by examples > (which usually leads to bugs, especially where ICAP is involved). > > > > What I do is to reply like this: > > > > (dI./M..ICAP/1.0 200 OK > > ISTag: "SjIzlRA4te41axxcDOoiSl6rBRg4ZK" > > Date: Sun, 17 Jan 2021 19:34:12 GMT > > Server: BaseICAP/1.0 Python/3.6.12 > > Encapsulated: req-hdr=0, req-body=360 > > > > POST http://www.dst-server.com:22222/v1/test HTTP/1.1 > > x-new-header: {"key": "value"} > > user-agent: python-requests/2.25.1 > > accept-encoding: gzip, deflate > > accept: */* > > content-length: 16 > > content-type: application/json > > host: www.dst-server.com:22222 > <http://www.dst-server.com:22222> <http://www.dst-server.com:22222> > > > FYI: The above incomplete ICAP response promises an HTTP request body, > both on the ICAP level (req-body) and on the HTTP level (content-length: > 16). > > > > As I said, I use 'Preview: 0' since I don't mind the body. The > question > > is whether declaring the body starts at X (req-body=X) is OK even > though > > I don't have a body to send? > > It is not OK not to send the body. Encapsulated:req-body does more than > declaring where the encapsulated headers end. It also promises an > embedded HTTP body after those headers. You must encapsulate the body if > the HTTP message should have one. You cannot adapt the header of an HTTP > message with a body without also sending the HTTP body (virgin or > adapted). > > Preview is pretty much irrelevant in this context -- the ICAP protocol > does not care how the ICAP service gets the HTTP body to include in the > ICAP response. > > There are unofficial ICAP extensions that make it possible to tell the > ICAP client to reuse the body it has buffered while adapting the header, > but you should get the baseline case working before bothering with those > extensions -- they are optimizations that are not applicable to some > transactions. > > > > I think having req-null=X is bad since it > > probably tells squid that I decided the adapted request should have no > > body, but that's only a guess. > > If you meant to say "null-body", then you guessed correctly -- null-body > means the adapted HTTP message has no body. That is not what you want to > say when adapting most HTTP POST messages. > > > HTH, > > Alex. > _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
