Search squid archive

Re: Blank pages off www.msn.com

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I've set "forwarded_for off" and now msn.com search results load.

We also had the same problem with their site http://maps.live.com.  It
also fixed that one as well.

The other suggestion of adding an acl for "header_access
Accept-Encoding deny" for msn did not work for us.

Thank you Andrew for providing deeper debugging and the solution.

M$ must be up to something...


-Bill

On Fri, Mar 27, 2009 at 9:34 AM, Andrew Farr <a.farr@xxxxxxxxxxxx> wrote:
> wrh@xxxxxxxxxxx wrote:
>> Hello,
>>
>> This just started happening -
>>
>> We are getting blank pages returned from our Squid 2.6.STABLE4 proxy
>> from the search results pages off www.msn.com.  And MSN's
>> maps&directions site comes up blank as well.  The www.msn.com home
>> page is fine.
>>
>> Happening with IE7 and FF3.
>>
>> No explanation for it on the squid server, nothing in the logs.
>>
>> If I don't use squid, the pages appear.
>>
>> Our squid has been unchanged for a long while.
>>
>> Is anyone else experiencing this issue?
>>
>>
>>
>> Thanks,
>>
>> -Bill
>>
> I may be wrong, but it looks like MSN explicitly refuses proxied
> connections. If I hide the proxy with "forwarded_for off" it suddenly works!
>
> The following shows radically different headers depending on whether the
> X-Forwarded-For header is present or not.
>
> Trying the URL
> http://search.msn.com/results.aspx?q=debian+lenny+calamaris&FORM=MSNH11
> using wget -S
>
> First with standard proxy settings
> ==================================
> --2009-03-27 14:10:40--
> http://search.msn.com/results.aspx?q=debian+lenny+calamaris&FORM=MSNH11
> Resolving localhost... 127.0.0.1
> Connecting to localhost|127.0.0.1|:8080... connected.
> Proxy request sent, awaiting response...
>  HTTP/1.1 200 OK
>  X-Powered-By: ASP.NET
>  Content-Length: 0
>  Expires: Fri, 27 Mar 2009 14:10:40 GMT
>  Cache-Control: max-age=0, no-cache, no-store
>  Pragma: no-cache
>  Date: Fri, 27 Mar 2009 14:10:40 GMT
>  X-Cache: MISS from cpc3-staf1-0-0-cust235.sol2.cable.ntl.com
>  Via: 1.1 cpc3-staf1-0-0-cust235.sol2.cable.ntl.com:8080
> (squid/2.7.STABLE6)
>  Connection: close
> Length: 0
> Saving to: `results.aspx?q=debian+lenny+calamaris&FORM=MSNH11'
>
>    [
> <=>
>
> ] 0           --.-K/s   in 0s
>
> 2009-03-27 14:10:40 (0.00 B/s) -
> `results.aspx?q=debian+lenny+calamaris&FORM=MSNH11' saved [0/0]
>
> Without the proxy
> =================
> --2009-03-27 14:10:56--
> http://search.msn.com/results.aspx?q=debian+lenny+calamaris&FORM=MSNH11
> Resolving search.msn.com... 84.53.178.209, 84.53.178.218
> Connecting to search.msn.com|84.53.178.209|:80... connected.
> HTTP request sent, awaiting response...
>  HTTP/1.0 200 OK
>  Content-Length: 31048
>  Content-Type: text/html; charset=utf-8
>  X-Powered-By: ASP.NET
>  P3P: CP="NON UNI COM NAV STA LOC CURa DEVa PSAa PSDa OUR IND",
> policyref="http://privacy.msn.com/w3c/p3p.xml";
>  Date: Fri, 27 Mar 2009 14:10:56 GMT
>  Connection: keep-alive
>  Set-Cookie: AFORM=MSNH11; expires=Mon, 20-Jul-2015 23:59:59 GMT; path=/
>  Set-Cookie: mkt1=norm=-GB; domain=.msn.com; path=/
>  Set-Cookie: mkt2=ui=en-GB; domain=search.msn.com; path=/
>  Set-Cookie: MUID=95278E835C21487EB0C3B5390C5735BE; expires=Mon,
> 20-Jul-2015 23:59:59 GMT; domain=.msn.com; path=/
>  Set-Cookie: OrigMUID=95278E835C21487EB0C3B5390C5735BE; expires=Mon,
> 20-Jul-2015 23:59:59 GMT; domain=.msn.com; path=/
>  Set-Cookie:
> OVR=flt=0&PerfTracking=0&DomainVertical=0&Cashback=0&MSCorp=kiev;
> domain=.msn.com; path=/
>  Set-Cookie: RMS=SERP=320&T=48; path=/
>  Set-Cookie: SRCHD=D=650290&AF=MSNH11; expires=Mon, 20-Jul-2015
> 23:59:59 GMT; domain=.msn.com; path=/
>  Set-Cookie:
> SRCHSESS=GUID=659B63EAB2A64E9597884654B10FF49E&TS=1238163056;
> expires=Fri, 27-Mar-2009 14:30:56 GMT; path=/
>  Set-Cookie: SRCHUID=V=2&GUID=83EF0003D92A4C429D5D6DBCA44B86C8;
> expires=Mon, 20-Jul-2015 23:59:59 GMT; path=/
>  Set-Cookie: SRCHUSR=AUTOREDIR=0&GEOVAR=&DOB=20090327; expires=Sun,
> 27-Mar-2011 14:10:56 GMT; path=/
>  Cache-Control: private
> Length: 31048 (30K) [text/html]
> Saving to: `results.aspx?q=debian+lenny+calamaris&FORM=MSNH11.1'
>
> 100%[==========================================================================================>]
> 31,048       199K/s   in 0.2s
>
> 2009-03-27 14:10:57 (199 KB/s) -
> `results.aspx?q=debian+lenny+calamaris&FORM=MSNH11.1' saved [31048/31048]
>
> Now set "forwarded_for off" in squid.conf
> =========================================
> --2009-03-27 14:27:51--
> http://search.msn.com/results.aspx?q=debian+lenny+calamaris&FORM=MSNH11
> Resolving localhost... 127.0.0.1
> Connecting to localhost|127.0.0.1|:8080... connected.
> Proxy request sent, awaiting response...
>  HTTP/1.1 200 OK
>  Content-Length: 31048
>  Content-Type: text/html; charset=utf-8
>  X-Powered-By: ASP.NET
>  P3P: CP="NON UNI COM NAV STA LOC CURa DEVa PSAa PSDa OUR IND",
> policyref="http://privacy.msn.com/w3c/p3p.xml";
>  Date: Fri, 27 Mar 2009 14:27:51 GMT
>  Set-Cookie: AFORM=MSNH11; expires=Mon, 20-Jul-2015 23:59:59 GMT; path=/
>  Set-Cookie: mkt1=norm=-GB; domain=.msn.com; path=/
>  Set-Cookie: mkt2=ui=en-GB; domain=search.msn.com; path=/
>  Set-Cookie: MUID=71DF4074D8BE456381CDA9945EE5AE96; expires=Mon,
> 20-Jul-2015 23:59:59 GMT; domain=.msn.com; path=/
>  Set-Cookie: OrigMUID=71DF4074D8BE456381CDA9945EE5AE96; expires=Mon,
> 20-Jul-2015 23:59:59 GMT; domain=.msn.com; path=/
>  Set-Cookie:
> OVR=flt=0&PerfTracking=0&DomainVertical=0&Cashback=0&MSCorp=kiev;
> domain=.msn.com; path=/
>  Set-Cookie: RMS=SERP=320&T=48; path=/
>  Set-Cookie: SRCHD=D=650307&AF=MSNH11; expires=Mon, 20-Jul-2015
> 23:59:59 GMT; domain=.msn.com; path=/
>  Set-Cookie:
> SRCHSESS=GUID=E8E98A7C50064824A06EC246214D405D&TS=1238164071;
> expires=Fri, 27-Mar-2009 14:47:51 GMT; path=/
>  Set-Cookie: SRCHUID=V=2&GUID=BE07F5751A95407FB6E94683FB41EDBB;
> expires=Mon, 20-Jul-2015 23:59:59 GMT; path=/
>  Set-Cookie: SRCHUSR=AUTOREDIR=0&GEOVAR=&DOB=20090327; expires=Sun,
> 27-Mar-2011 14:27:51 GMT; path=/
>  Cache-Control: private
>  X-Cache: MISS from cpc3-staf1-0-0-cust235.sol2.cable.ntl.com
>  Via: 1.1 cpc3-staf1-0-0-cust235.sol2.cable.ntl.com:8080
> (squid/2.7.STABLE6)
>  Connection: close
> Length: 31048 (30K) [text/html]
> Saving to: `results.aspx?q=debian+lenny+calamaris&FORM=MSNH11.2'
>
> 100%[==========================================================================================>]
> 31,048      --.-K/s   in 0.1s
>
> 2009-03-27 14:27:51 (214 KB/s) -
> `results.aspx?q=debian+lenny+calamaris&FORM=MSNH11.2' saved [31048/31048]
>
>
>


[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux