One interesting thing we did find on a packet capture on the back end between the proxy server and the application server is that 3 seconds after the proxy server apparently closed the port, the Weblogic application server tried to send data. It almost like Weblogic did not realize the port was closed. Very odd.
At this point I may try implementing OHS and see if we have the same results. However it is based on Apache 2.2, and I was hoping to take advantage of the improved performance offered by Apache 2.4. Also reading through the documentation it looks like a PITA to install (we have to install X to the run "Oracle Universal Installer" ? Seriously ?) Maybe Oracle support will take us more seriously and give us better support if we are using all their products.
-Joe
-----Original Message-----
From: Rainer Jung [mailto:rainer.jung@xxxxxxxxxxx]
Sent: Saturday, June 25, 2016 4:45 AM
To: users@xxxxxxxxxxxxxxxx
Subject: Re: Issues migrating Weblogic proxies from Sun One 6.1 to Apache 2.4
Am 24.06.2016 um 21:58 schrieb Joe Muller:
>
> Just an update on this issue, if anybody is following. We have eliminated the Load Balancer and the Firewall as a possible cause. Our last test was directly between the IE client browser and the Apache 2.4 proxy server, resulting in the same errors in the Apache logs similar to this:
>
>>> *****Exception type] [WRITE_ERROR_TO_CLIENT raised at line 560 of
>>> BaseProxy.cpp, refer:
>>> https://intgalf.xyz.com/ALFA/selectFileType.do?fileType=O1MM
>
> We also tested using different browsers (IE, Fire Fox, Chrome, and had mixed results, but eventually we saw the same errors from the Weblogic 12c plug-in the Apache logs at some point.
>
> So at this point it is down to the Apache proxy and/or the Oracle
> Weblogic 12c Plug-in for Apache. Anybody have any ideas ? I am out
> of them right now :-(
>
> Perhaps it has something to do with the Apache 2.4 compilation options ? If I had known before that going with Apache 2.4 instead of IPlanet would have been this difficult we would have just purchased Oracle HTTP Server. At least we would have full Oracle support through and through, although that may not be saying much, since it is based on Apache 2.2. Also I am worried about Oracle compatibility with 3rd party plug-ins like RSA Access Manager (SSO) and Tea Leaf.
>
> To be honest I am surprised Oracle doesn't give more guidance on this. I would think we wouldn't be the first people to encounter this, but based on the lack of information in their knowledge base and the response from their technical support it seems otherwise.
I'd sniff the network in front of Apache and when the problem happens filter the raw sniff packet dump file using the connection information from the error log line ("[client 10.165.254.1:28171]"). Then use the time stamp of the error message and look at the sniff of the single connection what happens before the error is logged.
In case you are using https and you can switch on a reproduction system to http this might help understanding the packet dump, but in general this approach could also be used for https.
Regards,
Rainer
> -----Original Message-----
> From: Mike Rumph [mailto:mike.rumph@xxxxxxxxxx]
> Sent: Monday, June 20, 2016 4:08 PM
> To: users@xxxxxxxxxxxxxxxx
> Subject: Re: Issues migrating Weblogic proxies from Sun
> One 6.1 to Apache 2.4
>
> Hello Joe,
>
> It seems to me that mod_reqtimeout applies to the amount of time a allowed for the server to receive the request from the client (Read from client).
>
> But your errors are indicating a problem on the Write to client.
> As I am understanding our WLS plugin developer, the problem seems to be that when the server has readied a response to send back to the client, the client has already dropped the connection.
> So in your topology this could be client browser, firewall or load balancer.
>
> Have you considered the email responses from Todd Simons and Daniel?
> The response from Daniel concerning the load balancer may be the most pertinent.
>
> Thanks,
>
> Mike
>
> On 6/20/2016 11:11 AM, Joe Muller wrote:
>> I checked our httpd.conf and we do have reqtimeout_module already loaded, however there is nothing set. According to the Apache 2.4 documentation the default values are:
>>
>> Default: header=20-40,MinRate=500 body=20,MinRate=500
>>
>> Wouldn't you think that should be high enough ? I know for a fact
>> that the data is not taking 20-40 seconds to post in these
>> applications, as the users are not waiting that long. They are only
>> seeing slowdowns of a few seconds between pages (which in network
>> time is pretty long, ecspecially accessing the applications locally
>> and not coming in from over the internet.)
>>
>>
>> -----Original Message-----
>> From: Mike Rumph [mailto:mike.rumph@xxxxxxxxxx]
>> Sent: Monday, June 20, 2016 10:45 AM
>> To: users@xxxxxxxxxxxxxxxx
>> Subject: Re: Issues migrating Weblogic proxies from Sun
>> One 6.1 to Apache 2.4
>>
>> Hello Joe,
>>
>> I am not in a position to offer an official Oracle statement for your situation.
>> But I do work closely with the owners of the WLS plug-ins.
>>
>> Here is an initial evaluation from one of the developers:
>>
>> "WRITE_ERROR_TO_CLIENT is typically seen when there is an error
>> writing the response to the client (WLS plug-in client). This usually
>> occurs when the user sends a request, but closes the browser (or hits
>> a stop
>> button) before the response is received by the client. In such a scenario, from the plug-in perspective, whenever response is received from WLS, it cannot relay it to the client as the connection is broken, and it logs WRITE_ERROR_TO_CLIENT error. This is usually a harmless error. If the above is not true (closing the browser etc), then it may be possible that the client timeout is too low (or lower than the WLS response time for the request). In such cases, the timeout needs to be increased. I am not aware of what parameters to look out for here, but mod_reqtimeout may be a good beginning."
>>
>> I hope that this is helpful to you.
>> For more specific details, you would still need to contact official Oracle support channels.
>>
>> Thanks,
>>
>> Mike
>>
>> On 6/17/2016 7:30 AM, Joe Muller wrote:
>>> I am working on a project to migrate all our IPlanet 6.1 SP19 webserver proxies (formerly Sun One) to Apache 2.4, since IPlanet 6.1 does not support TLS 1.2 and IPlanet 7.0 is being EOL. Our backend application servers are Weblogic 9.2 / Weblogic 12c. The IPlanet proxies have performed FLAWLESSLY for over 10 years, despite the product being no longer supported and their WL Plug-in not officially supported with Weblogic 12c.
>>>
>>> However now that we are trying to use a more supported configuration (self-compiled Apache 2.4.18 running Weblogic Server Plugin 12.1.3) we are constantly seeing these errors, which results in performance degradation for our applications, and in some cases I think maybe even lost data.
>>>
>>>
>>> [Tue Jun 14 09:27:36.239682 2016] [weblogic:error] [pid 12513:tid
>>> 140185150932736] [client 10.165.254.1:28171] <1251314659108487>
>>> Write to the client failed: calling URL::close at line 559 of
>>> BaseProxy.cpp,
>>> referer:
>>> https://intgalf.xyz.com/ALFA/selectFileType.do?fileType=O1MM
>>>
>>> [Tue Jun 14 09:27:36.239747 2016] [weblogic:error] [pid 12513:tid 140185150932736] [client 10.165.254.1:28171] <1251314659108487> **
>>> *****Exception type] [WRITE_ERROR_TO_CLIENT raised at line 560 of
>>> BaseProxy.cpp, referer:
>>> https://intgalf.xyz.com/ALFA/selectFileType.do?fileType=O1MM
>>>
>>> [Tue Jun 14 09:27:36.239952 2016] [weblogic:error] [pid 12513:tid
>>> 140185150932736] [client 10.165.254.1:28171] <1251314659108487>
>>> request
>>> [/ALFA/servlet/DecryptDownload?linkName=al_o1mm_carr20150630.csv]
>>> did NOT process successfully..................,
>>> referer:https://intgalf.xyz.com/ALFA/selectFileType.do?fileType=O1MM
>>>
>>>
>>>
>>> Our topology is like this:
>>>
>>> Client Browser <--> Firewall <--> Load Balancer <--> Web Proxies
>>> <--> Firewall <--> Weblogic Application Servers
>>>
>>> Oracle support suggested as work around that we increase WLSocketsTimeOut in the plug-in, but I think that only masks the issue, as we still the errors.
>>>
>>> We did a network trace and it looks like the Apache plug-in is pre-maturely closing the connection to the WL server, but I can't be certain. We know that our firewall is not responsible.
>>>
>>> Any ideas ? I thought Apache would work better then Sun One, but this has been the opposite. Is there some fundamental webserver tunable parameter that is so different between out of the box Sun One and out of the box Apache that could be causing this ?
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
|