Facing an issue with a portal throwing socket Timeout Exception

[Vasudevan Comandur]

Hi,

I am using HTMLUnit version 2.23 which has httpclient 4.5.2 and httpcore
4.4.4
bundled along with that.

I am connecting to a job board and after successful login, the client
application
fails with socket exception which waiting for the response for a GET
request. The
GET request was sent from the client 4 times to the server. I took the
http wire
frame log and I am attaching herewith for your analysis.

I also took the wireshark trace to find out if there is any error at the
TCP
layer. I saw the response was coming from the server all the 4 times.

I can share the wireshark trace file for your analysis as well. Wireshark
version is 2.2.1.

I would appreciate if you could tell me why it was failing and this
issue has
become critical and what is the solution.

Thanks for your help in advance

Thanks & Regards.
Vasu

[Oleg Kalnichevski]

Hi,
   I am using HTMLUnit version 2.23 which has httpclient 4.5.2 and
httpcore 4.4.4
   bundled along with that.
   I am connecting to a job board and after successful login, the
client application
   fails with socket exception which waiting for the response for a
GET request. The
   GET request was sent from the client 4 times to the server.  I 
took the http wire 
   frame log and I am attaching  herewith for your analysis.
   I also took the wireshark trace to find out if there is any error
at the TCP 
   layer. I saw the response was coming from the server all the 4
times.
   I can share the wireshark trace file for your analysis as well.
Wireshark
   version is 2.2.1.
   I would appreciate if you could tell me why it was failing and
this issue has 
   become critical and what is the solution.
    Thanks for your help in advance

---
2017-08-24 21:38:34,884 [pool-4-thread-1] DEBUG http.wire - http-outgoing-4 >> "GET /jobposter/mycb.aspx HTTP /1.1[\r][\n]"
...
2017-08-24 21:38:34,885 [pool-4-thread-1] DEBUG http.wire - http-outgoing-4 >> "[\r][\n]"
2017-08-24 21:38:36,303 [pool-4-thread-1] DEBUG http.wire - http-outgoing-4 << "end of stream"
---

I clearly see the socket returning -1 (end of stream) pretty much
immediately after the request which can basically mean two things:

1. server failed to respond with a HTTP status due to an internal error
2. the connection got stale while kept alive in the pool

Oleg  


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-***@hc.apache.org
For additional commands, e-mail: httpclient-users-***@hc.apache.org

[Vasudevan Comandur]

Hi Oleg,

Thanks for your reply. When I had a look at the wireshark trace, I saw
that server was sending the data
in chunks and all the chunks were assembled finally with HTTP headers.
The TCP flags had [FIN, PSH, ACK]
set for the HTTP protocol and the data was also there. The retry happend
4 times and the data was coming
all the tme from the server with same TCP flags settings.

I saw TCP packet from client with [ACK] flag followed by another TCP
packet with [RST, ACK] flags.
But the app did not receive the message.

I am running JDK 1.8.0.102 and OS is windows 8.

I can share the wireshark log trace if you need.

By changing the java version, will it help to solve the issue.

Looking forward to your suggestion.

Thanks & Regards
Vasu

Post by Oleg Kalnichevski

Post by Vasudevan Comandur
Hi,
I am using HTMLUnit version 2.23 which has httpclient 4.5.2 and
httpcore 4.4.4
bundled along with that.
I am connecting to a job board and after successful login, the
client application
fails with socket exception which waiting for the response for a
GET request. The
GET request was sent from the client 4 times to the server. I
took the http wire
frame log and I am attaching herewith for your analysis.
I also took the wireshark trace to find out if there is any error
at the TCP
layer. I saw the response was coming from the server all the 4 times.
I can share the wireshark trace file for your analysis as well. Wireshark
version is 2.2.1.
I would appreciate if you could tell me why it was failing and
this issue has
become critical and what is the solution.
Thanks for your help in advance

---
2017-08-24 21:38:34,884 [pool-4-thread-1] DEBUG http.wire -
http-outgoing-4 >> "GET /jobposter/mycb.aspx HTTP /1.1[\r][\n]"
...
2017-08-24 21:38:34,885 [pool-4-thread-1] DEBUG http.wire -
http-outgoing-4 >> "[\r][\n]"
2017-08-24 21:38:36,303 [pool-4-thread-1] DEBUG http.wire -
http-outgoing-4 << "end of stream"
---
I clearly see the socket returning -1 (end of stream) pretty much
1. server failed to respond with a HTTP status due to an internal error
2. the connection got stale while kept alive in the pool
Oleg
---------------------------------------------------------------------

[Oleg Kalnichevski]

Post by Vasudevan Comandur
Hi Oleg,
   Thanks for your reply. When I had a look at the wireshark trace, I
saw
that server was sending the data
   in chunks and all the chunks were assembled finally with HTTP
headers.
The TCP flags had [FIN, PSH, ACK]
  set for the HTTP protocol and the data was also there. The retry
happend
4 times and the data was coming
   all the tme from the server with same TCP flags settings.
    I saw TCP packet from client with [ACK] flag followed by another
TCP
packet with [RST, ACK] flags.
    But the app did not receive the message.
    I am running JDK 1.8.0.102 and OS is windows 8.
    I can share the wireshark log trace if you need.
     By changing the java version, will it help to solve the issue.
     Looking forward to your suggestion.
Thanks & Regards
  Vasu

Maybe the server sent a response, maybe it did not. What looks pretty
clear to me is that the JRE believes the socket was closed by an
opposite endpoint without returning any data.

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-***@hc.apache.org
For additional commands, e-mail: httpclient-users-***@hc.apache.org

[Vasudevan Comandur]

Thanks Oleg for your reply.

The Wireshark trace shows clearly that data was coming in chunks/frames and
also got assembled finally with HTTP headers.
Can JRE close the socket on its own when it receives TCP Flags [FIN, PSH,
ACK] in the frame without the caller closing the socket?

Is there any other solution other than trying with different JDK versions?

Regards
Vasu

Post by Oleg Kalnichevski

Post by Vasudevan Comandur
Hi Oleg,
Thanks for your reply. When I had a look at the wireshark trace, I saw
that server was sending the data
in chunks and all the chunks were assembled finally with HTTP headers.
The TCP flags had [FIN, PSH, ACK]
set for the HTTP protocol and the data was also there. The retry happend
4 times and the data was coming
all the tme from the server with same TCP flags settings.
I saw TCP packet from client with [ACK] flag followed by another TCP
packet with [RST, ACK] flags.
But the app did not receive the message.
I am running JDK 1.8.0.102 and OS is windows 8.
I can share the wireshark log trace if you need.
By changing the java version, will it help to solve the issue.
Looking forward to your suggestion.
Thanks & Regards
Vasu

Maybe the server sent a response, maybe it did not. What looks pretty
clear to me is that the JRE believes the socket was closed by an
opposite endpoint without returning any data.
Oleg
---------------------------------------------------------------------

[Oleg Kalnichevski]

Post by Vasudevan Comandur
Thanks Oleg for your reply.
The Wireshark trace shows clearly that data was coming in
chunks/frames and
also got assembled finally with HTTP headers.
Can JRE close the socket on its own when it receives TCP Flags [FIN, PSH,
ACK] in the frame without the caller closing the socket?

All sorts of things can happen between the OS TCP/IP stack and the
application layer of JRE especially on Windows. I doubt JRE would close
the socket on its own but the OS could.

Post by Vasudevan Comandur
Is there any other solution other than trying with different JDK versions?

1.8.0.102 sounds reasonably stable to me. 

What I would do though is to make sure that connections do not stay
alive too long by imposing a total time to live, say, under 30 seconds,
as false NoHttpResponseException tend to occur with connections that
stay idle for some while.

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-***@hc.apache.org
For additional commands, e-mail: httpclient-users-***@hc.apache.org

[Vasudevan Comandur]

Post by Oleg Kalnichevski

Post by Vasudevan Comandur
Thanks Oleg for your reply.
The Wireshark trace shows clearly that data was coming in
chunks/frames and
also got assembled finally with HTTP headers.
Can JRE close the socket on its own when it receives TCP Flags [FIN, PSH,
ACK] in the frame without the caller closing the socket?

All sorts of things can happen between the OS TCP/IP stack and the
application layer of JRE especially on Windows. I doubt JRE would close
the socket on its own but the OS could.

Are you saying that Linux platform is better?. Unfortunately, some
clients use
Windows environment only.

Post by Oleg Kalnichevski

Post by Vasudevan Comandur
Is there any other solution other than trying with different JDK versions?

1.8.0.102 sounds reasonably stable to me.
What I would do though is to make sure that connections do not stay
alive too long by imposing a total time to live, say, under 30 seconds,
as false NoHttpResponseException tend to occur with connections that
stay idle for some while.

I am using HTMLUnit and how do I set the ttl value?. Is there a way to
set the value?

Regards
Vasu

Post by Oleg Kalnichevski
Oleg
---------------------------------------------------------------------