Discussion:
HttpClient with PoolingClientConnectionManager throws read time out
Jose Escobar
12 years ago
Permalink
Hello,

I'm using httpclient on a spring aplication to send http posts.
I have a shared singleton bean instance of DefaultHttpClient that I
use to execute httpPost, this DefaultHttpClient have a
PoolingClientConnectionManager and it's configured as show :

...
PoolingClientConnectionManager connectionManager=new
PoolingClientConnectionManager(schemeRegistry, 65,
java.util.concurrent.TimeUnit.SECONDS);

// Increase max total connection to 100
connectionManager.setMaxTotal(100);
// Increase default max connection per route to 15
connectionManager.setDefaultMaxPerRoute(25);
// Increase max connections for localhost:80 to 50
HttpHost localhost = new HttpHost("locahost", 80);
connectionManager.setMaxPerRoute(new HttpRoute(localhost), 50);

...

@Bean
public DefaultHttpClient httpClient(){
DefaultHttpClient httpClient=new DefaultHttpClient(connectionManager());
//httpClient.setParams(params)

HttpParams params = httpClient.getParams();
HttpConnectionParams.setConnectionTimeout(params, HTTPCLIENT_TIMEOUT);
HttpConnectionParams.setSoTimeout(params, HTTPCLIENT_TIMEOUT);

return httpClient;
}



It works correctly until the aplication have to send multiple post at
same time to the same route. It start to throws readTimeOut Exception

java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:61)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:713)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:518)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at com.edicom.edicomnet.asxserver.logic.as2.impl.As2MessageSenderImpl.sendMessage(As2MessageSenderImpl.java:117)
at sun.reflect.GeneratedMethodAccessor575.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy52.sendMessage(Unknown Source)
at com.edicom.edicomnet.asxserver.businessServices.impl.MessageSenderImpl.sendCreatedAS2Message(MessageSenderImpl.java:119)
at sun.reflect.GeneratedMethodAccessor551.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:80)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)


I hope you can help me with this problem (maybe something about concurrency?)


Thank you!
William Speirs
12 years ago
Permalink
Do you have the wire logs?

Bill-
...
Jose Escobar
12 years ago
Permalink
Murphy's law, since I activated wire log there's no timeout errors...

As soon as I get an error I'll mail it
...
Jose Escobar
12 years ago
Permalink
This post might be inappropriate. Click to display it.
Oleg Kalnichevski
12 years ago
Permalink
Post by Jose Escobar
Finally I've captured the wire log of the anomalous situation. There
are multiple executors (threads) trying to send different http posts
Jose

The thing is I/O socket operations as well as socket timeouts are all
handled by the JRE through native calls. So, essentially, i/o operations
can only time out when the server fails to send data within the
specified timeout period. Either your timeout settings are too
aggressive or something happens on the server side that prevents it from
serving requests.

Oleg
...
Jose Escobar
12 years ago
Permalink
I have 60 seconds of timeout, not to aggressive. Maybe it's a limit on
my gnu/linux on outgoing sockets or something like this.

However I found a strange value for keep-alive time on logs:

o.a.h.i.c.PoolingClientConnectionManager - Connection [id: 220][route:
{}->http://yyyyyy.com] can be kept alive for 9223372036854775807
MILLISECONDS

This time wasn't at request header and I have a limit of 15 secs when
no keep-alive time header is received:

httpClient.setKeepAliveStrategy(new DefaultConnectionKeepAliveStrategy() {
@Override
public long getKeepAliveDuration(
HttpResponse response,
HttpContext context) {
long keepAlive = super.getKeepAliveDuration(response, context);
if (keepAlive == -1) {
keepAlive = 15000;
}
return keepAlive;
}

});


It's that normal?
...
Mark Claassen
12 years ago
Permalink
A quick check using an online too shows that that 9223372036854775807 is
111111111111111111111111111111111111111111111111111111111111111 in binary.

-----Original Message-----
From: Jose Escobar [mailto:eb.jose-***@public.gmane.org]
Sent: Wednesday, October 17, 2012 5:24 AM
To: HttpClient User Discussion
Subject: Re: HttpClient with PoolingClientConnectionManager throws read time out

I have 60 seconds of timeout, not to aggressive. Maybe it's a limit on my gnu/linux on outgoing sockets or something like this.

However I found a strange value for keep-alive time on logs:

o.a.h.i.c.PoolingClientConnectionManager - Connection [id: 220][route:
{}->http://yyyyyy.com] can be kept alive for 9223372036854775807 MILLISECONDS

This time wasn't at request header and I have a limit of 15 secs when no keep-alive time header is received:

httpClient.setKeepAliveStrategy(new DefaultConnectionKeepAliveStrategy() {
@Override
public long getKeepAliveDuration(
HttpResponse response,
HttpContext context) {
long keepAlive = super.getKeepAliveDuration(response, context);
if (keepAlive == -1) {
keepAlive = 15000;
}
return keepAlive;
}

});


It's that normal?
...
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe-Bio+***@public.gmane.org
For additional commands, e-mail: httpclient-users-help-Bio+***@public.gmane.org
Oleg Kalnichevski
12 years ago
Permalink
Post by Mark Claassen
A quick check using an online too shows that that 9223372036854775807 is
111111111111111111111111111111111111111111111111111111111111111 in binary.
Yeah, it is just bad logging. Basically that value means keep alive
indefinitely. I thought I had fixed all the instances of that problem.
Apparently not.

Oleg
...
Jose Escobar
12 years ago
Permalink
Ok, I guess this is MAX_INT and it could means keep-alive
indefinitely, but how can it get this value if I had override
DefaultConnectionKeepAliveStrategy to set 15000 ms keep-alive??

Maybe is super.getKeepAliveDuration(response, context); returning
MAX_INT and it is not getting inside the if(keepAlive==-1) ??


I'm using httpclient 4.2
...
Oleg Kalnichevski
12 years ago
Permalink
Post by Jose Escobar
Ok, I guess this is MAX_INT and it could means keep-alive
indefinitely, but how can it get this value if I had override
DefaultConnectionKeepAliveStrategy to set 15000 ms keep-alive??
Maybe is super.getKeepAliveDuration(response, context); returning
MAX_INT and it is not getting inside the if(keepAlive==-1) ??
I just double-checked. -1 is default.

Oleg
...
Mike Boyers
12 years ago
Permalink
I'm upgrading to 4.2 from 4.1 and am wondering what the various definitions are for these methods in PoolStats:

getLeased()
getPending()
getAvailable()
getMax()

Specifically, I want to know how many total connections are in the pool. Is that available + leased?

Thanks,
Mike
Oleg Kalnichevski
12 years ago
Permalink
Post by Mike Boyers
getLeased()
persistent connections tracked by the connection manager currently being
used to execute requests.
Post by Mike Boyers
getPending()
connection requests being blocked awaiting a free connection. This can
happen only if there are more worker threads contending for fewer
connections.
Post by Mike Boyers
getAvailable()
idle persistent connections.
Post by Mike Boyers
getMax()
max number of allowed persistent connections.
Post by Mike Boyers
Specifically, I want to know how many total connections are in the pool. Is that available + leased?
Yes, it is.

Oleg
Post by Mike Boyers
Thanks,
Mike
---------------------------------------------------------------------
Mike Boyers
12 years ago
Permalink
Got it.

Thanks Oleg!




----- Original Message -----
From: Oleg Kalnichevski <olegk-1oDqGaOF3Lkdnm+***@public.gmane.org>
To: HttpClient User Discussion <httpclient-users-Bio+***@public.gmane.org>
Cc:
Sent: Monday, November 5, 2012 3:22 PM
Subject: Re: PoolStats question
Post by Mike Boyers
getLeased()
persistent connections tracked by the connection manager currently being
used to execute requests.
Post by Mike Boyers
getPending()
connection requests being blocked awaiting a free connection. This can
happen only if there are more worker threads contending for fewer
connections.
Post by Mike Boyers
getAvailable()
idle persistent connections.
Post by Mike Boyers
getMax()
max number of allowed persistent connections.
Post by Mike Boyers
Specifically, I want to know how many total connections are in the pool.  Is that available + leased?
Yes, it is.

Oleg
Post by Mike Boyers
Thanks,
Mike
---------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe-Bio+***@public.gmane.org
For additional commands, e-mail: httpclient-users-help-Bio+***@public.gmane.org
Gary Gregory
12 years ago
Permalink
FYI: I added these comments to the Javadoc for PoolStats.

Gary
...
--
E-Mail: garydgregory-***@public.gmane.org | ggregory-1oDqGaOF3Lkdnm+***@public.gmane.org
JUnit in Action, 2nd Ed: <http://goog_1249600977>http://bit.ly/ECvg0
Spring Batch in Action: <http://s.apache.org/HOq>http://bit.ly/bqpbCK
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory
Oleg Kalnichevski
12 years ago
Permalink
Post by Gary Gregory
FYI: I added these comments to the Javadoc for PoolStats.
Gary
Fantastic! Thank you, Gary

Oleg
...
Gary Gregory
12 years ago
Permalink
Post by Oleg Kalnichevski
Post by Gary Gregory
FYI: I added these comments to the Javadoc for PoolStats.
Gary
Fantastic! Thank you, Gary
YW :)

G
...
--
E-Mail: garydgregory-***@public.gmane.org | ggregory-1oDqGaOF3Lkdnm+***@public.gmane.org
JUnit in Action, 2nd Ed: <http://goog_1249600977>http://bit.ly/ECvg0
Spring Batch in Action: <http://s.apache.org/HOq>http://bit.ly/bqpbCK
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory
rnavarrocantu
12 years ago
Permalink
Guys, I'm having this same issue, but it's not clear to me if there was a
resolution, and what was the settings required to avoid hitting this
timeout. Thanks



--
View this message in context: http://httpcomponents.10934.n7.nabble.com/HttpClient-with-PoolingClientConnectionManager-throws-read-time-out-tp18437p20125.html
Sent from the HttpClient-User mailing list archive at Nabble.com.
Chun Tat David Chu
12 years ago
Permalink
Did you set the CoreConnectionPNames.SO_TIMEOUT somewhere in your code?

According to the httpclient-tutorial.pdf, if you didn't set the socket
timeout, your read operation should never be timed out. Here's the copied
and pasted section from the PDF.

CoreConnectionPNames.SO_TIMEOUT='http.socket.timeout': defines the socket
timeout (SO_TIMEOUT) in milliseconds, which is the timeout for waiting for
data or, put differently, a maximum period inactivity between two
consecutive data packets). A timeout value of zero is interpreted as an
infinite timeout. This parameter expects a value of type java.lang.Integer.
If this parameter is not set, read operations will not time out (infinite
timeout).

Good luck!

dc
Post by rnavarrocantu
Guys, I'm having this same issue, but it's not clear to me if there was a
resolution, and what was the settings required to avoid hitting this
timeout. Thanks
--
http://httpcomponents.10934.n7.nabble.com/HttpClient-with-PoolingClientConnectionManager-throws-read-time-out-tp18437p20125.html
Sent from the HttpClient-User mailing list archive at Nabble.com.
---------------------------------------------------------------------
Loading...