nosuchelementexception causing I/O reactor to be terminated

[Nitu Choudhary]

Hi,

Under high concurrency we see cases where I/O reactor is getting terminated
when closing the input stream. In our application we also close the input
stream as we want to cleanup the memory under various exception conditions.
What is the recommended way to consume the http entity? It shows that on
timeout the nio client is trying to consume the entity, but it has already
been consumed by our application as it got a done signal, and we retrieved
the entity and closed it.

Here is the stack for your reference:

org.apache.http.impl.nio.client.InternalHttpAsyncClient - I/O reactor
terminated abnormally java.util.NoSuchElementException: null at java.util.
ArrayList$Itr.next(ArrayList.java:854) at java.util.Collections$3.nextEl
ement(Collections.java:5216) at java.io.SequenceInputStream.nextStream(
SequenceInputStream.java:110) at java.io.SequenceInputStream.close(
SequenceInputStream.java:232) at org.apache.http.nio.entity.Ent
ityAsyncContentProducer.close(EntityAsyncContentProducer.java:92) at org.
apache.http.nio.protocol.BasicAsyncRequestProducer.close(
BasicAsyncRequestProducer.java:155) at org.apache.http.impl.nio.client.
DefaultClientExchangeHandlerImpl.releaseResources(DefaultC
lientExchangeHandlerImpl.java:84) at org.apache.http.impl.nio.client.
AbstractClientExchangeHandler.failed(AbstractClientExchangeHandler.java:429)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.
connectionRequestFailed(AbstractClientExchangeHandler.java:348) at org.
apache.http.impl.nio.client.AbstractClientExchangeHandler.access$100(
AbstractClientExchangeHandler.java:62) at org.apache.http.impl.nio.client.
AbstractClientExchangeHandler$1.failed(AbstractClientExchangeHandler.java:
392) at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:134)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$1.
failed(PoolingNHttpClientConnectionManager.java:289) at org.apache.http.
concurrent.BasicFuture.failed(BasicFuture.java:134) at org.apache.http.nio.
pool.RouteSpecificPool.timeout(RouteSpecificPool.java:168) at org.apache.
http.nio.pool.AbstractNIOConnPool.requestTimeout(AbstractNIOConnPool.java:
584) at org.apache.http.nio.pool.AbstractNIOConnPool$InternalSession
RequestCallback.timeout(AbstractNIOConnPool.java:851) at org.apache.http.
impl.nio.reactor.SessionRequestImpl.timeout(SessionRequestImpl.java:183) at
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processTimeouts(
DefaultConnectingIOReactor.java:210) at org.apache.http.impl.nio.reactor.
DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:155)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(
AbstractMultiworkerIOReactor.java:348) at org.apache.http.impl.nio.conn.
PoolingNHttpClientConnectionManager.execute(PoolingNHttpClie
ntConnectionManager.java:194) at org.apache.http.impl.nio.client.
CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) at
java.lang.Thread.run(Thread.java:748)

regards,
nitu

[Oleg Kalnichevski]

Post by Nitu Choudhary
Hi,
Under high concurrency we see cases where I/O reactor is getting terminated
when closing the input stream. In our application we also close the input
stream as we want to cleanup the memory under various exception conditions.
What is the recommended way to consume the http entity? It shows that on
timeout the nio client is trying to consume the entity, but it has already
been consumed by our application as it got a done signal, and we retrieved
the entity and closed it.

Hi Nitu

Unlike its blocking counterpart HttpAsyncClient automatically
deallocates resources once a message exchange is complete (successfully
or unsuccessfully). One does not need to ensure the response entity is
fully consumed in order to ensure proper deallocation of the connection
associated with it. This is taken care of by the framework.

However I would strongly recommend to avoid mixing non-blocking I/O
with inherently blocking InputStream / OutputStream based APIs. I would
recommend using proper non-blocking HttpAsyncRequestProducer and
HttpAsyncResponseConsumer implementations for any non-trivial request /
response exchanges with HttpAsyncClient. In other words try getting rid
of SequenceInputStream entirely.

Hope this helps

Oleg

Post by Nitu Choudhary
org.apache.http.impl.nio.client.InternalHttpAsyncClient - I/O reactor
terminated abnormally java.util.NoSuchElementException: null at java.util.
ArrayList$Itr.next(ArrayList.java:854) at
java.util.Collections$3.nextEl
ement(Collections.java:5216) at
java.io.SequenceInputStream.nextStream(
SequenceInputStream.java:110) at java.io.SequenceInputStream.close(
SequenceInputStream.java:232) at org.apache.http.nio.entity.Ent
ityAsyncContentProducer.close(EntityAsyncContentProducer.java:92) at org.
apache.http.nio.protocol.BasicAsyncRequestProducer.close(
BasicAsyncRequestProducer.java:155) at
org.apache.http.impl.nio.client.
DefaultClientExchangeHandlerImpl.releaseResources(DefaultC
lientExchangeHandlerImpl.java:84) at org.apache.http.impl.nio.client.
AbstractClientExchangeHandler.failed(AbstractClientExchangeHandler.java:429)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.
connectionRequestFailed(AbstractClientExchangeHandler.java:348) at org.
apache.http.impl.nio.client.AbstractClientExchangeHandler.access$100(
AbstractClientExchangeHandler.java:62) at
org.apache.http.impl.nio.client.
392) at
org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:134)
at
org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$1.
failed(PoolingNHttpClientConnectionManager.java:289) at
org.apache.http.
concurrent.BasicFuture.failed(BasicFuture.java:134) at
org.apache.http.nio.
pool.RouteSpecificPool.timeout(RouteSpecificPool.java:168) at
org.apache.
584) at org.apache.http.nio.pool.AbstractNIOConnPool$InternalSession
RequestCallback.timeout(AbstractNIOConnPool.java:851) at
org.apache.http.
impl.nio.reactor.SessionRequestImpl.timeout(SessionRequestImpl.java:183) at
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processTimeouts(
DefaultConnectingIOReactor.java:210) at
org.apache.http.impl.nio.reactor.
DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:155)
at
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(
AbstractMultiworkerIOReactor.java:348) at
org.apache.http.impl.nio.conn.
PoolingNHttpClientConnectionManager.execute(PoolingNHttpClie
ntConnectionManager.java:194) at org.apache.http.impl.nio.client.
CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) at
java.lang.Thread.run(Thread.java:748)
regards,
nitu

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-***@hc.apache.org
For additional commands, e-mail: httpclient-users-***@hc.apache.org

[Nitu Choudhary]

thanks Oleg. What happens if in a successful case the future is not
consumed by the application? When will that memory be released?

regards,
Nitu

Post by Oleg Kalnichevski

Post by Nitu Choudhary
Hi,
Under high concurrency we see cases where I/O reactor is getting terminated
when closing the input stream. In our application we also close the input
stream as we want to cleanup the memory under various exception conditions.
What is the recommended way to consume the http entity? It shows that on
timeout the nio client is trying to consume the entity, but it has already
been consumed by our application as it got a done signal, and we retrieved
the entity and closed it.

Hi Nitu
Unlike its blocking counterpart HttpAsyncClient automatically deallocates
resources once a message exchange is complete (successfully or
unsuccessfully). One does not need to ensure the response entity is fully
consumed in order to ensure proper deallocation of the connection
associated with it. This is taken care of by the framework.
However I would strongly recommend to avoid mixing non-blocking I/O with
inherently blocking InputStream / OutputStream based APIs. I would
recommend using proper non-blocking HttpAsyncRequestProducer and
HttpAsyncResponseConsumer implementations for any non-trivial request /
response exchanges with HttpAsyncClient. In other words try getting rid of
SequenceInputStream entirely.
Hope this helps
Oleg

Post by Nitu Choudhary
org.apache.http.impl.nio.client.InternalHttpAsyncClient - I/O reactor
terminated abnormally java.util.NoSuchElementException: null at java.util.
ArrayList$Itr.next(ArrayList.java:854) at java.util.Collections$3.nextEl
ement(Collections.java:5216) at java.io.SequenceInputStream.nextStream(
SequenceInputStream.java:110) at java.io.SequenceInputStream.close(
SequenceInputStream.java:232) at org.apache.http.nio.entity.Ent
ityAsyncContentProducer.close(EntityAsyncContentProducer.java:92) at org.
apache.http.nio.protocol.BasicAsyncRequestProducer.close(
BasicAsyncRequestProducer.java:155) at org.apache.http.impl.nio.client.
DefaultClientExchangeHandlerImpl.releaseResources(DefaultC
lientExchangeHandlerImpl.java:84) at org.apache.http.impl.nio.client.
AbstractClientExchangeHandler.failed(AbstractClientExchangeH
andler.java:429)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.
connectionRequestFailed(AbstractClientExchangeHandler.java:348) at org.
apache.http.impl.nio.client.AbstractClientExchangeHandler.access$100(
AbstractClientExchangeHandler.java:62) at org.apache.http.impl.nio.clien
t.
AbstractClientExchangeHandler$1.failed(AbstractClientExchang
392) at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.
java:134)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$1.
failed(PoolingNHttpClientConnectionManager.java:289) at org.apache.http.
concurrent.BasicFuture.failed(BasicFuture.java:134) at
org.apache.http.nio.
pool.RouteSpecificPool.timeout(RouteSpecificPool.java:168) at org.apache.
http.nio.pool.AbstractNIOConnPool.requestTimeout(AbstractNIO
584) at org.apache.http.nio.pool.AbstractNIOConnPool$InternalSession
RequestCallback.timeout(AbstractNIOConnPool.java:851) at org.apache.http.
impl.nio.reactor.SessionRequestImpl.timeout(SessionRequestImpl.java:183) at
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.
processTimeouts(
DefaultConnectingIOReactor.java:210) at org.apache.http.impl.nio.reactor.
DefaultConnectingIOReactor.processEvents(DefaultConnectingIO
Reactor.java:155)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(
AbstractMultiworkerIOReactor.java:348) at org.apache.http.impl.nio.conn.
PoolingNHttpClientConnectionManager.execute(PoolingNHttpClie
ntConnectionManager.java:194) at org.apache.http.impl.nio.client.
CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) at
java.lang.Thread.run(Thread.java:748)
regards,
nitu

---------------------------------------------------------------------

[Oleg Kalnichevski]

Post by Nitu Choudhary
thanks Oleg. What happens if in a successful case the future is not
consumed by the application? When will that memory be released?

Upon completion of the message exchange (assuming both request producer
and response consumer correctly deallocate their resources).

Oleg

Post by Nitu Choudhary
regards,
Nitu

On Wed, Nov 22, 2017 at 9:19 PM, Nitu Choudhary

Post by Nitu Choudhary
Hi,
Under high concurrency we see cases where I/O reactor is getting terminated
when closing the input stream. In our application we also close the input
stream as we want to cleanup the memory under various exception conditions.
What is the recommended way to consume the http entity? It shows that on
timeout the nio client is trying to consume the entity, but it has already
been consumed by our application as it got a done signal, and we retrieved
the entity and closed it.

Hi Nitu
Unlike its blocking counterpart HttpAsyncClient automatically deallocates
resources once a message exchange is complete (successfully or
unsuccessfully). One does not need to ensure the response entity is fully
consumed in order to ensure proper deallocation of the connection
associated with it. This is taken care of by the framework.
However I would strongly recommend to avoid mixing non-blocking I/O with
inherently blocking InputStream / OutputStream based APIs. I would
recommend using proper non-blocking HttpAsyncRequestProducer and
HttpAsyncResponseConsumer implementations for any non-trivial request /
response exchanges with HttpAsyncClient. In other words try getting rid of
SequenceInputStream entirely.
Hope this helps
Oleg

Post by Nitu Choudhary
org.apache.http.impl.nio.client.InternalHttpAsyncClient - I/O reactor
terminated abnormally java.util.NoSuchElementException: null at java.util.
ArrayList$Itr.next(ArrayList.java:854) at
java.util.Collections$3.nextEl
ement(Collections.java:5216) at
java.io.SequenceInputStream.nextStream(
SequenceInputStream.java:110) at java.io.SequenceInputStream.close(
SequenceInputStream.java:232) at org.apache.http.nio.entity.Ent
ityAsyncContentProducer.close(EntityAsyncContentProducer.java:92) at org.
apache.http.nio.protocol.BasicAsyncRequestProducer.close(
BasicAsyncRequestProducer.java:155) at
org.apache.http.impl.nio.client.
DefaultClientExchangeHandlerImpl.releaseResources(DefaultC
lientExchangeHandlerImpl.java:84) at
org.apache.http.impl.nio.client.
AbstractClientExchangeHandler.failed(AbstractClientExchangeH
andler.java:429)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.
connectionRequestFailed(AbstractClientExchangeHandler.java:348) at org.
apache.http.impl.nio.client.AbstractClientExchangeHandler.access$100(
AbstractClientExchangeHandler.java:62) at
org.apache.http.impl.nio.clien
t.
AbstractClientExchangeHandler$1.failed(AbstractClientExchang
392) at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.
java:134)
at
org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$1.
failed(PoolingNHttpClientConnectionManager.java:289) at
org.apache.http.
concurrent.BasicFuture.failed(BasicFuture.java:134) at
org.apache.http.nio.
pool.RouteSpecificPool.timeout(RouteSpecificPool.java:168) at org.apache.
http.nio.pool.AbstractNIOConnPool.requestTimeout(AbstractNIO
584) at
org.apache.http.nio.pool.AbstractNIOConnPool$InternalSession
RequestCallback.timeout(AbstractNIOConnPool.java:851) at
org.apache.http.
impl.nio.reactor.SessionRequestImpl.timeout(SessionRequestImpl.java:183) at
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.
processTimeouts(
DefaultConnectingIOReactor.java:210) at
org.apache.http.impl.nio.reactor.
DefaultConnectingIOReactor.processEvents(DefaultConnectingIO
Reactor.java:155)
at
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(
AbstractMultiworkerIOReactor.java:348) at
org.apache.http.impl.nio.conn.
PoolingNHttpClientConnectionManager.execute(PoolingNHttpClie
ntConnectionManager.java:194) at org.apache.http.impl.nio.client.
CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) at
java.lang.Thread.run(Thread.java:748)
regards,
nitu

---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-***@hc.apache.org
For additional commands, e-mail: httpclient-users-***@hc.apache.org

[Nitu Choudhary]

sorry, I am still a little confused. Can you explain the message exchange
in more detail?

Say I get a future when I make a request. Now I will typically do a
future.get and get the future http response and consume it. However, in
some cases I might not be able to call a get on the future, but the future
is still holding the response. When does this memory get released?

regards,
Nitu

Post by Oleg Kalnichevski

Post by Nitu Choudhary
thanks Oleg. What happens if in a successful case the future is not
consumed by the application? When will that memory be released?

Upon completion of the message exchange (assuming both request producer
and response consumer correctly deallocate their resources).
Oleg

Post by Nitu Choudhary
regards,
Nitu

Post by Oleg Kalnichevski
Hi,

Post by Nitu Choudhary
Under high concurrency we see cases where I/O reactor is getting terminated
when closing the input stream. In our application we also close the input
stream as we want to cleanup the memory under various exception conditions.
What is the recommended way to consume the http entity? It shows that on
timeout the nio client is trying to consume the entity, but it has already
been consumed by our application as it got a done signal, and we retrieved
the entity and closed it.

Hi Nitu
Unlike its blocking counterpart HttpAsyncClient automatically deallocates
resources once a message exchange is complete (successfully or
unsuccessfully). One does not need to ensure the response entity is fully
consumed in order to ensure proper deallocation of the connection
associated with it. This is taken care of by the framework.
However I would strongly recommend to avoid mixing non-blocking I/O with
inherently blocking InputStream / OutputStream based APIs. I would
recommend using proper non-blocking HttpAsyncRequestProducer and
HttpAsyncResponseConsumer implementations for any non-trivial request /
response exchanges with HttpAsyncClient. In other words try getting rid of
SequenceInputStream entirely.
Hope this helps
Oleg

Post by Nitu Choudhary
org.apache.http.impl.nio.client.InternalHttpAsyncClient - I/O reactor
terminated abnormally java.util.NoSuchElementException: null at java.util.
ArrayList$Itr.next(ArrayList.java:854) at
java.util.Collections$3.nextEl
ement(Collections.java:5216) at java.io.SequenceInputStream.ne
xtStream(
SequenceInputStream.java:110) at java.io.SequenceInputStream.close(
SequenceInputStream.java:232) at org.apache.http.nio.entity.Ent
ityAsyncContentProducer.close(EntityAsyncContentProducer.java:92) at org.
apache.http.nio.protocol.BasicAsyncRequestProducer.close(
BasicAsyncRequestProducer.java:155) at org.apache.http.impl.nio.clien
t.
DefaultClientExchangeHandlerImpl.releaseResources(DefaultC
lientExchangeHandlerImpl.java:84) at org.apache.http.impl.nio.client.
AbstractClientExchangeHandler.failed(AbstractClientExchangeH
andler.java:429)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.
connectionRequestFailed(AbstractClientExchangeHandler.java:348) at org.
apache.http.impl.nio.client.AbstractClientExchangeHandler.access$100(
AbstractClientExchangeHandler.java:62) at
org.apache.http.impl.nio.clien
t.
AbstractClientExchangeHandler$1.failed(AbstractClientExchang
392) at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.
java:134)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionMa
nager$1.
failed(PoolingNHttpClientConnectionManager.java:289) at
org.apache.http.
concurrent.BasicFuture.failed(BasicFuture.java:134) at
org.apache.http.nio.
pool.RouteSpecificPool.timeout(RouteSpecificPool.java:168) at org.apache.
http.nio.pool.AbstractNIOConnPool.requestTimeout(AbstractNIO
584) at org.apache.http.nio.pool.AbstractNIOConnPool$InternalSession
RequestCallback.timeout(AbstractNIOConnPool.java:851) at
org.apache.http.
impl.nio.reactor.SessionRequestImpl.timeout(SessionRequestI
mpl.java:183)
at
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.
processTimeouts(
DefaultConnectingIOReactor.java:210) at org.apache.http.impl.nio.react
or.
DefaultConnectingIOReactor.processEvents(DefaultConnectingIO
Reactor.java:155)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReacto
r.execute(
AbstractMultiworkerIOReactor.java:348) at
org.apache.http.impl.nio.conn.
PoolingNHttpClientConnectionManager.execute(PoolingNHttpClie
ntConnectionManager.java:194) at org.apache.http.impl.nio.client.
CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClient
Base.java:64)
at
java.lang.Thread.run(Thread.java:748)
regards,
nitu

---------------------------------------------------------------------

---------------------------------------------------------------------

[Oleg Kalnichevski]

Post by Nitu Choudhary
sorry, I am still a little confused. Can you explain the message exchange
in more detail?
Say I get a future when I make a request. Now I will typically do a
future.get and get the future http response and consume it. However, in
some cases I might not be able to call a get on the future, but the future
is still holding the response. When does this memory get released?
regards,
Nitu

Hi Nitu

The response future does not necessarily hold a reference to the entire
response message (though it might in the most over-simplistic scenario,
which I would recommend to avoid). It rather holds reference to a
result of response processing generated by the associated
HttpAsyncResponseConsumer. See this sample for a reference

http://hc.apache.org/httpcomponents-asyncclient-4.1.x/httpasyncclient/examples/org/apache/http/examples/nio/client/AsyncClientHttpExchangeStreaming.java

HttpAsyncResponseConsumer is responsible for resource allocation and
deallocation. HttpAsyncClient makes sure HttpAsyncResponseConsumer-s
get to complete their work (normally or abnormally) regardless of what
happens to the response future. As long as the result object does not
hold any resources beyond objects in the heap, it does not need to be
explicitly closed or released.

Hope this helps

Oleg

Post by Nitu Choudhary

On Mon, Nov 27, 2017 at 6:57 PM, Nitu Choudhary

Post by Nitu Choudhary
thanks Oleg. What happens if in a successful case the future is not
consumed by the application? When will that memory be released?

Upon completion of the message exchange (assuming both request producer
and response consumer correctly deallocate their resources).
Oleg

Post by Nitu Choudhary
regards,
Nitu
On Thu, Nov 23, 2017 at 3:08 AM, Oleg Kalnichevski

On Wed, Nov 22, 2017 at 9:19 PM, Nitu Choudhary
Hi,

Post by Nitu Choudhary
Under high concurrency we see cases where I/O reactor is getting terminated
when closing the input stream. In our application we also close
the
input
stream as we want to cleanup the memory under various exception conditions.
What is the recommended way to consume the http entity? It
shows that
on
timeout the nio client is trying to consume the entity, but it
has
already
been consumed by our application as it got a done signal, and we retrieved
the entity and closed it.

Hi Nitu
Unlike its blocking counterpart HttpAsyncClient automatically deallocates
resources once a message exchange is complete (successfully or
unsuccessfully). One does not need to ensure the response entity
is
fully
consumed in order to ensure proper deallocation of the connection
associated with it. This is taken care of by the framework.
However I would strongly recommend to avoid mixing non-blocking I/O with
inherently blocking InputStream / OutputStream based APIs. I would
recommend using proper non-blocking HttpAsyncRequestProducer and
HttpAsyncResponseConsumer implementations for any non-trivial request /
response exchanges with HttpAsyncClient. In other words try getting rid
of
SequenceInputStream entirely.
Hope this helps
Oleg

Post by Nitu Choudhary
org.apache.http.impl.nio.client.InternalHttpAsyncClient - I/O reactor
terminated abnormally java.util.NoSuchElementException: null at java.util.
ArrayList$Itr.next(ArrayList.java:854) at
java.util.Collections$3.nextEl
ement(Collections.java:5216) at java.io.SequenceInputStream.ne
xtStream(
SequenceInputStream.java:110) at
java.io.SequenceInputStream.close(
SequenceInputStream.java:232) at org.apache.http.nio.entity.Ent
ityAsyncContentProducer.close(EntityAsyncContentProducer.java:92)
at
org.
apache.http.nio.protocol.BasicAsyncRequestProducer.close(
BasicAsyncRequestProducer.java:155) at
org.apache.http.impl.nio.clien
t.
DefaultClientExchangeHandlerImpl.releaseResources(DefaultC
lientExchangeHandlerImpl.java:84) at
org.apache.http.impl.nio.client.
AbstractClientExchangeHandler.failed(AbstractClientExchangeH
andler.java:429)
at
org.apache.http.impl.nio.client.AbstractClientExchangeHandler.
connectionRequestFailed(AbstractClientExchangeHandler.java:348)
at
org.
apache.http.impl.nio.client.AbstractClientExchangeHandler.access$100(
AbstractClientExchangeHandler.java:62) at
org.apache.http.impl.nio.clien
t.
AbstractClientExchangeHandler$1.failed(AbstractClientExchang
392) at
org.apache.http.concurrent.BasicFuture.failed(BasicFuture.
java:134)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionMa
nager$1.
failed(PoolingNHttpClientConnectionManager.java:289) at
org.apache.http.
concurrent.BasicFuture.failed(BasicFuture.java:134) at
org.apache.http.nio.
pool.RouteSpecificPool.timeout(RouteSpecificPool.java:168) at org.apache.
http.nio.pool.AbstractNIOConnPool.requestTimeout(AbstractNIO
584) at
org.apache.http.nio.pool.AbstractNIOConnPool$InternalSession
RequestCallback.timeout(AbstractNIOConnPool.java:851) at
org.apache.http.
impl.nio.reactor.SessionRequestImpl.timeout(SessionRequestI
mpl.java:183)
at
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.
processTimeouts(
DefaultConnectingIOReactor.java:210) at
org.apache.http.impl.nio.react
or.
DefaultConnectingIOReactor.processEvents(DefaultConnectingIO
Reactor.java:155)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReacto
r.execute(
AbstractMultiworkerIOReactor.java:348) at
org.apache.http.impl.nio.conn.
PoolingNHttpClientConnectionManager.execute(PoolingNHttpClie
ntConnectionManager.java:194) at
org.apache.http.impl.nio.client.
CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClient
Base.java:64)
at
java.lang.Thread.run(Thread.java:748)
regards,
nitu

---------------------------------------------------------------------

---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-***@hc.apache.org
For additional commands, e-mail: httpclient-users-***@hc.apache.org

[Nitu Choudhary]

Thanks Oleg. I looked at your example, but it was not clear to me as to how
I can avoid passing an input stream from the http request. So in our case
the incoming data from the client is obtained in form of an input stream
and we create a HttpAsyncRequestProducer and pass the entity to it. Do you
have any examples for such a use case to see how we can improve our code?

regards,
Nitu

Post by Oleg Kalnichevski

Post by Nitu Choudhary
sorry, I am still a little confused. Can you explain the message exchange
in more detail?
Say I get a future when I make a request. Now I will typically do a
future.get and get the future http response and consume it. However, in
some cases I might not be able to call a get on the future, but the future
is still holding the response. When does this memory get released?
regards,
Nitu

Hi Nitu
The response future does not necessarily hold a reference to the entire
response message (though it might in the most over-simplistic scenario,
which I would recommend to avoid). It rather holds reference to a result of
response processing generated by the associated HttpAsyncResponseConsumer.
See this sample for a reference
http://hc.apache.org/httpcomponents-asyncclient-4.1.x/
httpasyncclient/examples/org/apache/http/examples/nio/clien
t/AsyncClientHttpExchangeStreaming.java
HttpAsyncResponseConsumer is responsible for resource allocation and
deallocation. HttpAsyncClient makes sure HttpAsyncResponseConsumer-s get to
complete their work (normally or abnormally) regardless of what happens to
the response future. As long as the result object does not hold any
resources beyond objects in the heap, it does not need to be explicitly
closed or released.
Hope this helps
Oleg

Post by Nitu Choudhary

Post by Nitu Choudhary
thanks Oleg. What happens if in a successful case the future is not

Post by Nitu Choudhary
consumed by the application? When will that memory be released?

Upon completion of the message exchange (assuming both request producer
and response consumer correctly deallocate their resources).
Oleg
regards,

Post by Nitu Choudhary
Nitu

On Wed, Nov 22, 2017 at 9:19 PM, Nitu Choudhary
Hi,

Post by Nitu Choudhary
Under high concurrency we see cases where I/O reactor is getting terminated
when closing the input stream. In our application we also close the input
stream as we want to cleanup the memory under various exception conditions.
What is the recommended way to consume the http entity? It shows
that
on
timeout the nio client is trying to consume the entity, but it has already
been consumed by our application as it got a done signal, and we retrieved
the entity and closed it.

Hi Nitu
Unlike its blocking counterpart HttpAsyncClient automatically deallocates
resources once a message exchange is complete (successfully or
unsuccessfully). One does not need to ensure the response entity is fully
consumed in order to ensure proper deallocation of the connection
associated with it. This is taken care of by the framework.
However I would strongly recommend to avoid mixing non-blocking I/O with
inherently blocking InputStream / OutputStream based APIs. I would
recommend using proper non-blocking HttpAsyncRequestProducer and
HttpAsyncResponseConsumer implementations for any non-trivial request /
response exchanges with HttpAsyncClient. In other words try getting rid
of
SequenceInputStream entirely.
Hope this helps
Oleg

Post by Nitu Choudhary
org.apache.http.impl.nio.client.InternalHttpAsyncClient - I/O reactor
terminated abnormally java.util.NoSuchElementException: null at java.util.
ArrayList$Itr.next(ArrayList.java:854) at
java.util.Collections$3.nextEl
ement(Collections.java:5216) at java.io.SequenceInputStream.ne
xtStream(
SequenceInputStream.java:110) at java.io.SequenceInputStream.close(
SequenceInputStream.java:232) at org.apache.http.nio.entity.Ent
ityAsyncContentProducer.close(EntityAsyncContentProducer.java:92)
at
org.
apache.http.nio.protocol.BasicAsyncRequestProducer.close(
BasicAsyncRequestProducer.java:155) at
org.apache.http.impl.nio.clien
t.
DefaultClientExchangeHandlerImpl.releaseResources(DefaultC
lientExchangeHandlerImpl.java:84) at org.apache.http.impl.nio.clien
t.
AbstractClientExchangeHandler.failed(AbstractClientExchangeH
andler.java:429)
at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.
connectionRequestFailed(AbstractClientExchangeHandler.java:348) at org.
apache.http.impl.nio.client.AbstractClientExchangeHandler.ac
cess$100(
AbstractClientExchangeHandler.java:62) at
org.apache.http.impl.nio.clien
t.
AbstractClientExchangeHandler$1.failed(AbstractClientExchang
392) at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.
java:134)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionMa
nager$1.
failed(PoolingNHttpClientConnectionManager.java:289) at
org.apache.http.
concurrent.BasicFuture.failed(BasicFuture.java:134) at
org.apache.http.nio.
pool.RouteSpecificPool.timeout(RouteSpecificPool.java:168) at org.apache.
http.nio.pool.AbstractNIOConnPool.requestTimeout(AbstractNIO
584) at org.apache.http.nio.pool.Abstr
actNIOConnPool$InternalSession
RequestCallback.timeout(AbstractNIOConnPool.java:851) at org.apache.http.
impl.nio.reactor.SessionRequestImpl.timeout(SessionRequestI
mpl.java:183)
at
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.
processTimeouts(
DefaultConnectingIOReactor.java:210) at
org.apache.http.impl.nio.react
or.
DefaultConnectingIOReactor.processEvents(DefaultConnectingIO
Reactor.java:155)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReacto
r.execute(
AbstractMultiworkerIOReactor.java:348) at
org.apache.http.impl.nio.conn.
PoolingNHttpClientConnectionManager.execute(PoolingNHttpClie
ntConnectionManager.java:194) at org.apache.http.impl.nio.client.
CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClient
Base.java:64)
at
java.lang.Thread.run(Thread.java:748)
regards,
nitu

------------------------------------------------------------
---------

---------------------------------------------------------------------

---------------------------------------------------------------------

[Oleg Kalnichevski]

Post by Nitu Choudhary
Thanks Oleg. I looked at your example, but it was not clear to me as to how
I can avoid passing an input stream from the http request. So in our case
the incoming data from the client is obtained in form of an input stream
and we create a HttpAsyncRequestProducer and pass the entity to it. Do you
have any examples for such a use case to see how we can improve our code?

Nitu

InputStream / OutputStream interfaces are inherently blocking. If you
are absolutely sure the request data cannot be represented in any other
form you will likely be better off using the classic (blocking) Apache
HttpClient.

Oleg

Post by Nitu Choudhary
regards,
Nitu

On Tue, Nov 28, 2017 at 7:43 PM, Nitu Choudhary

Post by Nitu Choudhary
sorry, I am still a little confused. Can you explain the message exchange
in more detail?
Say I get a future when I make a request. Now I will typically do a
future.get and get the future http response and consume it.
However, in
some cases I might not be able to call a get on the future, but the future
is still holding the response. When does this memory get released?
regards,
Nitu

Hi Nitu
The response future does not necessarily hold a reference to the entire
response message (though it might in the most over-simplistic scenario,
which I would recommend to avoid). It rather holds reference to a result of
response processing generated by the associated
HttpAsyncResponseConsumer.
See this sample for a reference
http://hc.apache.org/httpcomponents-asyncclient-4.1.x/
httpasyncclient/examples/org/apache/http/examples/nio/clien
t/AsyncClientHttpExchangeStreaming.java
HttpAsyncResponseConsumer is responsible for resource allocation and
deallocation. HttpAsyncClient makes sure
HttpAsyncResponseConsumer-s get to
complete their work (normally or abnormally) regardless of what happens to
the response future. As long as the result object does not hold any
resources beyond objects in the heap, it does not need to be
explicitly
closed or released.
Hope this helps
Oleg

Post by Nitu Choudhary
On Tue, Nov 28, 2017 at 12:15 AM, Oleg Kalnichevski

On Mon, Nov 27, 2017 at 6:57 PM, Nitu Choudhary
thanks Oleg. What happens if in a successful case the future is not

Post by Nitu Choudhary
consumed by the application? When will that memory be released?

Upon completion of the message exchange (assuming both request producer
and response consumer correctly deallocate their resources).
Oleg
regards,

Post by Nitu Choudhary
Nitu
On Thu, Nov 23, 2017 at 3:08 AM, Oleg Kalnichevski

On Wed, Nov 22, 2017 at 9:19 PM, Nitu Choudhary
Hi,

Post by Nitu Choudhary
Under high concurrency we see cases where I/O reactor is
getting
terminated
when closing the input stream. In our application we also
close the
input
stream as we want to cleanup the memory under various
exception
conditions.
What is the recommended way to consume the http entity? It shows
that
on
timeout the nio client is trying to consume the entity, but
it has
already
been consumed by our application as it got a done signal,
and we
retrieved
the entity and closed it.

Hi Nitu
Unlike its blocking counterpart HttpAsyncClient automatically deallocates
resources once a message exchange is complete (successfully or
unsuccessfully). One does not need to ensure the response
entity is
fully
consumed in order to ensure proper deallocation of the
connection
associated with it. This is taken care of by the framework.
However I would strongly recommend to avoid mixing
non-blocking I/O
with
inherently blocking InputStream / OutputStream based APIs. I would
recommend using proper non-blocking HttpAsyncRequestProducer and
HttpAsyncResponseConsumer implementations for any non-trivial request /
response exchanges with HttpAsyncClient. In other words try
getting
rid
of
SequenceInputStream entirely.
Hope this helps
Oleg

Post by Nitu Choudhary
org.apache.http.impl.nio.client.InternalHttpAsyncClient - I/O reactor
terminated abnormally java.util.NoSuchElementException: null
at
java.util.
ArrayList$Itr.next(ArrayList.java:854) at
java.util.Collections$3.nextEl
ement(Collections.java:5216) at
java.io.SequenceInputStream.ne
xtStream(
SequenceInputStream.java:110) at
java.io.SequenceInputStream.close(
SequenceInputStream.java:232) at
org.apache.http.nio.entity.Ent
ityAsyncContentProducer.close(EntityAsyncContentProducer.java:92)
at
org.
apache.http.nio.protocol.BasicAsyncRequestProducer.close(
BasicAsyncRequestProducer.java:155) at
org.apache.http.impl.nio.clien
t.
DefaultClientExchangeHandlerImpl.releaseResources(DefaultC
lientExchangeHandlerImpl.java:84) at
org.apache.http.impl.nio.clien
t.
AbstractClientExchangeHandler.failed(AbstractClientExchangeH
andler.java:429)
at
org.apache.http.impl.nio.client.AbstractClientExchangeHandler.
connectionRequestFailed(AbstractClientExchangeHandler.java:348)
at
org.
apache.http.impl.nio.client.AbstractClientExchangeHandler.ac
cess$100(
AbstractClientExchangeHandler.java:62) at
org.apache.http.impl.nio.clien
t.
AbstractClientExchangeHandler$1.failed(AbstractClientExchang
392) at
org.apache.http.concurrent.BasicFuture.failed(BasicFuture.
java:134)
at
org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionMa
nager$1.
failed(PoolingNHttpClientConnectionManager.java:289) at org.apache.http.
concurrent.BasicFuture.failed(BasicFuture.java:134) at
org.apache.http.nio.
pool.RouteSpecificPool.timeout(RouteSpecificPool.java:168) at org.apache.
http.nio.pool.AbstractNIOConnPool.requestTimeout(AbstractNIO
584) at org.apache.http.nio.pool.Abstr
actNIOConnPool$InternalSession
RequestCallback.timeout(AbstractNIOConnPool.java:851) at org.apache.http.
impl.nio.reactor.SessionRequestImpl.timeout(SessionRequestI
mpl.java:183)
at
org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.
processTimeouts(
DefaultConnectingIOReactor.java:210) at
org.apache.http.impl.nio.react
or.
DefaultConnectingIOReactor.processEvents(DefaultConnectingIO
Reactor.java:155)
at
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReacto
r.execute(
AbstractMultiworkerIOReactor.java:348) at
org.apache.http.impl.nio.conn.
PoolingNHttpClientConnectionManager.execute(PoolingNHttpClie
ntConnectionManager.java:194) at
org.apache.http.impl.nio.client.
CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClient
Base.java:64)
at
java.lang.Thread.run(Thread.java:748)
regards,
nitu

------------------------------------------------------------
---------

---------------------------------------------------------------------

---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-***@hc.apache.org
For additional commands, e-mail: httpclient-users-***@hc.apache.org