atom feed7 messages in org.apache.tomcat.usersRe: Long Polling : Tomcat 7.0.50 / 8.0.9
FromSent OnAttachments
anurag guptaAug 21, 2014 10:03 pm 
Mark ThomasAug 22, 2014 1:39 am 
anurag guptaAug 22, 2014 1:47 am 
Mark ThomasAug 22, 2014 2:06 am 
anurag guptaAug 22, 2014 3:22 am 
anurag guptaAug 22, 2014 4:55 am 
anurag guptaAug 28, 2014 11:01 pm 
Subject:Re: Long Polling : Tomcat 7.0.50 / 8.0.9
From:anurag gupta (anur@gmail.com)
Date:Aug 28, 2014 11:01:56 pm
List:org.apache.tomcat.users

Can anyone help regarding this ?

Update:--

A simple test on Tomcat 7.0.50/7.0.55 of Longpolling implementation using JAX-RS 2.0 AsyncResponse mechanism. I'm seeing the following the errors in the logs and a lot many CLOSE_WAIT connections, Why ?

Exception in thread "http-nio-8080-ClientPoller-0" java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926) at java.util.HashMap$KeyIterator.next(HashMap.java:960) at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1067) at org.apache.tomcat.util.net.NioEndpoint$Poller.timeout(NioEndpoint.java:1437) at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:1231) at java.lang.Thread.run(Thread.java:744)

[ERROR] [2014-08-29 05:54:19,622] [-8080-ClientPoller-0] [he.tomcat.util.net.NioEndpoint] Error allocating socket processor java.lang.NullPointerException at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:742) at org.apache.tomcat.util.net.NioEndpoint$Poller.processKey(NioEndpoint.java:1273) at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:1226) at java.lang.Thread.run(Thread.java:744)

[ERROR] [2014-08-29 05:48:35,941] [-8080-ClientPoller-1] [he.tomcat.util.net.NioEndpoint] Error allocating socket processor java.lang.NullPointerException at org.apache.tomcat.util.net.NioEndpoint.processSocket(NioEndpoint.java:742) at org.apache.tomcat.util.net.NioEndpoint$Poller.processKey(NioEndpoint.java:1273) at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:1226) at java.lang.Thread.run(Thread.java:744)

Exception in thread "http-nio-8080-ClientPoller-0" java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926) at java.util.HashMap$KeyIterator.next(HashMap.java:960) at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1067) at org.apache.tomcat.util.net.NioEndpoint$Poller.timeout(NioEndpoint.java:1437) at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:1231) at java.lang.Thread.run(Thread.java:744)

Ok, So the requests will be idle upto the long poll timeout if no response is generated.

So in our test setup we have 60 clients and each makes 5000 requests. These 5000 requests are made at the same time and renewed(i.e. a new request is made in a loop ) as soon as the app server sends response (which in the worst case i.e no response was available, will be a empty json)

​A few minutes back I tried with processorCache="500000", but still tomcat(8.0.9) logged OOM GC Overhead Limit Exceeded and on server around 70K sockets were open (from /proc/net/sockstat​) .

On Fri, Aug 22, 2014 at 5:03 PM, Mark Thomas <mar@apache.org> wrote:

On 22/08/2014 11:22, anurag gupta wrote:

Executors:-

<Executor name="tomcatThreadPool" namePrefix="catalina-exec-" maxThreads="2048" minSpareThreads="1024" maxQueueSize="10000" prestartminSpareThreads="true"/>

This is the connector config :-

<Connector port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol" redirectPort="8443" acceptCount="100000" maxConnections="-1" acceptorThreadCount="5" executor="tomcatThreadPool" connectionLinger="-1" socket.soLingerOn="false" socket.soLingerTime="0" socket.soReuseAddress="true" connectionTimeout="1000" socket.soTimeout="1000" keepAliveTimeout="0" maxKeepAliveRequests="1" socket.soKeepAlive="false" />

The only way to know for sure is if you use a profiler and find out

where you application is using memory.

Yes we do cache the AsyncResponse objects till the timeout happens or some response is generated.

How long does a request take to process? Exactly how many concurrent

requests are you trying to support? A long poll request has a timeout of 10 mins (in this test), but we want to have it upto 60 mins if feasible. We are trying to figure out the max achievable concurrent requests.

Concurrent requests != concurrent connections.

Concurrent requests (i.e. where the server is actively doing something with a connection) will be limited to 2048 with that configuration (maximum number of available threads).

Concurrent connections will depend on you test environment. For a single Tomcat HTTP connector, there is a hard limit of 64k connections per client but you can use multiple clients (each with their own IP address) to get around that. After that, you'll hit OS limits - that should be around several hundred k.