PDA

View Full Version : resin-destroy closing with 410 active requests.


Kiilerich
03-17-2011, 02:17 PM
Hi,

We have a Resin 3.1.9 that suddenly stops taking requests.

jvm-default.log gets spammed with these:
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:597)
at com.caucho.util.ThreadPool$ThreadLauncher.startCon nection(ThreadPool.java:842)
at com.caucho.util.ThreadPool$ThreadLauncher.run(Thre adPool.java:881)
at java.lang.Thread.run(Thread.java:619)


When I shut down the service, one of the apps write this in the stdout:

{resin-destroy} WebApp[http://localhost:8080/secret] closing with 410 active requests.

Does this mean there is alot of requests blocked in the server, and hence it runs out of threads?

When the server enters this limbo mode, access to the admin pages won't work, and as such I cannot see the threads active.

Typically this server serves 30-150 million requests before this happends, so It's not easy to reproduce (in a short time).


Any ideas?

reza
03-23-2011, 10:07 PM
Kiilerich,

It sounds like thread-max is set too high and/or you might have some rogue code blocking threads. It also might be that you are simply running out of the physical resources on the machine and either need to upgrade hardware or move to a Resin cluster.

I would debug this by setting the thread-max lower so that the machine is not overrun and profile the application over a period of time to see if there are obvious bottle-necks.

Hope it helps,
Reza