Problem with Resin 3.1, Resin hangs after sometime

[sahil]

We had been using Resin 2.1.17 with Java 1.4 on Apache for more than 4 years (tried and tested setup). But recently due to requirement of UTF support, we upgraded to Resin 3.1.10 with Java 1.6.11.

At first, everything started smoothly and the setup looked promising. We upgraded all production servers with this same setup in no time. But we noticed that after running for couple of hours, Resin just hung. Moreover, it happened on all servers almost at the same time. Apache started giving out 503 error. There was nothing in logs that we could trace and fix. Servers were running fine, Apache was processing requests as normal (other than JSP) but Resin was not responding at all.

Server configuration:
CentOS release 4.4
Resin 3.1.10
Java 1.6.11
Apache 2
RAM (4GB)

Resin Conf:
JVM Arguments (-Xms750m -Xmx1024m -Xss1m)
thread-max (500)


When Resin is restarted, everything works fine again for awhile before it starts giving 503 error again. One more error we found in log files:

java.lang.OutOfMemoryError: unable to create new native thread

Please provide information on the cause of above errors at the earliest. Any suggestion is much appreciated.

Thanks
Sahil

[emil]

Hi Sahil,

You'll need to do a thread dump and a heap dump to diagnose the problem. Sounds like a GC issue. In Resin 3.1, you can get a thread dump using kill -QUIT on the Resin process id (note that Resin 3.1 has two java processes - the watchdog and Resin - so make sure you get the right one). You can also get a heap dump using jmap -dump:format=b,file=<filename> <pid> on the Resin process.

Thanks,
Emil

[Xapad77]

Hi Sahil,
Did you ever find out what the cause of the problem was?
Thanks for your response :)

Cheers