Sure!

with comments...

Yeah, I saw that blurb. That's unfortunate, because in truth this is the crux of your problem. It is not so bad that a popular JSP gets cached into RAM, that is a good thing. That is what RAM is for. However, in this case, even the rarest and most unpopular of your JSPs are getting cached permanently. Owie.

Are you able to isolate a particular instance/server so that current users can continue with the site, while no new users go there? If that is possible with your configuration, then you do not really have a session problem as they should be invalid by the user logging out/going away/timing out.

To be fair, replacing JSP with Velocity, particularly a JSP generated by another program (i.e. not written directly by a CBL (Carbon Based Lifeform)) implies that the execution of the compiled JSP page is slower than the intpretation of the Velocity template (modulo the fact that Tomcat does not GC JSPs, but that is a detail for this discussion). I do not have any data either way as to which is faster.

In truth, there is not anything really wrong with your overall design. Most complaints about JSP focus on maintainability of the JSP files, and the intermixing of code and HTML etc. Velocity implicity suffers from that same issue, though perhaps its syntax and model may be better. But these are not direct issues with your system, as your JSPs are derivative products of your CMS system. In my opinion, your use of JSPs here is no different from using something akin to the Olde Tyme Server Side Includes. The facts seem to be that save for the memory issue, you system is behaving fine, and, as you noted, your JSP solution appears to scale better.

And well it should with all of your pages cached in RAM :-).

So, I would look at the other message in the list that mentioned the Jasper class loaders hash table and perhaps changing that to support an LRU queue rather than a hash table.

I know it is a complicated and not necessarily well understood area for most developers, but it IS the source of your problem. I think that if you spend a little bit of time in that section of code, you have a better chance for success in a shorter amount of time. Also, once you load test the heck out of if and stress it as best as you can, it is probably much safer as from a logical or even implementation point of view, your application does not change at all. Ideally, it can simply be a change from a HashTable (or HashMap, whatever they are using) to an LRUMap (of somekind, this is left as an exercise for the reader :-) -- maybe you can safely use WeakReferences here), with all of the Real Hard Work (tm) done by the surrounding code.

Granted, this means you must go through the nightmare of actually BUILDING Tomcat (not for the meek).

The code in place already deals with loading class files, checking the map for loaded classes, synchronization issues, etc. So what is the difference between checking a HashMap for a class and loading it if it does not exist (in this case the first time the page is loaded), and seeing that the class is not there because it happened to drop off the end of the queue? It should not matter why the class is not in the map, but simply that it is not there.

For normal operations and classes, this is less of an issue, but your classes are really content so they suck up an extraordinary amount of memory. Also, for normal classes this can be a real issue because each time a class is loaded, it is "different" than before (I think). But JSPs are not normal classes like, say, some kind of utility class. The real issue is that there is no way to query the JVM for instances of a class to know whether it is safe to drop the actual class data from a custom loader :-/.

I glanced at the code, but you will need to talk to the Tomcat developers for details. It appears that the Jasper loader delegates to the parent class loader, but it seems to me that those PARENT loaders would be either the StandardClassLoader or WebAppClassLoader, so your change may well be contained here.

So, anyway, it may not be a trivial mod but it will solve your problem and I think it is a doable modification unless the Tomcat Dev Team cries foul and heresy. They may know some subtle problems that such a scheme will aggravate. But you would think this should not be too horrible as by design the container must be able to reload JSP classes (unless of course it does not and simply use jspClass1, jspClass2, jspClass3, etc. -- eek!).

Good Luck and let us know what you do and how it works for you.

(Look at the list, Craig has some comments regarding this...in the Memory Usage and Garbage Collection thread...)

Regards,