|Javier Puerto||Mar 14, 2012 5:54 am||.diff, .diff|
|Steven Dolg||Mar 14, 2012 7:37 am|
|Javier Puerto||Mar 14, 2012 8:16 am|
|Thorsten Scherler||Mar 14, 2012 7:25 pm|
|Igor Malinin||Mar 15, 2012 1:12 am|
|Robby Pelssers||Mar 15, 2012 1:40 am|
|Igor Malinin||Mar 15, 2012 1:54 am|
|Robby Pelssers||Mar 15, 2012 1:57 am|
|Javier Puerto||Mar 15, 2012 2:22 am|
|Igor Malinin||Mar 15, 2012 2:49 am|
|Robby Pelssers||Mar 15, 2012 3:13 am|
|Steven Dolg||Mar 15, 2012 4:03 am|
|Igor Malinin||Mar 15, 2012 4:03 am|
|Steven Dolg||Mar 16, 2012 3:06 am|
|Javier Puerto||Mar 16, 2012 6:32 am|
|Thorsten Scherler||Mar 16, 2012 7:25 pm|
|Steven Dolg||Mar 29, 2012 3:14 am|
|Thorsten Scherler||Mar 29, 2012 9:01 am|
|Javier Puerto||Mar 29, 2012 4:36 pm|
|Subject:||Re: [C3] Concurrency issues with ComponentProvider|
|From:||Javier Puerto (jpue...@gmail.com)|
|Date:||Mar 16, 2012 6:32:41 am|
El 16 de marzo de 2012 11:07, Steven Dolg <stev...@indoqa.com> escribió:
using the test project from Thorsten  I created a very simple client, running several threads to send requests for a randomly selected static resource (one of the image files).
I was working on it yesterday too. :)
(Of course I reverted the "synchronized" in SpringComponentProvider before running the tests!)
Be careful if you have the snapshots maven repository configured, I had to run my test in offline mode.
I ran several scenarios, the biggest one was 1,000,000 requests with 100 concurrent threads. After each request, the size of the response was compared to the actual size of the requested file. I ran this several times, both with and without caching, and didn't get a single error.
The system handled nearly 1,000 requests per second, so I don't think it's not enough load. I could probably push it a little more, but I doubt this would show a different result.
This leads me to believe that there's nothing wrong with handling concurrent requests, even under heavy load, when handling static resources or other "simple" pipelines.
I got the same conclusions. For the tests, I scraped some pages from internet put them on COB-INF and serve them with a FileReader component. My JMeter stress test didn't reproduced the problem.
(Which is also consistent with our Jira, which is not crawling with "random strangeness" issues) Since you *do* have an issue, we should look for other potential sources.
Controllers in general (or your specific one) could be causing problems. (The "synchronized" in the SpringComponentProvider could even cause parts of the execution before it to be executed sequential instead of parallel). There's also no controller involved in Thorsten's test project, so this would be consistent with what I saw yesterday.
We need time to isolate the problem in our webapp removing components till we found the problematic component. We can provide then a better test block and probably the fix for the problematic component if we can detect it.
What is your controller doing and could you replace it with some kind of static resource to test if that's causing your problem? Could you try to remove the "synchronized" modifiers one by one and see which of them is actually fixing your problem? (I guess it's the one at "createComponent", but who knows...).
Yeah, I added synchronized to all the methods but I think the same, probably "createComponent" is enough.
Could you verify that the HTML page generated by your controller (that's how I understood your application) is actually correct when you see a wrong result?
It was my first test with JMeter and seems to work fine, also when I saw the problem with the FileReader patch, I inspected the HTML and the problematic file source was right. If I request the file again the image is fine.
The other thing I noticed yesterday, is that the "synchronized" had less a performance impact than I expected. The number of requests handled per second was almost unchanged, probably due to the method being synchronized having such a short execution time.
I think that the "syncs" are only to retrieve the component instances so it should be a lightweight operation. Also the process can continue to be concurrently, only the instances are retrieved sequentially. So the performance impact will be bigger for component with higher creation costs. Maybe I'm wrong.
Still I think we should try to avoid this, if at all possible, since we are forcing the creating of pipelines to be handled sequentially.
I'm agree, it should be better to fix the problematic component but the patch will fix the problem until we found the problematic component. For more testing I was trying to make the profiling block to work without luck. Does somebody tried to profile a C3 application yet? I didn't found the documentation, I was guiding myself with the source code.
Thank Steven for your tests and efforts.
Am 14.03.2012 13:55, schrieb Javier Puerto:
Hi Cocoon developers,
we are working in a project based on C3 and we found a strange behaviour when loading the static resources. We are developing a Web 2.0 application and therefore we have a lot of resources (js, images and css). The weird thing is that sometimes the system is returning the wrong resources, returning a js instead of a css or switching images...
For my tests I did a little modification in class FileReaderComponent to see the source of the file that is being served. See attached file "FileReaderComponent.diff". This modification uses System.out to show the source of the file before and after read it. The component seems to be fine, also it's very simple so it seems to not be the cause. The patch allow me to discard the component as cause of problem, the source passed as argument when there's too much resources requests are wrong sometimes.
I've review how the components are instantiated and I saw a possible design problem for ComponentProvider class.
ref="org.apache.cocoon.**sitemap.spring.ActionFactory" /> <property name="**languageInterpreterFactory" ref="org.apache.cocoon.** sitemap.expression.**LanguageInterpreterFactory" /> <property name="**pipelineComponentFactory" ref="org.apache.cocoon.** sitemap.spring.**PipelineComponentFactory" /> <property name="pipelineFactory" ref="org.apache.cocoon.**sitemap.spring. **PipelineFactory" /> </bean>
ref="org.apache.cocoon.**sitemap.ComponentProvider" /> </bean>
Above is the sitemap component configuration, the InvocationImpl is the component that will create the different components thought the ComponentProvider that will delegate on different factories. The problem I see is that while the Invocation is declared as "prototype", the ComponentProvider is a shared instance so it could be that on high demand the methods overlaps between request. I've solved the problem declaring the methods as "synchronized" (att. SpringComponentProvider.diff).
This will solve the problems with concurrency, but I wonder if it's the best solution. WDYT?
It's very hard to reproduce the problem with tools like JMeter (better reloading directly in browser). Anyways it's happend sometimes with my current functional test plan but I can't reproduce it consistently. Also the JMeter test plan is designed for our current web project so I can't attach it. The functional tests is configure as:
* 30 threads * No delay between request * Ramp up 1s * Infinite loop * Random controller with 6 static requests, each request with size assertion.
Please, tell me if you need more detailed tests or it's enough. IMO anybody could see the problem if there's a lot of concurrent resources involved.