atom feed18 messages in net.sourceforge.lists.htmlunit-developRe: [HtmlUnit] HtmlUnit & GAE/J
FromSent OnAttachments
Marc GuillemotJun 29, 2009 7:33 am 
Daniel GredlerJun 29, 2009 9:34 am 
Ahmed AshourJun 29, 2009 9:52 am 
Katharina ProbstJun 29, 2009 10:05 am 
Daniel GredlerJun 29, 2009 10:10 am 
Marc GuillemotJun 29, 2009 11:01 am 
Ahmed AshourJun 29, 2009 11:41 am 
Marc GuillemotJun 30, 2009 1:31 am 
Katharina ProbstJun 30, 2009 6:08 am 
Ahmed AshourJun 30, 2009 6:43 am 
Marc GuillemotJun 30, 2009 8:18 am 
Katharina ProbstJul 1, 2009 5:39 am 
Marc GuillemotJul 2, 2009 2:22 am 
Katharina ProbstJul 2, 2009 3:36 am 
Marc GuillemotJul 2, 2009 4:01 am 
Katharina ProbstJul 2, 2009 6:22 am 
Marc GuillemotJul 3, 2009 2:47 am 
Katharina ProbstJul 3, 2009 6:00 am 
Subject:Re: [HtmlUnit] HtmlUnit & GAE/J
From:Katharina Probst (
Date:Jul 1, 2009 5:39:33 am

Yeah, I see your point(s). However, I don't think that this should hold us back more or less indefinitely. Would it make sense to have a version that we *think* runs on GAE/J, but tell people to do it at their own risk? (Until we can get proper, automatic, GAE/J testing into place).


On Tue, Jun 30, 2009 at 11:18 AM, Marc Guillemot <>wrote:

Hi Kathrin,

the only way to prove that HtmlUnit runs on GAE/J is... to let it run on GAE/J. All other strategies may work for some time but don't bring the right security against regressions. Of course it should continue to run without GAE/J.

From my experience, even the simulated local GAE environment is not good enough to ensure that something works fine when deployed on GAE/J (see for instance this issue:

Cheers, Marc.

-- Web: Blog:

Katharina Probst wrote:

Hi all,

would it not be possible to take out all dependencies that GAE/J doesn't support and then do unit testing independently of GAE/J? Ideally, a GAE/J-compatible HtmlUnit could be run in AppEngine as well as elsewhere, right?

Because HtmlUnit doesn't currently do any database requests or anything else that is necessarily specific to GAE, I'm not entirely sure why it can't be tested outside of it. For example, if we move from URLStreamHandler to URLConnection (for http), then presumably most of the unit test logic would stay the same, right?


On Tue, Jun 30, 2009 at 4:31 AM, Marc Guillemot < <>> wrote:


I'm generally against reflection unless it is really well covered by unit tests (and even in this case, a bridge à la Rhino JVM Bridge is probably better).

This brings us to the point: if we want to make HtmlUnit (mostly) compatible with GAE/J, then *we need to be able to test it* and this has to be done *automatically*.

This won't be trivial because GAE/J isn't currently CI friendly (see )

and because we can't run long during tasks. The task queue support may help and allow to split unit tests in different tasks but it is still only in the roadmap: Last but not least, it would be needed to integrate deployment to GAE/J from our CI and to capture results here.

Features that aren't covered by unit tests simply don't exist ;-)

Cheers, Marc. -- Web: Blog:

Ahmed Ashour wrote: > Hi all, > > I am not sure if GAE follows the same class loading mechanism as JDK. > > If the classes are loaded only upon specific action, then yes, we can just > document "don't use xyz feature in HtmlUnit with GAE" (you are right Daniel, > this is the case for for 'Insecure SSL' in JDK). > > But if the classes are already loaded with usual usage (e.g. XNIException in > HTMLParser), then removing it (or depending on reflection in few lines) > would not affect the codebase much. > > Yours, > Ahmed > ---- > Blog: > > > Marc Guillemot wrote: >> I fully agree with Daniel. >> While it is surely interesting to make HtmlUnit able to work on GAE/J, >> it is still not our main focus. >> >> Cheers, >> Marc. >> -- >> Web: >> Blog: >> >> Daniel Gredler wrote: >>> Hi Ahmed, >>> >>> This is actually the approach I would like to avoid... I don't think GAE >>> should drive design decisions of this magnitude. I may be wrong, but I >>> think a rule like "don't use the insecure SSL option on GAE" is enough, >>> since the JDK doesn't load classes until they are used. What do others >>> think? >>> >>> Take care, >>> >>> Daniel >>> >>> >>> >>> On Mon, Jun 29, 2009 at 12:52 PM, Ahmed Ashour < <> >>> < <>>> wrote: >>> >>> Hi all, >>> >>> I would start by moving out all items not supported by GAE >>> (applets/*..geom/insecure SSL) to special package, to be be used by >>> reflection in normal JDK, and ignored by GAE. >>> >>> Then I will try to make a "-GAE" .jar >>> >>> Regarding URLStreamHandler, I will wait to see how GAE reply to Marc >>> feedback, before changing HtmlUnit. >>> >>> Yours, >>> Ahmed >>> ---- >>> Blog: >>> >>>

>>> *From:* Marc Guillemot < <> >>> < <>>> >>> *To:* >>> *Sent:* Monday, June 29, 2009 5:33:49 PM >>> *Subject:* [HtmlUnit] HtmlUnit & GAE/J >>> >>> Hi, >>> >>> [summarizing an offlist discussion] >>> >>> HtmlUnit can't currently run on Google App Engine due to its >>> restrictions nevertheless it would be interesting to improve HtmlUnit >>> to >>> make it able to run on GAE/J (per default or using some >>> configuration). >>> >>> As of this writing, two problems exist: >>> >>> 1- *usage of URLStreamHandler*: >>> A custom URLStreamHandler is used in HtmlUnit to allow the >>> instantiation >>> of javascript:, about:, or data: urls. URLStreamHandler doesn't >>> belong >>> to the white list of allowed classes on GAE/J >>> >>> I hope that it can be fixed on GAE/J side as we don't need to >>> register a >>> global protocol handler: HtmlUnit just uses its (dummy) >>> URLStreamHandler >>> implementations as parameter of URL c'tor. >>> >>> 2- *Multi-threading*: >>> All background JS tasks (setTimeut, setInterval, XHR) are handled >>> using >>> threads and it's not possible to create own threads on GAE/J. This >>> means >>> that the JavaScriptEngine (in particular the JavaScriptJobManager) >>> needs >>> to be rewritten to run in the main thread. If I correctly remember, >>> we >>> rejected for some time an issue requesting exactly this feature. >>> Nevertheless GAE/J really changes the game and I think that it makes >>> sense to have this option now. >>> >>> Cheers, >>> Marc. >>> -- >>> Web: >>> Blog: >

_______________________________________________ HtmlUnit-develop mailing list <>