69 messages in org.codehaus.groovy.devRe: [groovy-dev] Building Groovy
FromSent OnAttachments
Russel WinderOct 6, 2008 4:36 am 
MingfaiOct 6, 2008 4:47 am 
Hans DockterOct 6, 2008 4:50 am 
Hans DockterOct 6, 2008 4:55 am 
Jochen TheodorouOct 6, 2008 4:56 am 
Guillaume LaforgeOct 6, 2008 7:17 am 
Hans DockterOct 6, 2008 7:51 am 
Russel WinderOct 6, 2008 7:59 am 
Guillaume LaforgeOct 6, 2008 8:19 am 
Guillaume LaforgeOct 6, 2008 8:25 am 
MingfaiOct 6, 2008 8:28 am 
Guillaume LaforgeOct 6, 2008 8:36 am 
Hans DockterOct 6, 2008 1:46 pm 
Guillaume LaforgeOct 6, 2008 1:54 pm 
Hans DockterOct 6, 2008 1:54 pm 
Jochen TheodorouOct 6, 2008 2:03 pm 
Guillaume LaforgeOct 6, 2008 2:09 pm 
Paul DuffyOct 6, 2008 7:06 pm 
Luke DaleyOct 6, 2008 8:47 pm 
Guillaume LaforgeOct 6, 2008 9:44 pm 
Russel WinderOct 6, 2008 11:25 pm 
Russel WinderOct 6, 2008 11:54 pm 
Russel WinderOct 7, 2008 12:03 am 
Jason DillonOct 7, 2008 12:23 am 
Russel WinderOct 7, 2008 12:24 am 
Guillaume LaforgeOct 7, 2008 12:30 am 
Hans DockterOct 7, 2008 12:35 am 
Jason DillonOct 7, 2008 12:35 am 
Hans DockterOct 7, 2008 12:36 am 
Jason DillonOct 7, 2008 12:41 am 
Guillaume LaforgeOct 7, 2008 12:54 am 
Jason DillonOct 7, 2008 1:40 am 
Guillaume LaforgeOct 7, 2008 1:50 am 
Jason DillonOct 7, 2008 1:55 am 
Guillaume LaforgeOct 7, 2008 2:25 am 
Guillaume LaforgeOct 7, 2008 2:35 am 
Jason DillonOct 7, 2008 3:09 am 
Guillaume LaforgeOct 7, 2008 3:12 am 
Russel WinderOct 7, 2008 3:17 am 
Jason DillonOct 7, 2008 3:24 am 
Paul KingOct 7, 2008 4:04 am 
ma...@dockter.bizOct 7, 2008 4:19 am 
ma...@dockter.bizOct 7, 2008 4:25 am 
Jason DillonOct 7, 2008 4:36 am 
Jason DillonOct 7, 2008 4:39 am 
Jochen TheodorouOct 7, 2008 5:20 am 
Jason DillonOct 7, 2008 8:19 am 
Jochen TheodorouOct 7, 2008 9:51 am 
Jason DillonOct 7, 2008 10:49 am 
Jochen TheodorouOct 7, 2008 12:03 pm 
Hans DockterOct 7, 2008 2:34 pm 
Luke DaleyOct 7, 2008 3:52 pm 
Jason DillonOct 8, 2008 1:28 am 
Jason DillonOct 8, 2008 1:35 am 
Hans DockterOct 8, 2008 3:11 am 
Hans DockterOct 8, 2008 3:49 am 
Hans DockterOct 8, 2008 4:30 am 
Hans DockterOct 8, 2008 4:40 am 
Jason DillonOct 8, 2008 4:52 am 
Jason DillonOct 8, 2008 5:21 am 
Jochen TheodorouOct 8, 2008 6:23 am 
Jochen TheodorouOct 8, 2008 6:47 am 
Jochen TheodorouOct 8, 2008 6:59 am 
Hans DockterOct 8, 2008 8:33 am 
Hans DockterOct 8, 2008 8:43 am 
Paul DuffyOct 9, 2008 8:58 am 
Paul KingOct 9, 2008 1:15 pm 
Danno FerrinOct 9, 2008 1:27 pm 
Paul KingOct 10, 2008 2:31 am 
Actions with this message:
Paste this link in email or IM:
Paste this link in email or IM:
Atom feed for this thread
Paste this URL into your reader:
Subject:Re: [groovy-dev] Building GroovyActions...
From:Jason Dillon (jaso@gmail.com)
Date:Oct 7, 2008 10:49:11 am
List:org.codehaus.groovy.dev

On Oct 7, 2008, at 11:52 PM, Jochen Theodorou wrote:

First you don't need to deploy anything to use a plugin, you can include the plugin in your reactor build and you are fine.

can you point me to some documentation about that?

I'm not sure anyone has a specific guide on this, but a M2 plugin is just a jar file with some metadata encoded which lives in a repository. So if in your build you have a plugin module and a module which uses the plugin, M2 will build the plugin first, install it into the local repo, then when the dependent module runs it uses that plugin. Nothing more fancy than that.

Second, using scripting tools like GMaven (or the JRuby plugin or whatever) you can *easily* add extra functionality to your build it basically the exact same fashion as all of these Groovy DSL-based build tools can. In fact as far as getting the build to "properly express what I need remains" goes, there is no difference, simply configure the plugin and write up your groovy to do the remaining bits.

that is groovy embedded in XML, yes? If it is external files it won't be better. Also the question would be if I can create a plugin for the currently running m2 build while the build is running using groovy embedded in XML... I mean that would still be a media breach, but at last I don't have to scatter my build parts all of the place if I don't want to

You can embed or use external files, urls, resources, whichever you feel like.

Sure if you need something outside the set of existing tasks you need to program a new task, or use one of the scripting tasks that are available, which brings us right back to the same point I'm making above.

again, that is groovy embedded in XML. That is still a media breach. Also, if most of your build consists of groovy, then why is it xml based at all?

Interesting, so what type of thing you are building should determine the type of system used to build? Based on that, groovy-core should use a Java-based system, since its got more .java files than .groovy or .xml files ;-)

<snip> Sanity:groovy-core jason$ find src/main/ -name \*.java | wc -l 681 Sanity:groovy-core jason$ find src/main/ -name \*.groovy | wc -l 121 Sanity:groovy-core jason$ find src/test/ -name \*.java | wc -l 182 Sanity:groovy-core jason$ find src/test/ -name \*.groovy | wc -l 582 </snip>

Thats 863 (.java main+test) vs. 703 (.groovy main+test)

IMO the format of the build tool input has nothing to do with the build tool outputs.

Um, is using a Java-based tool to build a Groovy project a breach as well? I'm not buying this argument at all... but I understand its your opinion and I respect that. :-)

no, using a java based tool is not a breach. I don't see a breach when you use existing tasks in ant for example. I see a breach when you have to embed groovy in XML in order to define a task... is it even possible to write a task in a groovy task? The gant people might be able to say if that is so. I think it is possible to register a task... what I want to say is that there is a difference between writing a small script, because something is not as it should be and producing reusable code by for example writing methods that can be used from different parts of the build. Even in a bash script I can use tasks and source other scripts. Having multiple embedded groovy scripting entities that can not really communicate with each other is not good. If they want to communicate, then they have to do so using the ant or maybe maven infrastructure. hat's a level of indirection that is uncomfortable.. well in my opinion ;)

Yes you probably could write an Ant task in a <groovy> task, though I've never actually tried to do that.

M1 had lots of problems, one of them being that it, like ant, ended up allowing developers so much freedom they end up shooting themselves in the foot, or really drown themselves in the mess of configuration they cooked up. IMO this is a fundamental problem of build systems. Too much flexibility, no enforced standards and in a year you have something which no one really understands and only a few can safely maintain.

and I think that is the basic difference between our views. I think flexibility is not bad. A build system should support the user, not force him to do something. Of course I know that forcing somebody to do something can be more easy for the developers of the build system, as well as for the one that gets forced. But a system build on force gets so much inflexibility, that as soon as you have a case not covered by the current rules, nobody actually knows what to do... if it can be done even.

So far I've not found this to be the case, with any build tool (make, ant, m1 or m2)... just a matter of learning how to make it work. And in many cases its more of a job of learning a new way which is more system friendly or simply living with it as it is, instead of just the way I want it.

Important is how much cost is involved with this flexibility to see if it is worth it. Now for ant I can say, that even then not really complicated groovy build is already too much. The cost function is quite bad for ant... meaning you start low and climb very soon very high. I am sure m2 is better here... but I think also it is not cheap enough. Of course it is just a personal feeling and often this comes from a lack of knowledge, which often comes from a lack of support and documentation.

Hrm... well I know that some of the M2 plugin docs are poor, though it does provide some automated tools to generate basic configuration details. Though, there are books about Maven just like there are about Ant. The user's list is quite active, so is the IRC channel on freenode. So support seems to be on par with Ant there too. Even have companies which provide professional support where its needed.

I don't really believe that M2 convetions and/or restrictions take away from any easy usage though.

if easy usage includes that you do not have to restructure your project, then it takes away ;)

Depends a lot on how the original project is structured to start with. If you are hinting at the work I did as a POC for groovy-core, most of that restructuring was to show how a big codebase could be split up into smaller chunks. If you don't want smaller chunks, then most ant-ish structures will work, though as I have mentioned before not using the standard locations (like src/main/java, src/test/java, etc) for things means adding more overrides to the default pom.xml configuration. Not a big deal, but using the standard locations does simplify configuration.

Surely it does add some extra xml muck, but IMO its easier to configure a simple M2 pom to compile a project than it is to configure Ant.

you may have seen already, that I am not so keen about the configuration approach at all.

Sure ;-) But I'm just warning that a code-based system is eventually going to atrophy just like the Ant system has, and will become more and more complicated for new folks to comprehend and maintain. Again, my opinion... I've been wrong before though :-P

You don't have to know anything about plugins (or tasks) just the basic pom elements to set the groupId, artifactId, version... and where to put your source files. Fairly easy IMO... and really not all that different from Ant, where you have to know which tasks to configure and how to configure them, as well as pick some place to put your files. M2 removes the need for folks to know about plugins or configuration for simple projects, simply make a pom.xml and put your goodies into src/main/java or src/test/java and you are done. IMO that is much easier. When things get more complicated I believe its about the same for Ant or M2 (or even M2) have to *know* more about the tool, how to configure, what to configure, etc.

sure, yes... but we already agreed that in simple cases anything will do, and I also said that for the intended use case, these tools all shine. It gets interesting when you leave the road they built. And since m2 wants to enforce something you have to climb a wall to even leave the road... or not?

Just curious, what are the enforcement bits in M2 which have caused you problems in the past?

You really can make M2 do most anything you want... its just a matter of how much extra work you want to put in to make it behave exactly how you want... vs. using the standards and letting M2 do the work for you.

And M2 does allow you a lot of flexibility, but really... if M2 is an apple and you want orange juice, good luck trying to make it happen. You can probably do it, but you are going to end up with a messy problem on your hands later, and you will probably end up hating the tool in the progress. So why bother? If you follow M2 conventions, you can end up with an easy to use, easy to maintain system, if you don't... well its a lot more work.

because I don't want a build tool to force me to use a given structure. If it does not support my structure, that is ok. But if the tool makes it even more complicated in that case, then the tools is simply nothing for me.

What structure are you referring too?

yes, but the conclusions are very different. While m2 seems to follow the idea that build logic should not be in the build file and that a build should be only a configuration, I intend go the other way and say a build is almost nether only a configuration and you want to have build logic in your build file. So I think m1 didn't suck because configuration and build logic was mixed,

No, configuration was in project.xml and build logic was in maven.xml, I don't really consider that mixed at all.

I think a build system can be most powerful if it is based on a programming language, allowing you to write tasks/plugins without media breach. Where you can hook into the build logic itself to bend it to do your non standard things... maybe even making a more or less new build system out of it, if you need that. So instead of avoiding logic in the build file, the build file should consist of build logic. That is completely against the ideas in maven. But most people using build files are more or less good programmers and why should this potential power not be used?

Yikes... when I had a *normal* corporate job I would almost never let folks go hack on build files for the very reason that I found that most were more or less poor programers, sloppy, only caring about their own job. So putting build logic into a file which anyone could drop random muck into would have been a catastrophe ;-) But maybe I just had bad luck finding jobs with smart people :-P

and did they wrote normal programs too? Didn't you have the same problem there too? I mean why is a bad programmer more of a catastrophe when he writes a build script, then when he write any normal program? I think you have bad luck in both cases. And even in m2 he can really mess things up if m2 is as flexible as you state

Sure, well any build tool can fall under this, if a programmer (good/ bad/whatever) mucks things up, well it will break. I was just pointing out that because most of the bits in pom.xml are not really code-like, but configuration-like, that its harder for folks to just put in random things, as usually when loading the pom.xml validation will barf. Like I can just litter my build with System.out.println() stuff, where in a pure code system I could certainly do that right? Sure you could define a ton of plugin executions to invoke groovy:execute and run a little System.out.println() script, but sounds insane to most programmers ;-)

and certainly its not possible to have one build configuration which fits everyones personal taste.

ehm... I think that is the wrong way to see it. What you need is a build that works for as many people as possible. If you don't need to change the build, then you don't care about the build internals as long as it works.

So XML configuration is fine then as long as it works? Or say hessian configuration driven system or rot13? All might work, but not everyone is going to be happy with how its done.

For me its more like code-style. Any style generally will work, so will any build tool, but not every developer is going to like your code-style and there certainly is no code-style that everyone likes. Its all personal preference na.

If you had said that there is not one project layout that will fit everyone's taste, then yes, I agree with you. If you had said there is not one build file everyone will like, then again I agree with you. But is this really important?

Build configuration IMO includes the build file, the structure, whatever is needed to take your source and produce your results.

Important is what the one that changes the build file likes. And I think even if gradle would become very famous, there will still be people using m2 or ant.

I would say that what is important is the longevity of the build configuration over the lifetime of the project, in terms of functionality, maintainability and developer comprehension. In open- source, the needs of the many outweigh the needs of the one (yes, live long and prosper and all that too).

In smaller groups, where folks pretty much know about all of the codebase this might work. But in larger enterprise or open-source groups this is almost certainly a recipe for disaster, as you get new folks coming in all the time, they go look at build goo and don't understand it, or worse yet don't understand it and try to change it.

and why do you think that will not happen for m2? We already found that many people do wrong things because they don't know the documentation.

Misuse or misunderstand will happen with any build tool if folks have not used it before or have not read the documentation. But since M2 is convention-driven (for the most part) if you know the convention, then you will have a much easier time figuring out what is actually going on in a build than you would otherwise. I could be wrong. Could be something mystical about M2 that just confuses the crap out of people. I can't tell, I know its internals far to well to be a judge of that.

For the record though, I was confused as hell by M1 way back when I was making complex Ant builds, and before that confused as hell by Ant when I was making Make builds. The transition from M1 to M2 was a little smoother but it still kinda confused me. So I can extrapolate and say that most/all build tools can be confusing at first, or that I'm just an idiot... or should drink less beer while trying out new build tools ;-)

Developer comprehension of build tools is IMO very important, especially when they have to muck with it, and IMO a build tool where you code up everything in any way you feel like at the moment, will probably cause some problems for new folks to understand easily.

which can be said for any program!

Sure of course, I'm not suggesting that M2 gives folks grand clarity, only that I think there is a slight advantage due to its convention based muck.

And I doubt m2 will help me with all the non standard things I would like to do. So a switch to gradle seems to be the right thing for me.

What non-standard things are those? Is not simply using the groovy:execute goal in a pom going to be sufficient for most non- standard things? Can you give me an example?

a non standard thing... how about keeping the current project structure ;)

As I mentioned earlier that is possible. But since groovy-core isn't quite conforming to the M2 standard, things like the location of main and test sources will need to be configured, location of resources needs to be configured, etc. You can do it sure, do I recommend it no. Its easier to just mv sources to src/main/java than it is to maintain the configuration of src/java, etc.

For splitting up the distributable artifacts into smaller chunks, that really should be implemented with a structure change, as the amount of custom configuration needed to manage poms, build jars, install jars, deploy jars, generate documentation, etc becomes quickly very very difficult to maintain. This is one of those learning things, that you can make the build configuration a whole lot simpler and easy to manage by simply following the conventions.

But by no means are you forced to do that... just the returns on the investment won't be very high, and will probably end up costing about the same as the current Ant build when you have to go and fix it in a few years.

--jason