atom feed27 messages in org.apache.hadoop.hdfs-devRe: Heads up - 2.0.5-beta
FromSent OnAttachments
Arun C MurthyApr 25, 2013 6:34 pm 
Suresh SrinivasApr 25, 2013 6:36 pm 
Arun C MurthyApr 26, 2013 11:02 am 
Arun C MurthyApr 26, 2013 11:15 am 
Eli CollinsApr 26, 2013 12:07 pm 
Luke LuApr 26, 2013 1:37 pm 
Suresh SrinivasApr 26, 2013 2:42 pm 
Eli CollinsApr 26, 2013 3:09 pm 
Konstantin ShvachkoApr 26, 2013 4:00 pm 
Konstantin ShvachkoApr 26, 2013 4:34 pm 
Arpit AgarwalApr 26, 2013 4:52 pm 
Arun C MurthyApr 26, 2013 6:06 pm 
Arun C MurthyApr 26, 2013 6:27 pm 
Arun C MurthyApr 28, 2013 9:03 pm 
Konstantin ShvachkoApr 30, 2013 4:28 pm 
Konstantin ShvachkoMay 1, 2013 12:13 pm 
Arun C MurthyMay 1, 2013 1:15 pm 
Konstantin ShvachkoMay 1, 2013 4:08 pm 
Arun C MurthyMay 1, 2013 6:24 pm 
Chris DouglasMay 2, 2013 12:06 am 
Konstantin ShvachkoMay 2, 2013 2:08 am 
Konstantin ShvachkoMay 2, 2013 2:11 am 
Arun C MurthyMay 2, 2013 3:05 pm 
Suresh SrinivasMay 2, 2013 3:32 pm 
Chris DouglasMay 2, 2013 5:52 pm 
Konstantin ShvachkoMay 3, 2013 1:06 am 
Robert EvansMay 3, 2013 8:23 am 
Subject:Re: Heads up - 2.0.5-beta
From:Konstantin Shvachko (shv.@gmail.com)
Date:May 1, 2013 4:08:30 pm
List:org.apache.hadoop.hdfs-dev

On Wed, May 1, 2013 at 1:15 PM, Arun C Murthy <ac@hortonworks.com> wrote:

On Apr 30, 2013, at 4:28 PM, Konstantin Shvachko wrote:

If the next release has to be 2.0.5 I would like to make an alternative proposal, which would include - stabilization of current 2.0.4 - making all API changes to allow freezing them post 2.0.5 And nothing else.

I think it's hard to clearly define - 'nothing else'. For e.g.

YARN-398/YARN-392. It's a 'feature' but worth putting in right-away since it so low-risk. MAPREDUCE-5108 is a 'feature' but is critical for ensuring a smooth transition from MR1 to MR2 etc. etc.

Don't see contradictions to the plan here. Both YARN-398, YARN-392 are important optimizations. They require API changes, so it is better to commit them into 2.0.5. If RM sees a low risk in including the implementations, I don't see a problem. MAPREDUCE-5108 as a compatibility issue should go in, imho.

Rather than get tied up in knots, it would be useful to go with API

changes as *mandatory* and everything as optional and not hold up the release for them (which is what we have done in hadoop-2.x since forever). IAC, risk should be quantified by people working on individual jiras.

People were and are complaining that every release 2.0 was incompatible with the previous. I would not say any API changes, but those that help locking them post 2.0.5 "everything as optional" is too wide in my understanding as it can be anything, including changes that break downstream components. In order to avoid that, the changes should be minimized to bug fixes.

Also, it will be useful to actually start testing things as they stand

rather than continue to discuss endlessly - would you be willing to help test on of hadoop-2.x? If so, could you please share your plans? I'm sure everyone will appreciate it.

Thank you for asking. We did comprehensive testing internally of hadoop 2.0.3 and hadoop 2.0.4 as they stand now using standard Hadoop tools and BigTop for integration. Cos introduced Jenkins build for branch 2, which wasn't set up. Testing things as they currently EVOLVE doesn't make sense to me, as the volume of changes proposed will invalidate any current testing. Endless discussions are not productive. I put up the vote for this release plan.

Thanks, --Konstantin