atom feed23 messages in org.apache.incubator.whirr-userRe: Amazon EC2 and HTTP 503 errors: R...
FromSent OnAttachments
Paolo CastagnaOct 28, 2011 8:32 am 
Andrei SavuOct 29, 2011 5:37 am 
Paolo CastagnaOct 31, 2011 4:41 am 
Andrei SavuOct 31, 2011 7:31 am 
Paolo CastagnaNov 1, 2011 2:52 am 
Andrei SavuNov 1, 2011 5:14 am 
Paolo CastagnaNov 2, 2011 7:35 am 
Andrei SavuNov 2, 2011 7:54 am 
Paolo CastagnaNov 2, 2011 7:58 am 
Andrei SavuNov 2, 2011 8:04 am 
Andrei SavuNov 2, 2011 8:11 am 
Andrei SavuNov 2, 2011 8:11 am 
Paolo CastagnaNov 2, 2011 8:25 am 
Andrei SavuNov 2, 2011 8:28 am 
Paolo CastagnaNov 2, 2011 8:33 am 
Andrei SavuNov 2, 2011 8:38 am 
Paolo CastagnaNov 2, 2011 8:51 am 
Andrei SavuNov 2, 2011 8:56 am 
Paolo CastagnaNov 2, 2011 9:03 am 
Andrei SavuNov 2, 2011 9:04 am 
Andrei SavuNov 2, 2011 9:05 am 
Paul BaclaceNov 2, 2011 10:18 pm 
Andrei SavuNov 3, 2011 12:57 am 
Subject:Re: Amazon EC2 and HTTP 503 errors: RequestLimitExceeded
From:Andrei Savu (savu@gmail.com)
Date:Oct 29, 2011 5:37:06 am
List:org.apache.incubator.whirr-user

Paul -

I think you are hitting an upper bound on the size of the clusters that can be started with Whirr right now.

One possible workaround you can try is to enable lazy image fetching in jclouds: http://www.jclouds.org/documentation/userguide/using-ec2

I have created a new JIRA issue so that we can add this automatically when the image-id is known: https://issues.apache.org/jira/browse/WHIRR-416

What if you start a smaller size cluster but with more powerful machines?

Cheers,

-- Andrei Savu

On Fri, Oct 28, 2011 at 6:32 PM, Paolo Castagna < cast@googlemail.com> wrote:

Hi, it's me again, I am trying to use Apache Whirr 0.6.0-incubating to start a 20 nodes Hadoop cluster on Amazon EC2.

Here is my recipe:

---- whirr.cluster-name=hadoop whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,20 hadoop-datanode+hadoop-tasktracker whirr.instance-templates-max-percent-failures=100 hadoop-namenode+hadoop-jobtracker,50 hadoop-datanode+hadoop-tasktracker whirr.max-startup-retries=1 whirr.provider=aws-ec2 whirr.identity=${env:AWS_ACCESS_KEY_ID_LIVE} whirr.credential=${env:AWS_SECRET_ACCESS_KEY_LIVE} whirr.hardware-id=m1.large whirr.image-id=eu-west-1/ami-ee0e3c9a whirr.location-id=eu-west-1 whirr.private-key-file=${sys:user.home}/.ssh/whirr whirr.public-key-file=${whirr.private-key-file}.pub whirr.hadoop.version=0.20.204.0 whirr.hadoop.tarball.url= http://archive.apache.org/dist/hadoop/core/hadoop-${whirr.hadoop.version}/hadoop-${whirr.hadoop.version}.tar.gz

----

I see a lot of these errors:

org.jclouds.aws.AWSResponseException: request POST https://ec2.eu-west-1.amazonaws.com/ HTTP/1.1 failed with code 503, error: AWSError{requestId='b361f3f6-73f1-4348-964a-31265ec70eeb', requestToken='null', code='RequestLimitExceeded', message='Request limit exceeded.', context='{Response=, Errors=}'} at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:74) at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:71) at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.shouldContinue(BaseHttpCommandExecutorService.java:200) at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:165) at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:134) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)

I have more than 20 slots available on this Amazon account.

Is it Whirr sending requests too fast to Amazon?

How can I solve this problem?

Regards, Paolo