 | Start a set with this search |
 | Include this search in one of my sets |
 | Exclude this search from one of my sets |
 | Permalink to these results Paste this link in email or IM: |
 | Atom feed for tracking future search results Paste this URL into your reader: |
7 messages in com.googlegroups.boto-usersRe: SimpleDB concurrency| From | Sent On | Attachments |
|---|---|---|
| Herb...@gmail.com | Jan 16, 2008 11:33 pm | |
| Herb...@gmail.com | Jan 17, 2008 3:07 pm | |
| Herb...@gmail.com | Jan 17, 2008 4:37 pm | |
| Herb...@gmail.com | Jan 17, 2008 10:38 pm | |
| Herb...@gmail.com | Jan 18, 2008 12:52 pm | |
| mitch | Jan 21, 2008 2:36 pm | |
| mitch | Jan 21, 2008 4:42 pm |
| Permalink for this message Paste this link in email or IM: |
| Permalink for this thread Paste this link in email or IM: |
| Atom feed for this thread Paste this URL into your reader: |
| Subject: | Re: SimpleDB concurrency | Actions... |
|---|---|---|
| From: | Herb...@gmail.com (Herb...@googlemail.com) | |
| Date: | Jan 17, 2008 3:07:07 pm | |
| List: | com.googlegroups.boto-users | |
Mitchell, thanks for the quick answer.
I'm reading up on threads, I'll probably implement this the simplest way possible for now, just some code that gets it all in threads, and returns it assembled, like you mention. I'll post some code here once I'm finished.
Ty, Herb
On Jan 17, 4:52 am, "Mitchell Garnaat" <mitc...@gmail.com> wrote:
There is nothing in boto that prevents you from doing this but there is also nothing that facilitates it. I would definitely like to find a way to make this easier in boto by somehow managing/hiding the complexity of the threading.
I haven't thought about it a lot but perhaps something simple like a method that takes a list of item names (or even a query that produces a list of item names) and then returns a list Item objects with all attributes returned. Behind the scenes it could spawn multiple threads and wait for all of them to finish and assemble the results.
If anyone has other ideas about how we can introduce some concurrency into the SimpleDB library, I'd love to hear them.
Mitch
On Jan 17, 2008 2:33 AM, Herb...@gmail.com <Herb...@googlemail.com> wrote:
I have a question, I read the following comment in the amazon webservices forum:
<quote> With SimpleDB, you'd issue your query, get a list of 50 item keys/ids. Then you'd issue 50 more queries, one for each item returned. Now I'm guessing doing this serially would probably take ages (ages meaning seconds). Following Mitch's guidance, you could do this all at once in 50 threads, wait for the results from all 50 threads, then return your results page and I would imagine this would speed things up significantly, but would it still be anywhere close to the single MySQL query? Considering this is all over http protocol with xml overhead. </quote>
Is this currently possible with boto? For me, queries that return more than about 10 result sets, and then have to do an extra 10 queries to get 10 items, run slower than I'd want. Could this be solved with threads in boto, and is it planned as a boto feature?
Thanks,