You can raise the time limit:

http://docs.marklogic.com/5.0doc/docapp.xqy#display.xqy?fname=http://pubs/5.0doc/xml/admin/http.xml&query=request+timeout

Turning to your query, I see some repeated work that could probably be factored
out.

Let's remove that duplicate name lookup: the result will be constant for every
$row in a given table_data element, and I presume there are many of those.

for $table at $index in /*/*/table_data let $table-name := $table/@name/string() for $row in $table/row let $record := element { $table-name } { $row/field[text()]/element { @name } { text() } } ...

This part is especially troubling and probably adds a lot of duplicated work:
aren't you going back to the entire database again?

The semicolon at the end is superfluous. I think this might do what you want:

... let $uri = concat( replace(xdmp:path($row), '(\[[0-9]+\])', ''), '/', $index) return xdmp:document-insert($uri, $xml)

That removes the uuid functions too. But if you do want a uuid implementation
that should be slightly faster, take a look at
http://markmail.org/message/mql6teskkwb574na

Given that all the work is in-memory except the document-insert, you might
actually be able to do this faster by not ingesting the table data first. It's
all one large document, right? You can read that from the filesystem. I don't
know if that will make the transform query faster or slower, but it will avoid
the need to insert all that table_data first.

for $table at $index in xdmp:document-get('/tmp/export.xml')/*/*/table_data let $table-name := $table/@name/string() for $row in $table/row let $record := element { $table-name } { $row/field[text()]/element { @name } { text() } } let $uri := concat( replace(xdmp:path($row), '(\[[0-9]+\])', ''), '/', $index) return xdmp:document-insert($uri, $record)

Finally, (as Tim just proposed in his reply) I would probably move the actual
insert into a spawned task. This requires a little more setup, but allows you to
run the XML processing in timestamped, lock-free mode. Each doc-insert would
then run asynchronously on the task server threads. You might have to increase
the task server queue size, which defaults to 100,000 I think. Otherwise you are
likely to see a MAXTASKS error. You might also want to increase the number of
task server threads. For this workload I would try one thread per CPU,
initially.

http://docs.marklogic.com/5.0doc/docapp.xqy#search.xqy?query=xdmp:spawn

(: task.xqy, a module on the filesystem in the app-server module root :) xquery version "1.0-ml"; declare variable $URI external ; declare variable $NEW external ; xdmp:document-insert($URI, $NEW)

(: query console :) for $table at $index in xdmp:document-get('/tmp/export.xml')/*/*/table_data let $table-name := $table/@name/string() for $row in $table/row let $record := element { $table-name } { $row/field[text()]/element { @name } { text() } } let $uri := concat( replace(xdmp:path($row), '(\[[0-9]+\])', ''), '/', $index) return xdmp:spawn('task.xqy', (xs:QName('URI'), $uri, xs:QName('NEW'), $record))

-- Mike

On 20 Feb 2012, at 09:23 , Todd Gochenour wrote: