6 messages in net.java.dev.jna.usersRE: [jna-users] Re: JNA (fork and exec)
FromSent OnAttachments
Timothy WallAug 27, 2007 12:59 pm 
Wayne MeissnerAug 27, 2007 5:28 pm 
Jagadeeswaran RajendiranAug 28, 2007 4:56 pm 
Timothy WallAug 28, 2007 6:51 pm 
Jagadeeswaran RajendiranAug 29, 2007 5:49 pm 
Timothy WallAug 29, 2007 8:48 pm 
Actions with this message:
Paste this link in email or IM:
Paste this link in email or IM:
Atom feed for this thread
Paste this URL into your reader:
Subject:RE: [jna-users] Re: JNA (fork and exec)Actions...
From:Jagadeeswaran Rajendiran (r.ja@loni.ucla.edu)
Date:Aug 28, 2007 4:56:13 pm
List:net.java.dev.jna.users

Thanks for the prompt replies, guys.

So, here are some more results/questions:

1. I tried using execl as follows.

libcInstance.execl("/bin/cp", "cp", "source", "destination");

But, it comes back with an errno of BAD_ADDRESS

If I change it to execv, it works (with the original problem of occassionally not working)

I need to use execv anyway, coz my program is going to spawn other programs with unknown number of arguments...and execv is the most convenient and functional coz it takes an array!

2. In the code excerpt, execv never returns, which probably then means that it was never executed

3. On a related note, as suggested, I tried running my application using strace:

"strace java -jar Pipeline.jar"

And it NEVER shows JNA entering fork or execv system calls, although I (after tracing JNA source code) can see that JNA goes through Function::invoke etc, calls native code...but I am not sure why I am not able to see the fork/execv call traces through strace...any ideas?

4. Also, heres the stack dump of a successful run:

CommandThread:JNA_User_0_0_1 is the thread that forks. The new thread (before execv) renames itself to Newly_Execed_Thread. I forced Function::invoke (in JNA) to throw an exception at the very start
(line 154) and at the very end (line 228) of every invocation.

HEres my interpretation of the stack dumps etc:

1. When JNA's fork method is called the child seems to return first from
Function::invoke. This always happens on successful runs. We can see that coz after fork you dont see the statement (I am the parent) until the child () completes the call to execv...

2. When the child calls execv Function::invoke calls the native::invoke
method and is done...this somehow triggers the parent thread to wake up and
return from the fork and then continue execution...coz theres a Function::invoke (line
228) exception being thrown by Proxy::fork and the line "I am the parent" gets
printed...

3. waitpid seems to work...

--

native library -c CommandThread:JNA_User_0_0_1 - is forking now CommandThread:JNA_User_0_0_1method name start - fork CommandThread:JNA_User_0_0_1library::before sync CommandThread:JNA_User_0_0_1library::invoke getting method CommandThread:JNA_User_0_0_1library::invoke defined callingconvention CommandThread:JNA_User_0_0_1NativeLibrary :: before sync :: getfunction CommandThread:JNA_User_0_0_1NativeLibrary :: before get CommandThread:JNA_User_0_0_1NativeLibrary :: after get CommandThread:JNA_User_0_0_1NativeLibrary :: created new function CommandThread:JNA_User_0_0_1NativeLibrary :: put new function CommandThread:JNA_User_0_0_1NativeLibrary :: before return CommandThread:JNA_User_0_0_1library::invoke nativelibrary getting function CommandThread:JNA_User_0_0_1library::invoke putting function CommandThread:JNA_User_0_0_1library::end of sync CommandThread:JNA_User_0_0_1method name end - fork before returning java.lang.Exception at com.sun.jna.Function.invoke(Function.java:154) at com.sun.jna.Library$Handler.invoke(Library.java:154) at $Proxy0.fork(Unknown Source) at
execution.AbstractCommandThread.runCommand(AbstractCommandThread.java:227) at execution.AbstractCommandThread.run(AbstractCommandThread.java:550) at java.lang.Thread.run(Thread.java:619) at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) CommandThread:JNA_User_0_0_1 :: function :: invoke java.lang.Exception at com.sun.jna.Function.invoke(Function.java:228) at com.sun.jna.Library$Handler.invoke(Library.java:154) at $Proxy0.fork(Unknown Source) at
execution.AbstractCommandThread.runCommand(AbstractCommandThread.java:227) at execution.AbstractCommandThread.run(AbstractCommandThread.java:550) at java.lang.Thread.run(Thread.java:619) at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) CommandThread:JNA_User_0_0_1 :: function :: before returning result CommandThread:JNA_User_0_0_1 - execing right now New_Execed Thread - just before exec New_Execed Threadmethod name start - execv New_Execed Threadlibrary::before sync New_Execed Threadlibrary::invoke getting method New_Execed Threadlibrary::invoke defined callingconvention New_Execed ThreadNativeLibrary :: before sync :: getfunction New_Execed ThreadNativeLibrary :: before get New_Execed ThreadNativeLibrary :: after get New_Execed ThreadNativeLibrary :: created new function New_Execed ThreadNativeLibrary :: put new function New_Execed ThreadNativeLibrary :: before return New_Execed Threadlibrary::invoke nativelibrary getting function New_Execed Threadlibrary::invoke putting function New_Execed Threadlibrary::end of sync New_Execed Threadmethod name end - execv function :: concatVarArgs :: arg type - class [Ljava.lang.String; function :: concatVarArgs :: arg is Array function :: concatVarArgs :: arg is NOT prim Array function :: concatVarArgs :: arg is String Array before returning java.lang.Exception at com.sun.jna.Function.invoke(Function.java:154) at com.sun.jna.Library$Handler.invoke(Library.java:154) at $Proxy0.execv(Unknown Source) at
execution.AbstractCommandThread.runCommand(AbstractCommandThread.java:243) at execution.AbstractCommandThread.run(AbstractCommandThread.java:550) at java.lang.Thread.run(Thread.java:619) at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) New_Execed Thread :: function :: invoke java.lang.Exception at com.sun.jna.Function.invoke(Function.java:228) at com.sun.jna.Library$Handler.invoke(Library.java:154) at $Proxy0.fork(Unknown Source) at
execution.AbstractCommandThread.runCommand(AbstractCommandThread.java:227) at execution.AbstractCommandThread.run(AbstractCommandThread.java:550) at java.lang.Thread.run(Thread.java:619) at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) CommandThread:JNA_User_0_0_1 :: function :: before returning result CommandThread:JNA_User_0_0_1 - I am the parent execution.CommandThread@145b02fran command CommandThread:JNA_User_0_0_1method name start - waitpid CommandThread:JNA_User_0_0_1library::before sync CommandThread:JNA_User_0_0_1library::invoke getting method CommandThread:JNA_User_0_0_1library::invoke defined callingconvention CommandThread:JNA_User_0_0_1NativeLibrary :: before sync :: getfunction CommandThread:JNA_User_0_0_1NativeLibrary :: before get CommandThread:JNA_User_0_0_1NativeLibrary :: after get CommandThread:JNA_User_0_0_1NativeLibrary :: created new function CommandThread:JNA_User_0_0_1NativeLibrary :: put new function CommandThread:JNA_User_0_0_1NativeLibrary :: before return CommandThread:JNA_User_0_0_1library::invoke nativelibrary getting function CommandThread:JNA_User_0_0_1library::invoke putting function CommandThread:JNA_User_0_0_1library::end of sync CommandThread:JNA_User_0_0_1method name end - waitpid function :: concatVarArgs :: arg type - class java.lang.Integer function :: concatVarArgs :: arg is NOT Array function :: concatVarArgs :: arg is NOT prim Array function :: concatVarArgs :: arg is NOT String Array before returning java.lang.Exception at com.sun.jna.Function.invoke(Function.java:154) at com.sun.jna.Library$Handler.invoke(Library.java:154) at $Proxy0.waitpid(Unknown Source) at
execution.AbstractCommandThread.waitForCompletion(AbstractCommandThread.java:402) at execution.AbstractCommandThread.run(AbstractCommandThread.java:555) at java.lang.Thread.run(Thread.java:619) at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) CommandThread:JNA_User_0_0_1 :: function :: invoke java.lang.Exception at com.sun.jna.Function.invoke(Function.java:228) at com.sun.jna.Library$Handler.invoke(Library.java:154) at $Proxy0.waitpid(Unknown Source) at
execution.AbstractCommandThread.waitForCompletion(AbstractCommandThread.java:402) at execution.AbstractCommandThread.run(AbstractCommandThread.java:555) at java.lang.Thread.run(Thread.java:619) at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) CommandThread:JNA_User_0_0_1 :: function :: before returning result

--

5. Heres the command stack from the unsuccessful run:

You will see that execv was not executed in this particular run...coz there is one Exception (line 154) and 2 Exceptions (for line 228) which means there were two threads created but the child died before it could execute ....and I am not sure why?

Enode - JNA_User_0has started running JNA_User_0spawning command children this - spawning - my child - JNA_User_00 this - spawnED - my child - JNA_User_00 JNA_User_0awaiting command child - CommandThread:JNA_User_0_0_111 - is forking now CommandThread:JNA_User_0_0_111method name start - fork CommandThread:JNA_User_0_0_111library::before sync CommandThread:JNA_User_0_0_111library::invoke getting method CommandThread:JNA_User_0_0_111library::end of sync CommandThread:JNA_User_0_0_111method name end - fork before returning java.lang.Exception at com.sun.jna.Function.invoke(Function.java:154) at com.sun.jna.Library$Handler.invoke(Library.java:154) at $Proxy0.fork(Unknown Source) at
execution.AbstractCommandThread.runCommand(AbstractCommandThread.java:227) at execution.AbstractCommandThread.run(AbstractCommandThread.java:550) at java.lang.Thread.run(Thread.java:619) at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) CommandThread:JNA_User_0_0_111 :: function :: invoke java.lang.Exception at com.sun.jna.Function.invoke(Function.java:228) at com.sun.jna.Library$Handler.invoke(Library.java:154) at $Proxy0.fork(Unknown Source) at
execution.AbstractCommandThread.runCommand(AbstractCommandThread.java:227) at execution.AbstractCommandThread.run(AbstractCommandThread.java:550) at java.lang.Thread.run(Thread.java:619) at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) CommandThread:JNA_User_0_0_111 :: function :: before returning result java.lang.Exception at com.sun.jna.Function.invoke(Function.java:228) at com.sun.jna.Library$Handler.invoke(Library.java:154) at $Proxy0.fork(Unknown Source) at
execution.AbstractCommandThread.runCommand(AbstractCommandThread.java:227) at execution.AbstractCommandThread.run(AbstractCommandThread.java:550) at java.lang.Thread.run(Thread.java:619) at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) CommandThread:JNA_User_0_0_111 :: function :: before returning result CommandThread:JNA_User_0_0_111 - I am the parent execution.CommandThread@c4396aran command CommandThread:JNA_User_0_0_111method name start - waitpid CommandThread:JNA_User_0_0_111library::before sync CommandThread:JNA_User_0_0_111library::invoke getting method CommandThread:JNA_User_0_0_111library::end of sync CommandThread:JNA_User_0_0_111method name end - waitpid function :: concatVarArgs :: arg type - class java.lang.Integer function :: concatVarArgs :: arg is NOT Array function :: concatVarArgs :: arg is NOT prim Array function :: concatVarArgs :: arg is NOT String Array before returning java.lang.Exception at com.sun.jna.Function.invoke(Function.java:154) at com.sun.jna.Library$Handler.invoke(Library.java:154) at $Proxy0.waitpid(Unknown Source) at
execution.AbstractCommandThread.waitForCompletion(AbstractCommandThread.java:402) at execution.AbstractCommandThread.run(AbstractCommandThread.java:555) at java.lang.Thread.run(Thread.java:619) at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) CommandThread:JNA_User_0_0_111 :: function :: invoke java.lang.Exception at com.sun.jna.Function.invoke(Function.java:228) at com.sun.jna.Library$Handler.invoke(Library.java:154) at $Proxy0.waitpid(Unknown Source) at
execution.AbstractCommandThread.waitForCompletion(AbstractCommandThread.java:402) at execution.AbstractCommandThread.run(AbstractCommandThread.java:555) at java.lang.Thread.run(Thread.java:619) at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) CommandThread:JNA_User_0_0_111 :: function :: before returning result

-----Original Message----- From: Timothy Wall [mailto:twal@dev.java.net] Sent: Mon 8/27/2007 6:54 PM To: Jagadeeswaran Rajendiran Subject: Fwd: [jna-users] Re: JNA (fork and exec)

Forwarded from the use@jna.dev.java.net mailing list:

Begin forwarded message:

From: Wayne Meissner <wmei@gmail.com> Date: August 27, 2007 8:28:27 PM EDT To: use@jna.dev.java.net Subject: Re: [jna-users] Re: JNA (fork and exec) Reply-To: use@jna.dev.java.net

Timothy Wall wrote:

On Aug 24, 2007, at 8:33 PM, Jagadeeswaran Rajendiran wrote:

-- Relevant Code --

//linux is the libc.INSTANCE

if((childPid = linux.fork()) == 0){ try{ System.out.println(Thread.currentThread ().getName() + " - execing right now"); Thread.currentThread().setName("New_Execed Thread"); System.out.println(Thread.currentThread ().getName() + " - just before exec"); System.out.println("return value" + linux.execv("/bin/sleep", testArray));

Try using execl instead. JNA will be treating testArray as a vararg array, which means it is translating the above into:

linux.execv("/bin/sleep", testArray[0], testArray[1], ..., null);

If you use execl(), that will actually do the right thing.

}catch(Exception e){ System.out.println("exception in abs"); e.printStackTrace(); }

You want to add: finally { System.exit(1); }

If the exec _does_ fail for some reason, or an exception gets thrown, then you want the new VM (created by fork()) to exit - otherwise you will have two VMs running with the same set of open files, shared memory, etc. That would be A Very Bad Thing.