atom feed20 messages in net.java.dev.rococoa.usersRe: "Invalid memory access" in NSAuto...
FromSent OnAttachments
Duncan McGregorJul 27, 2009 8:48 am 
Michael NagelJul 27, 2009 9:29 am 
Duncan McGregorJul 27, 2009 12:44 pm 
David KocherJul 27, 2009 3:35 pm 
Duncan McGregorJul 27, 2009 3:43 pm 
David KocherJul 27, 2009 4:08 pm 
Andrew ThompsonJul 27, 2009 7:02 pm 
Michael NagelJul 27, 2009 9:28 pm 
Duncan McGregorJul 28, 2009 1:29 am 
Duncan McGregorJul 28, 2009 3:42 am 
Duncan McGregorJul 28, 2009 4:07 am 
Duncan McGregorJul 28, 2009 4:33 am 
Andrew ThompsonJul 28, 2009 4:41 am 
Duncan McGregorJul 28, 2009 5:01 am 
Duncan McGregorJul 28, 2009 11:55 am 
Andrew ThompsonJul 28, 2009 6:17 pm 
Duncan McGregorAug 10, 2009 3:28 am 
Andrew ThompsonAug 10, 2009 8:56 pm 
Duncan McGregorSep 1, 2009 7:59 am 
Andrew ThompsonSep 1, 2009 6:12 pm 
Subject:Re: "Invalid memory access" in NSAutoreleasePool.finalize
From:Andrew Thompson (lord@mac.com)
Date:Jul 28, 2009 4:41:49 am
List:net.java.dev.rococoa.users

I'm going to try to have a go at reading what's below tonight... I think you're right. There's some interaction between the death of the Thread (and NSThread) and the autorelease pool being explicitly thread scoped.

However, sorry to harp on about this, but can someone explain why our current finalizer strategy is correct with regular boring old objectd?

I'll try to be clear, if I wrote this in Objective C

NSString *str = [[NSString alloc] init];

[str release]; [str release];

It is invalid code, and may crash, right? No autorelease pools or anything complex going on. I do an alloc/init so I own the object, then I release it twice.

Now, if I wrote this in Rococoa

NSString str = NSString.CLASS.alloc().init(); //ok, so the current version doesn't have init() but of course we could add it str.release();

My code is valid so far... but now along comes the finalizer...

Foundation.cfRelease(ocInstance);

As far as I can tell that's identical to the Objective C code given above. No autorelease pools involved... we simply alloc/init and object, taking ownership, then release it... which is obviously valid code... but then the finalizer releases it again.

Again, I'll ask, am i missing something obvious? Right now I can't explain to myself why our code isn't just incorrect in the simplest possible case.

On Jul 28, 2009, at 6:43 AM, Duncan McGregor wrote:

Thanks for following up Michael.

On 28 Jul 2009, at 05:28, Michael Nagel wrote:

I don't think this is an NSString issue because a) I initially encountered it with an NSURL and an NSSpeechSynthesizer and b) it crashes sometimes even when I don't use the string line. I just looked for a n example that's as simple as possible, so I used String instead of one of those two.

In fact if you remove the NSString altogether it still crashes.

The regression makes me think that this might be a case of order of execution, i.e. that this bug appears mostly when the involved pools are finalized in a certain order.

The one thing that seems to be required is arptt.pool = null, and looking at the trace the crash does happen right after the finalize.

It seems that this crash can be distilled down to

public class NSAutoreleasePoolThreadTest {

static { RococoaTestCase.initializeLogging(); }

@Test public void drainPoolAndFinalize() { NSAutoreleasePool pool = NSAutoreleasePool.new_(); pool.drain(); WeakReference<Object> reference = new WeakReference<Object>(pool); pool = null; while (reference.get() != null) { RococoaTestCase.gc(); } RococoaTestCase.gc(); }

@Test public void drainPoolAndFinalizeOnAnotherThread() throws InterruptedException { Thread thread = new Thread("test") { public void run() { drainPoolAndFinalize(); }}; thread.start(); thread.join(); RococoaTestCase.gc(); }

@Ignore("crashes") @Test public void drainPoolOnAnotherThreadAndFinalize() throws InterruptedException { Thread thread = new Thread("test") { public void run() { NSAutoreleasePool pool = NSAutoreleasePool.new_(); pool.drain(); }}; thread.start(); thread.join(); RococoaTestCase.gc(); } }

What I think this is saying is that if you create an NSAutoreleasePool on a thread (remembering that the pools are created in a thread-local stack) then you _can_ safely let that pool be released by the finalizer, but *only* if that completes before the thread exits. Given the thread-localness of the pools, I guess that this is understandable.

I also think that Andrew has a good point. Maybe NSAutoreleasePool's reference count mechanics differ fundamentally from those of other NSObjects,

(Andrew's point? Hey - I wrote the test cases ;-) Actually the documentation says that NSAutoreleasePool is special: - pool.retain and autorelease raise an exception - pool.drain releases the contents and pops from the stack - pool.release drains

so an obvious (and admittedly nasty) attempt to fix this would be to not release NSAutoreleasePool objects at all in finalize(). More specifically, they must not even be accessed in finalize() in any way, such as getting their reference count, because they might already be released, and therefore invalid, without Rococa knowing about it.

That is the patch that David has, but while it works, I don't think that it is the answer, or at least not for that reason. I was surprised by the result that we could continually release given a retain count of 1, as I was convinced that the crash we are seeing in the finalizer was caused by releasing an already released pool. Now I'm not so sure.

If it is true that it is releasing a pool that was created on a completed thread that crashes, rather than releasing an already released pool, then we should be able to test this hypothesis:

@Ignore("crashes") @Test public void cantDrainPoolCreatedOnAFinishedThread() throws InterruptedException { final NSAutoreleasePool[] poolHolder = new NSAutoreleasePool[1]; Thread thread = new Thread("test") { public void run() { poolHolder[0] = NSAutoreleasePool.new_(); }}; thread.start(); thread.join(); poolHolder[0].drain(); }

@Test public void drainPoolCreatedOnANotFinishedThread() throws InterruptedException { final NSAutoreleasePool[] poolHolder = new NSAutoreleasePool[1]; final CyclicBarrier beforeDrain = new CyclicBarrier(2); final CyclicBarrier afterDrain = new CyclicBarrier(2); Thread thread = new Thread("test") { public void run() { poolHolder[0] = NSAutoreleasePool.new_(); await(beforeDrain); await(afterDrain); }}; thread.start(); await(beforeDrain); poolHolder[0].drain();

await(afterDrain); thread.join(); }

@Test public void drainAlreadyDrainedPoolCreatedOnANotFinishedThread() throws InterruptedException { final NSAutoreleasePool[] poolHolder = new NSAutoreleasePool[1]; final CyclicBarrier beforeDrain = new CyclicBarrier(2); final CyclicBarrier afterDrain = new CyclicBarrier(2); Thread thread = new Thread("test") { public void run() { poolHolder[0] = NSAutoreleasePool.new_(); poolHolder[0].drain(); await(beforeDrain); await(afterDrain); }}; thread.start(); await(beforeDrain); poolHolder[0].drain();

await(afterDrain); thread.join(); }

private void await(CyclicBarrier barrier) { try { barrier.await(); } catch (Exception e) { e.printStackTrace(); throw new RuntimeException(e); } }

Yeah, I'm sorry, hairy threaded tests - can anyone express this better? The key thing is that it seems to confirm my hypothesis - we can drain/release already drained pools, we can't drain/release a pool in any state created on a finished thread.

I'd be very grateful if you guys would look over these results, maybe even run the tests yourselves, and give it a bit of peer review. Does anyone have a better hypothesis that fits the evidence?

BTW - if there are any plain old use@rococoa.dev.java.net out there, sorry to have this discussion in front of you, but it's got to the stage where continuity of this thread is important. You're living on the bleeding edge of an open-source project I'm afraid ;-)

AndyT (lordpixel - the cat who walks through walls) A little bigger on the inside

(see you later space cowboy, you can't take the sky from me)