atom feed9 messages in org.perl.perl5-portersRe: Inconsistent behaviour when remov...
FromSent OnAttachments
Sébastien Aperghis-TramoniOct 26, 2005 8:28 am 
Rafael Garcia-SuarezOct 26, 2005 8:39 am 
Sébastien Aperghis-TramoniOct 26, 2005 9:31 am 
demerphqOct 26, 2005 10:39 am 
Steve PetersOct 26, 2005 10:40 am 
Yitzchak Scott-ThoennesOct 26, 2005 11:34 pm 
demerphqOct 27, 2005 12:19 am 
Yitzchak Scott-ThoennesOct 27, 2005 1:20 am 
demerphqOct 27, 2005 1:47 am 
Subject:Re: Inconsistent behaviour when removing files on Cygwin
From:Yitzchak Scott-Thoennes (stho@efn.org)
Date:Oct 27, 2005 1:20:36 am
List:org.perl.perl5-porters

On Thu, Oct 27, 2005 at 09:19:32AM +0200, demerphq wrote:

On 10/27/05, Yitzchak Scott-Thoennes <stho@efn.org> wrote:

On Wed, Oct 26, 2005 at 05:28:30PM +0200, S?bastien Aperghis-Tramoni wrote:

Hello,

While correcting a bug in Net::Pcap Makefile.PL with a Windows developer, we stumbled upon the strange behaviour of the unlink() function under Cygwin. It looks like it tries to emulates the Unix behaviour but fails to achieve it. ... Ok, the equivalent program in C produces the same results, so the error is likely to be inside Cygwin API, but shouldn't this be documented somewhere? Maybe in perlport/"DOS and Derivatives"?

perlport says it short and sweet: (implied: For portable code:) "Don't C<unlink> or C<rename> an open file." I don't think repeating that in a dosish section would be more helpful.

cygwin may defer unlinks until the file is closed or until the process ends. Exactly what it does depends on the exact flavor of windows and how reliable (cygwin thinks) the relevant win32 api calls are there.

Actually, afaiui its not a "may" its a "must". The Win32 API doesnt support unlink directly.

I'd heard a rumor there was a way of opening a file that allowed it to be unlinked before close, hence the may.

The way you delete a file in Win32 is you use CreateFile with the DELETE_ON_CLOSE flag. When you close the file it is either marked as "Pending Delete" (if other processes also have the file open) or it is outright deleted (if no other processes have the file open). (See references below, this is better explained in the last posting on the last url in my reference list)

The Win32 C Libraries provide _unlink and _wunlink for compatibility purposes, but they just do what is described above.

Note however this applies ONLY to files opened with FILE_SHARE_DELETE. If a file is not opened with this sharing flag then nothing can delete or rename the file while it is open.

Anyway, as i said before the reason the Cygwin and the Win32 version behave differently is because Cygwin obviously opens files with FILE_SHARE_DELETE allowing the OS to mark the handle as "DELETE_ON_CLOSE" via unlink. Whereas Win32 Perl does NOT open files with the FILE_SHARE_DELETE flag set in normal circumstances (it does internally for such things as stat()).

The end result is that Cygwin has decided to go with the unixy behaviour of "you can delete open files"

To the extent possible, obviously.

and Win32 Perl has decided to go with the Win32 behaviour of "by default you cannot delete open files".

But the truth is that you CAN'T delete open files at all on Win32 systems, you can only ask the OS to delete them when the refcount on the handle falls to zero.

Im guessing that the bit you say about "until the process ends" implies that on Win95/98/ME cygwin somehow simulates the delete on close behaviour. Im not sure how as those versions of windows dont support FILE_SHARE_DELETE so if some process had the file open there is no way to delete the file until it is closed again.

It just uses an atexit-like handler (except it's invoked even on _exit) so if a different process had the file open, or the file ended up being deleted and recreated, it doesn't exactly work ideally; there's a comment in the code that does this:

Also, there is some further insanity in this picture: if two process have a file open with FILE_SHARE_DELETE is possible for one of them to delete the file, (and thus have it marked DELETE_PENDING) and then have the other one change the flag so it is not DELETE_PENDING so the file never actually gets deleted. The first process thinks the file has been deleted correctly, but the second overrides it. There are some other traps and zaps here too from what i have seen, so the default behaviour of not using FILE_SHARE_DELETE is maybe actually the least surprising approach.

Reminds me of the Win32/cygwin difference in handling perl -i with no suffix specified; cygwin makes a shot at allowing at unix-assuming code to run, while Win32 forces a change, but has no surprising edge cases.

The cygwin unlink code (for those who wish to delve) is:

extern "C" int unlink (const char *ourname) { int res = -1; DWORD devn;

path_conv win32_name (ourname, PC_SYM_NOFOLLOW);

if (win32_name.error) { set_errno (win32_name.error); goto done; }

if ((devn = win32_name.get_devn ()) == FH_PROC || devn == FH_REGISTRY || devn == FH_PROCESS) { set_errno (EROFS); goto done; }

syscall_printf ("_unlink (%s)", win32_name.get_win32 ());

if (!win32_name.exists ()) { syscall_printf ("unlinking a nonexistent file"); set_errno (ENOENT); goto done; } else if (win32_name.isdir ()) { syscall_printf ("unlinking a directory"); set_errno (EPERM); goto done; }

bool setattrs; if (!((DWORD) win32_name & (FILE_ATTRIBUTE_READONLY | FILE_ATTRIBUTE_SYSTEM))) setattrs = false; else { /* Allow us to delete even if read-only */ setattrs = SetFileAttributes (win32_name, (DWORD) win32_name & ~(FILE_ATTRIBUTE_READONLY | FILE_ATTRIBUTE_SYSTEM)); } /* Attempt to use "delete on close" semantics to handle removing a file which may be open.

CV 2004-09-17: Not if the file is on a remote share. If two processes have open handles on a file and one of them calls unlink, then it happens that the file is remove from the remote share even though the other process still has an open handle. This other process than gets Win32 error 59, ERROR_UNEXP_NET_ERR when trying to access the file.

For some reason, that does not happen when using DeleteFile, which nicely succeeds but still, the file is available for the other process. To reproduce, mount /tmp on a remote share and call

bash -c "cat << EOF"

Microsoft KB 837665 describes this problem as a bug in 2K3, but I have reproduced it on shares on Samba 2.2.8, Samba 3.0.2, NT4SP6, XP64SP1 and 2K3 and in all cases, DeleteFile works, "delete on close" does not. */ if (!win32_name.isremote () && wincap.has_delete_on_close ()) { HANDLE h; h = CreateFile (win32_name, 0, FILE_SHARE_READ, &sec_none_nih, OPEN_EXISTING, FILE_FLAG_DELETE_ON_CLOSE, 0); if (h != INVALID_HANDLE_VALUE) { if (wincap.has_hard_links () && setattrs) SetFileAttributes (win32_name, (DWORD) win32_name); BOOL res = CloseHandle (h); syscall_printf ("%d = CloseHandle (%p)", res, h); if (GetFileAttributes (win32_name) == INVALID_FILE_ATTRIBUTES || !win32_name.isremote ()) { syscall_printf ("CreateFile (FILE_FLAG_DELETE_ON_CLOSE)
succeeded"); goto ok; } else { syscall_printf ("CreateFile (FILE_FLAG_DELETE_ON_CLOSE) failed"); if (setattrs) SetFileAttributes (win32_name, (DWORD) win32_name &
~(FILE_ATTRIBUTE_READONLY | FILE_ATTRIBUTE_SYSTEM)); } } }

/* Try a delete with attributes reset */ if (DeleteFile (win32_name)) { syscall_printf ("DeleteFile after CreateFile/CloseHandle succeeded"); goto ok; }

DWORD lasterr; lasterr = GetLastError ();

SetFileAttributes (win32_name, (DWORD) win32_name);

/* Windows 9x seems to report ERROR_ACCESS_DENIED rather than sharing violation. So, set lasterr to ERROR_SHARING_VIOLATION in this case to simplify tests. */ if (wincap.access_denied_on_delete () && lasterr == ERROR_ACCESS_DENIED && !win32_name.isremote ()) lasterr = ERROR_SHARING_VIOLATION;

/* FILE_FLAGS_DELETE_ON_CLOSE was a bust. If this is a sharing violation, then queue the file for deletion when the process exits. Otherwise, punt. */ if (lasterr != ERROR_SHARING_VIOLATION) goto err;

syscall_printf ("couldn't delete file, err %d", lasterr);

/* Add file to the "to be deleted" queue. */ user_shared->delqueue.queue_file (win32_name);

/* Success condition. */ ok: res = 0; goto done;

/* Error condition. */ err: __seterrno (); res = -1;

done: syscall_printf ("%d = unlink (%s)", res, ourname); return res; }