atom feed20 messages in com.selenic.mercurial-develRe: Exceeding the windows API MAX_PAT...
FromSent OnAttachments
Angel EzquerraApr 20, 2012 3:58 am 
Mads KiilerichApr 20, 2012 4:03 am 
Angel EzquerraApr 20, 2012 4:08 am 
Adrian BuehlmannApr 20, 2012 5:03 am 
Angel EzquerraApr 20, 2012 5:12 am 
Noel GrandinApr 20, 2012 5:14 am 
Adrian BuehlmannApr 20, 2012 5:42 am 
Noel GrandinApr 20, 2012 5:48 am 
Angel EzquerraApr 20, 2012 5:50 am 
Adrian BuehlmannApr 20, 2012 7:43 am 
Noel GrandinApr 20, 2012 7:46 am 
Adrian BuehlmannApr 20, 2012 8:11 am 
Laurens HolstApr 20, 2012 9:08 am 
Aaron CohenApr 20, 2012 9:33 am 
Adrian BuehlmannApr 20, 2012 9:39 am 
Matt MackallApr 20, 2012 11:32 am 
Angel EzquerraApr 20, 2012 1:05 pm 
Matt MackallApr 20, 2012 1:44 pm 
Angel EzquerraApr 20, 2012 2:19 pm 
Matt MackallApr 21, 2012 10:48 am 
Subject:Re: Exceeding the windows API MAX_PATH limit
From:Adrian Buehlmann (adr@cadifra.com)
Date:Apr 20, 2012 5:03:45 am
List:com.selenic.mercurial-devel

On 2012-04-20 12:59, Angel Ezquerra wrote:

Hi,

I've recently had some users that have shot themselves in the foot by add some files with really long paths to a mercurial repo. The file paths were longer than the windows API MAX_PATH, which is 260.

The problem occurred when importing some data from a clearcase repository into mercurial. The ClearCase repository contained some files with very long paths (more than 200 characters). However, because ClearCase forces you (in our setup) to place all your repositories on some folder right on your C:\ drive, the total length of those file paths did not exceed 260 characters.

A user created a mercurial repository right on the clearcase repository folder. Everything was added fine because in there all file paths were below MAX_PATH. He then cloned this repo into our shared server (into c:\mercurial\bad_repo). Again, all paths were below MAX_PATH.

The problem came when another user tried to clone this repo into its machine. This user places all his repositories in "My Documents\repositories" (which I think is common for windows users). The clone went fine until it tried to update to tip, which failed because mercurial could not create some files whose paths would exceed MAX_PATH. The user can clone the repo right into its C:\ root without any issues.

I think this problem is particularly tricky because the user making a commit cannot tell if it will fail in someone else's machine!

So far I've come up with 4 solutions, none of which is really satisfactory:

1. Rename the files: it may be feasible in this particular case but there may be cases where this is just not possible (e.g. for source code that you do not control) 2. Always clone into the c:\ root or at least have all your repos in "c:\repos" or a similar path: This may be fine (and good advice) in general, but may not be ok on multi-user machines and in some cases you may not be able to chose where to clone into... 3. Use the subst command to create a temporary drive and work in there: I've seen this suggested online, but I don't know if this will cause problems since the files will live both on the "virtual drive" and on the original deep folder. 4. Use folder junctions: same issues as 3?

Do you guys have any other solution?

BTW, I've looked for info on this MAX_PATH limitation and it seems that this 260 MAX_PATH limit is a limitation of the windows API, not of the underlying NTFS file system, which can support files of up to 32,767 characters (with a limit of 255 per individual 'path element'). Apparently the unicode version of the windows API _mostly_ supports long paths. It seems that you can access the unicode APIs by prepending your paths with "\\?\":

http://msdn.microsoft.com/en-us/library/aa365247%28VS.85%29.aspx#maxpath http://blogs.msdn.com/b/bclteam/archive/2007/02/13/long-paths-in-net-part-1-of-3-kim-hamilton.aspx

The second link is particularly interesting (specially part 2). Unfortunately a quick test shows that python's open function does not support this.

This kind of discussion pops up from time to time.

I think the most important aspect today is still this:

Even if you might be able to morph mercurial into using the long path api variants of Windows, other tools - which includes explorer on Windows 7 (!) [1] - won't be able to handle paths with lengths > 260 anyway.

So I'd recommend to make sure that *all* your other tools can handle long paths _first_ before proposing to change mercurial's working copy handling on Windows to use the long path api's.

As a side note: explorer on Windows 7 also can't handle file names with reserved names (e.g. the famous file named "aux"), even though you can create these by using the long path api's [2].

[1] https://bitbucket.org/abuehl/downloads/downloads/path-too-long-Windows-7-explorer.PNG [2] https://bitbucket.org/abuehl/downloads/downloads/Windows7-Explorer-delete-aux-1.PNG