atom feed1 message in org.tigris.scons.usersVisual C++ incremental linking vs emb...
FromSent OnAttachments
Dave VitekAug 29, 2008 11:05 am 
Subject:Visual C++ incremental linking vs embedded manifests and build caching
From:Dave Vitek (dvi@grammatech.com)
Date:Aug 29, 2008 11:05:46 am
List:org.tigris.scons.users

Hi all,

I have some colleagues who have executables that take ~ 20 minutes to link, most likely due to extensive use of C++ templating. My colleagues also like to have their manifests embedded in their executables. If one embeds manifests using mt.exe, then this prevents incremental linking because the linker refuses to work off an image that has been tampered with by mt.exe (insert snide comments about MS here). However, there is a way of embedding manifests using link.exe that does work.

SCons issue 1465 <http://scons.tigris.org/issues/show_bug.cgi?id=1465> might be the most relevant issue. I thought I would point this out on the off chance it could be worked into the current plan.

This is an English explanation of what the VC IDE does: http://blogs.msdn.com/nikolad/archive/2005/06/05/425360.aspx

and here's a Makefile that implements the VC IDE's strategy. Apparently, the Makefile grows to about 5 times its original size in order to do this: http://blogs.msdn.com/nikolad/articles/425359.aspx

It isn't too bad, although it does possibly run the linker twice (in practice, this would usually only happen on the first clean build, and the second link is always incremental). I imagine the main issues are: 1) It would confuse users when they see the linker run twice 2) SCons should probably be aware of whether or not it is doing incremental linking and use mt.exe if it isn't.

-----------

There is a somewhat more daunting problem if you want to use build caching and incremental linking together. Build caching doesn't play nicely with pdb files so we use the legacy debug info mode in order to get symbols embedded in object files. However, incremental linking doesn't work with the embedded debugging symbols -- it needs it in one or more pdb files.

The only idea I have about this is to make the compiler tool output both the object file and a new (unique) sibling pdb file so that the pdbs get cached. The linker would also need to start considering all the pdb files as inputs, which it uses to produce a combined pdb file. IIRC the linker will look for pdbs that are siblings to obj files before going to an absolute path embedded in the obj for the pdb.

Is the builtin SCons compiler/linker relationship extensible enough to support this? Is there an expectation that compiler tools produce a singleton output? Another potential (unresearched) solution is to use Microsoft Symbol Server. Any other ideas?