It very much depends on what hardware you have in the system. Just
about every expansion card or I/O device will reserve some of the
address space for its own use. Some devices will need a lot of space - a graphics card with 256MB of RAM on it will use (at least) 256MB of the address space for example.
This doesn't seem like a good idea that video memory is always mapped to system memory. What if one day graphics card gets 4GB RAM? Then we won't even be able to have 32-bit OS working with such card and in 64-bit OS 4GB of memory would be grossly wasted.
At one point, there was a considerable advantage to have video card memory fully mapped into untranslated address space so that various things could read or write as they pleased (cf "VESA linear framebuffer"); generally they gained speed advantages from this.
With AGP's GART, the amount of memory available for textures, bump-maps, etc, could reside in video card memory, local RAM, or a combination. Modern video cards do not have keep their entire memory space mapped into address space; for example, a nVidia 275 card with 1792 MB of RAM doesn't seem to want more than 256MB of address space under 32-bit Windows platforms.