Found the problem the rights for the .ssh directory were wrong.
They need to be "700". Given that I used adduser to create that account I
think umask or something else for the root user may be wrong or perhaps
the adduser script.
The other point worth mentioning for anyone having a simmilar problem (ie
some IDs connect, but others don't) is a file /etc/login.access. Settings
in that file may allow some users to connect, but not others.
Obviously by the same token /etc/login.conf could also be an issue, but a
less common one (ie the user has some type of time restriction).