Damn, but that was a long-winded process. I’ve just spent the better part of 4 hours reading and researching why I get (got, its fixed now)  404 errors from wp-admin when I enabled .htaccess Authentication (htpasswd) for the wp-admin directory. It only happened with WordPress MU / Network config.

googling for this error ends up with a mountain of irrelevant threads, inconclusive discussion, and unanswered questions.

WordPress single user 3.1 is not affected by this as there are no ReWrite rules required for the wp-admin. The issue only occurs with MU or Network as it is now called in WP 3.0+ as there is a set of ReWrite rules in the .htaccess file that look like this:

RewriteCond %{REQUEST_FILENAME} -f [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^ – [L]
RewriteRule ^[_0-9a-zA-Z-]+/(wp-(content|admin|includes).*) $1 [L]
RewriteRule ^[_0-9a-zA-Z-]+/(.*.php)$ $1 [L]
RewriteRule . index.php [L]

The issue is that the prompt for the username and password that should happen automagically cannot occur due to the rewrite rules.

The rewrite rules are to look for a valid directory or file and if it is not a valid directory or file then redirect to the index.php which is what happens as the htpasswd AuthConfig process is not a valid file or directory (apparently – umm, so what is it? like is there another rewrite rule that would avoid this? obscure Apache voodoo)

The fix is to create a valid file as an ErrorDocument directive for a 401 or 403 error (Authorisation errors) and have this at the top of the .htaccess file so that Apache will return a valid file flag and therefore allow the AuthConfig prompt to occur.

Edit your top level .htaccess above the section for #Rewrite for WordPress

ErrorDocument 401 /myerror.html
ErrorDocument 403 /myerror.html

if you dont want to put this dummy file in your site root then add the path to the filename

ErrorDocument 401 /[path_to_file]/myerror.html
ErrorDocument 403 /[path_to_file]/myerror.html

Then just create an empty file with that name in the appropriate path.

The references that I used were:
http://www.scratch99.com/2008/10/password-protecting-the-wp-admin-folder/
Which is where I finally found a decent reference that was related to the issue and that blog referenced this one,

http://developedtraffic.com/2007/05/27/wordpress-admin-password-protection-404/
Which pointed me at TextPattern http://textpattern.com/faq/173/password-protected-directories-with-htaccess

and for good measure, this site helped to complete my understanding of why this process works.
http://www.ju-ju.com/2006/03/17/wordpress-404-error

Thanks to the authors of each of those sites.