If you have a project that only exposes the admin you should just use the 500/404 templates from the admin.
Put this in your project's urls.py:
from django.utils.functional import curry
from django.views.defaults import server_error, page_not_found
handler500 = curry(server_error, template_name='admin/500.html')
handler404 = curry(page_not_found, template_name='admin/404.html')
I wonder why django doesn't mention those templates in the docs ...
If you have other drop-in apps that need authentication (like rosetta or sentry) bare in mind that the admin doesn't have a reusable login view so you must hook one. You should just reuse django admin's login template. Put this in the urlpatterns (don't forget to match it to LOGIN_URL in the settings):
url(r'^accounts/login/$', 'django.contrib.auth.views.login', {'template_name': 'admin/login.html'}),
You might note that this is not very DRY but actually the LOGIN_URL might differ than the one in the urlpatterns (eg: you mount the django wsgi handler on a non-root path).