From a public web site site collection, we are trying to access user
profiles in a separate My Sites site collection. When accessing them
anonymously, running the code in an RunWithElevatedPrivileges block I'm
getting an authentication error. When I login to the public site (I'm
an admin) it returns just fine.

A code excerpt is below:

protected override void CreateChildControls()
{
//Get the current Site
//Get current Level
SPContext spContext = SPContext.Current;

SPSecurity.RunWithElevatedPrivileges(delegate()
{
using (SPSite currentSite = new
SPSite(spContext.Site.ID))
{
using (SPWeb currentWeb =
currentSite.OpenWeb(spContext.Web.ID))
{
SPWeb rootweb = currentSite.RootWeb;
ServerContext context =
ServerContext.GetContext(currentSite);
UserProfileManager profileManager = new
UserProfileManager(context);

...

// this line errors with
System.UnauthorizedAccessException:
// The user could not be authenticated to
the Web site being accessed
profile =
profileManager.GetUserProfile(user.LoginName);