I've developed  a similar web  part to show all the sites  that a user  has access
to in a farm. The performance  is pretty good because it uses a query against
the search index rather than trying to use the object model. The query
automatically security trims the results  so it only shows sites that you have
access too. By modifying the query you should be able to show only certain
types of sites or lists.

When you're coding against large sets of data, the search API is always the
way to go.

The search engine is the way to go. You may even be able to simply
customise the standard Search Core Results web  part to do what you need without
development.

See
sharepointsearch.com/.../sharepoint-searc\
h-techniques-property-based-search.aspx for information on the property filters
you can pass to the search engine to only return sites  or particular types of
lists, and
blogs.msdn.com/.../Searching-with-Property-Filte\
rs.aspx for a general discussion  of how you can use property filters in search.

Warning: the URLs above will get split in the e-mail, so you will need to cut
and paste them back together.

This one is linked from the previous link as well:
www.sharepoint911.com/.../i-love-the-sitedata-\
asmx-web-service.aspx

Your Very Own Permissions Dashboard
www.sharepoint911.com/.../your-very-own-permis\
sions-dashboard.aspx