How do you change the port on an existing SharePoint site?

For instance, I have a site that is on port 81. But I want to change
it to 8081 or something like that (pick your port number!) :-)

If I change it in IIS, I just get the default website, like the site
isn't registered anymore.

How can I accomplish changing a port number on an existing website?

The only way I can think of off hand is to create a new sharepoint
website, do a backup of the old site, and restore, but I'm sure there
has to be an alternative that is easier then this?