In MOSS 2007, When I click on "Manage Settings for Single Sign On" and
then "Manage Server Settings" I get the "Manage Server Settings for
Single Sign On" page. If I fill it out, I get an error with a red "X"
that says "YOu do not have the rights to perform this operation."

I'm a domain admin, an admin on the local box, I have sa access to
the database server, I'm in the Server Farm Administrators group and I
have full control of the central admin site collection.

What else do need to set the SSO server settings?