I believe the following article will do just that.

http://www.graphicalwonder.com/?p=666

The basic idea is that you add an extra querystring parameter called
"source" when you link to the form, and specify the redirect page.

I have already seen that article and it
makes no sense, the "Ok" button doesn't have an href perameter to add
a "source" to. See below.....

<td class="ms-toolbar" nowrap="nowrap">
<SharePoint:SaveButton runat="server" ControlMode="New"
id="savebutton2"/>
</td>

And adding a plain old HTML input button doesn't work either, that
article doesn't give near enough details on how-to get a redirect
working or am I missing something?

I haven't tried it so I can't say for sure, but it looks pretty
straightforward. When you link to the form, you simply add the source
parameter in the URL and the form  knows that when you're finished with it,
you should be taken to the page indicated in the querystring. For example,
the CKS User Group signup sample that Shane mentioned is just a link that
takes you to the following URL:

sharepoint.microsoft.com/.../NewForm.aspx?Sou\
rce=/sharepoint/cks/uge/Shared%20Resources/ThankYou.aspx

This would take you to the NewForm.aspx page and dictate that after you
submit it, you should be taken to the ThankYou.aspx page. Does that make it
more clear?