Is there any interaction between WSS's task lists and Outlook's tasks? I cannot find anything that suggests there is. And if not, I presume there is no concept of reminders on SharePoint's tasks?
Nope, no interaction at all. You can create alerts on tasks just like any other list though.
I believe (pure speculation) that Project Server has tight integrationwith the Tasks list and hence Outlook was unable to alter it for tightintegration.