I believe that to be working as designed. You are altering the permissions
on the list item. Think of it this way.
When you manage users  you are authorizing the user for the Sharepoint
application. When you modify access  for a specific application and remove
site groups etc you are now defining specific authorization to resources so
the fact that I can get to the page was because you allowed me to log on to
Sharepoint. The fact that I can't participate in a survey  is because you
took away my privileges.

Anyone wish to comment? PS: we wrote a web part to do a custom survey it
works but when we tried to programmatically alter permissions to make it a
read only survey after a predefined date we ran into all sorts of issues,
anyone do anything similar? If so send me an eMAIL please!