do one thing , click on edit page where your survey is being displayed on, then
click on modify shared webpart and from there select the survey web part you
made and on the properties pane which it opens up in side on the end of the
properties pane page there is an option to select audience so do that , and then
the survey will only be available to the audience you select