Why not just remove the user  from the security  group?

We can manage sharepoint  server only, all the security  groups will be managed by other group.

Are there any way through sharepoint object  model for restricting the security group  user ?

Yes, you can name each user  individually, but I suspect you do not want to take on that maintenance task ...

The OTB security  model is through users  (individually) or through security groups, anything else will require different Web parts for access, assuming you are not able to do another type of authentication.

You need an auditable process for adding/removing users  from the AD
security groups. When a site  is put into production, the site owner
(there should always be a primary and backup person(s) responsible
for the site) should be able to contact the security  group via the
help desk to have people added/removed from the group. No one else
should have this authority (not ability) to put the request in to
have people added/removed.
Using AD groups also makes it easy when people leave since their
accounts should be deleted and their access  is then automatically
removed.
I can explain the process further if you're interested.

The easiest way to manage security  gruops is through active directory. If you manage sites by individual users  you are going to have a maintenance nightmare. If someone switches teams you will have to remove them from these sites and add  them to those. If you have AD groups you can do it all in the back end by clicking on the users profile and seeing member of tab. The way I have seen this implemented in most companies is the help desk grants the sharepoint  team access  to create AD groups in a specific  OU. They can't change anything else other than create AD groups and add and remove users from them. Once they are created and add to a site  then all changes after deployment are routed through the help desk. It's a great model  and the best way to implement I have seen.

Well, it’s A way to implement

Just remember, if you do this then you don’t get presence for team members directly on the site  (which can be important to some people)

We have just found this was the best way to manage the permissions of 10 portals, 6500 - 8000 users. What was happening is one of the sharepoint  team members was spending 100% of there time trying to manage permissions. After we switched to the AD Group model  it reduced the load 10 fold.

So you get the users  disolayed in the members qeb part and can. See if they are
on line in the web part ?

No. I should have been clearer in my post.
When using groups:
On SPS, the presence information is available. On team sites, it's not.
On both SPS and WSS, the Members web part does not display individual
users; only the group  they are in. Sorry for the confusion.

I find the presence really useful when I am working in a team

So, we use groups for sps as it let's people into the portal and audiences

We use groups for the public facing wss  sites

Where anyone works as a teazm we tend to add  them indivudually with group  access
for read only type people