Thats exactly the way it works

If a domain group  "hr" has access to a list  then all members of domain
group "HR" have access to that list. Those members may be user logons or
other groups  which contain user logons (or, indeed, further groups).

SharePoint groups  cannot be nested  inside SharePoint groups.

Along the same lines...
IF I have a WSS Site that is open to just:
domain_name\Finance_AD_Group,
can I create a WSS Site below that and have it be less restrictive,
say have it open to Contributions from both
domain_name\Finance_AD_Group & domain_name\HR_AD_Group?

Break the inheritance by modifying the security.

Add both AD groups  to a SharePoint group  that has contribute
permissions.

I guess if using AD for authentication, then nested  groups work, but if
using forms based authentication, it won't work?

That all depends on your third party auth system
I'm no expert on this yet, but I would assume that if your FBA membership
provider allows for nested  groups, it should work, as all SharePoint would do,
would use that membership provider and it takes care of validating group
membership.