Usually, it's the document management policy of the company that determines how to keep documents  in a collaboration site. SharePoint document libraries  are not just a document store. They have advanced functionalities and improved security, one should use them. Check-in, check-out, versioning, approval work-flow, auditing, list, folder and item-level security, just to name few.

As an example, the client I work for uses two types of libraries in his collaboration sites. Public document libraries open to every one and private libraries with restricted access. The security groups (owners, members, etc.) to which the user belong determine his access level.

Couple of best practices here:

Use multiple libraries  rather than a single library  with a complex foldering structure
Use views to categorize information rather than folders where possible
Establish a set of content types that will be used across the organization... even if it's not a full list content types have tremendous value for searching, document retention, workflows and custom metadata capture
Configure appropriate views so that it is easier for users to find, sort and filter information (Adding "Checked Out To" to all views is very helpful)
Configure required and optional metadata where appropriate (valueable for searching and document retention)
Set up different web part views of the documents  on the Team site  homepage and utilize the Relevant Documents web part
Overall though our clients typically store the entire document within the SharePoint team  site and perform all collaboration, document management there. If you are coming from a file system scenario then it is usually a good starting point to review your folder structure on the file system and use that as a template when creating document libraries and content types.