Reviewing the data and pages you expose for external access ( Community users and Site Guest User) is an essential and critical piece to maintaining a healthy Salesforce instance. Here are a few suggestions that might help you get started.
Setup -> Security -> Health Check
Start with the Portal Health Check
Setup -> Security -> Health Check
Review the External Object Access
While under the health check window. Scroll down to locate the number of objects that are exposed to external users.
This particular screen denotes that 85 objects have a default sharing model of either Public Read/Write or Public Read Only. This means that if the Community profile has 'Read', 'Edit' Access to the object they get to read / edit all records of that object.
Keep in mind that any newly created custom object has a OWD of 'Public Read/Write' by default. As part of the deployment always ensure the OWD is always set appropriately.
Fixing External Object Access
Setup -> Security -> Sharing Settings
Enable 'External Sharing Model' if not enabled already
Modify the External Sharing and set it preferably to 'Private'. It is always best to keep the OWD as 'Private' for external access. Create object specific sharing rules, or set 'Read All' / 'Edit All' on the profile level if you need to share with External users.
What about publicly available pages? or Data available for not-logged-in users.
Community pages that are marked as 'Public' are accessible without a login. The default is login required. Find out the pages in your community that are marked as publicly available as below.
What data is exposed to a not logged-in user?
Data access for publicly visible pages is controlled using the Site Guest User profile. Access this profile in the following ways
Setup -> Sites -> Site corresponding with Community -> Public Access Settings
Setup -> All Communities -> Your Community -> Workspace -> Administration -> Pages -> go to Force.com -> Public Access Settings
Control object and data level access using this profile.
Note: You could click on 'Assigned Users' to set the timezone, locale and other settings for the default user.
