A user's access to a folder will be controlled by the lowest level of access that they have, whether for a group or their user record.
Therefore, if a user appears in the folder's permissions area as themselves or in a group with limited access, any other occurrences of the user will default to those settings.
E.G.
User N is in Groups A and B.
Group A has full access to the folder but group B cannot Add documents.
Therefore, user N will not be able to Add documents based on the permission set for group B.
It is important to note that if you change the permissions of the Public Group, all users will default to those permissions. Therefore, the Public Group should be removed from the folder permissions area entirely rather than the permissions being changed - otherwise all users might lose access to the folder as as everybody is in the Public Group (and cannot be removed).
Additionally, if a user is denied View access within the folder permissions (whether by group or individual setting) it then it will not be possible for them to interact with documents in that folder at all - because you need to be able to see the folder to be able to Add, Delete documents etc.