Access rights table design

I am developing a hospital system where patients can allow user access to their information as per their choice. There are different group of users for e.g doctors, nurses, registration clerks, insurers and managers. A user can access either patient personal details, patient medical details, patient medical details or all. Each user is identified by a unique "user_id" and each group by a unique "group_id".

A patient can block:

1. a single user (e.g a nurse)

2. a group (e.g all nurses)

3. a group with some exceptions (e.g block all nurses except nurse1 and nurse3)

4. specific users but allow their respective groups (e.g allow all nurses except nurse1 and nurse3)

Once access is granted to a user or group they will be assigned permissions like read, write and delete.

I'm having a lot of trouble figuring out how to design my table(s) along with their fields so that I can store these access rights information.