Prevent domain users to logon

I am building a VC 5.5 including SSO.

Maybe not a huge issue but so far any domain user can logon using the web client. They don't get access to do anything however I would prefer to allow only the ones who needs to access to the virtual center. Are there a possibility to prevent domain users and only allow the ones who have roles assigned access to the VC?

Thank you