So tonight I found out that players were seeing the buttons to https://umod.org/plugins/admin-panel So I pulled that.
And I had another player show me that he could use vanish a little while back. Not sure if he was using a "hack" or not. So I made the permissions per user just in case.
This is all on my build server. So it's entirely possible there's some player fckery afoot.