So on my server I only allow removal of up to stone, but that also counts for admins. Is there any way to bypass that?

Use admin mode
/remove <admin | a> [time (seconds)] -- Enable admin RemoverTool. In this mode, any entity can be removed.

ahhh, okay, I am dumb, thanks!