I changed a stack size (from 50 to 20), reloaded the plugin, it is still allowing more than the allocated config value.

I have checked both the UI Editor and the config values directly and am not experiencing any problem with stacks not being updated.

Perhaps you have another plugin interfering with it?

Are you on the latest oxide update?

Oxide.Rust Version: 2.0.5383

Are there known issues with the version I am running?  

I can only support the latest builds of oxide. So please update