Sometimes when I need to toggle my brightness on of off then the keys don't work. When I do this then it will only change the setting sometimes, other times it will not do anything. This is happenign with the b key, and the bracket keys.

I am running Sodium and a mod named LambDynamicLights.

Just put out a new update that may have resolved this issue. Drop a comment if the issue persists.

Might be just the thing with the latest release that if you toggle the brightness to maximum that the keys will change the normal brightness you had before instead of the maximum brightness currently in use, which is really confusing and unintuitive to be honest. I talked about that with the owner and made some changes and bug fixes myself. He pulled those changes and is currently still working on that I think, so you would have to wait for the release.
Im pretty sure its just that and not a real issue, because I don see a reason why the keybinds shouldnt work, its just that you dont see that as described above, so now you know, if you havent figured that out yourself by now.
If thats actually not it and its a real issue it would be great if you could specify "sometimes" a bit more.