Something like this looks sophisticated without being too brutal and has stability proofs: https://arc.aiaa.org/doi/10.2514/3.19988

We should also probably do something to try to diagonalize the inertia tensor (although with a non-symmetrical space station or something this may not be possible, but there must be algorithms to extract the leading-order diagonalized term from a matrix? And similarly with the control torques?

More or less did this enough. I should probably revist the quaternion PID or an LQR/LQI controller at some point, but life is finite, so I'm declaring this done .

https://www.tandfonline.com/doi/pdf/10.1080/00207176808905679 seems to be the basis of the literature.

https://ieeexplore.ieee.org/abstract/document/467669 is heavily cited and the abstract mentions "robust to modeling errors" which sounds very attractive.

https://arc.aiaa.org/doi/abs/10.2514/3.20418 actually seems reasonably simple and is robust to uncertainties in the inertia tensor, supports craft asymmetry and has gain tuning. In the presence of uncertainties in the inertia tensor the movement is not about the eigenaxis and is therefore not optimal, which seems utterly fine for KSP uses.

LQR design that results in a optimally tuned PD controller:

https://ascelibrary.org/doi/10.1061/%28ASCE%29AS.1943-5525.0000142

Also dropping a note for myself to investigate Extended Kalman Filters.

Linear Quadratic Regulators might also be useful although it looks like they may require spacecraft with small off-diagonal components of the inertia tensor.

Some interesting review articles:

https://www.sciencedirect.com/science/article/pii/S1367578812000387

http://ancs.eng.buffalo.edu/pdf/ancs_papers/2007/att_survey07.pdf <----- this is HIGHLY cited

https://www.mathworks.com/matlabcentral/fileexchange/65173-absolute-orientation-with-the-quest-algorithm

Unfortunately it looks like "determination" is separate from "control" and I went down a rathole with the QUEST algorithm there on "determination" which is not super relevant to KSP.

This is probably the thing to try implementing first and see if it works any better than the PID controllers everyone has designed so far:

https://www.tandfonline.com/doi/abs/10.1080/00207176808905679 (which I already posted... so I've done a full circle... should try to actually write some code now...)

Model Reference Adaptive Control that is still in the euler angle space: https://ieeexplore.ieee.org/document/4049028