So then you only identify users by a cert pair (public/private) and a fingerprint?
Do you see no value in having a consistent durable username across hosts?
Couldn’t you give a user a centrally signed cert that includes their username? No online verification is needed, only the math to confirm the CA signature.