Thanks for the screenshots. It shows me something but not the why. I firmly believe that some results can be caused perhaps by several influences. Just guessing is not my MO. (If I would guess then: your Height Scale is too large, I would wonder if that fixes it)
Please share a project folder with related textures for those kinds of questions. (Reduced to the problem, zipped and please via Dropbox, [Google, Apple, Adobe, Wetransfer is OK too, anything else hasn't been cleared [security].)
The improvements around version 3.5.11 or 3.5.12 for Normal mapping can be ignored and set to Legacy. Here is also to mention the Keller Algorithmus; see link. If Legacy works better, then it seems you found your setup.
I will try to get more information into the manual about when and how to use it.
All the best