Haven’t experienced this with the Jeep but my 03 Chevy 2500 has the NP261 t-case ( just a heavier duty version of the 231)and was doing the same,
there is one little set of synchronizers in it that allow the shift on the fly, and when I replaced the chain, I also replaced the sync. and it went away