Yea generally you're right.
If this is not your problem well, great. It seemed to me like there was the possibility that it may be worn and the diff then tends to fall down a smidge and puts more of an angle and pressure on that rear tailshaft joint.

Then, when loaded (which in my mind pushes the diff back up a smidge) as you say the vibration disappeared. Anyway, I'm glad it's not that balljoint.

I did one recently that didn't clunk like you would expect but rattle like a loose shockie when you went over bumps. Go figure