It’s true that your body adapts to the specific need and it’s true that specific skills do not transfer between activities.
But, neurological adaptions and muscle do.
We don’t know. Science actually knows less about this then you think.
But, if you train something extremely similar to a sport at hand, but with weights, like a weighted baseball. Those athletes, like a Baseball player get worse.
Just like boxers who practice punching while holding dumbbells.
They both end up training their bodies to move slower.
What does transfer is strength.
Check out Eddie Halls video below. Where it shows the 2017 world strongman break a CrossFit world record, all because the weight in question is so puny to him that it becomes almost pure cardio.