Yeah, I get that when skipping through parts of a track too. I think it's due to the fact that the rhythm in a music creates expectations in our brain, which is how we distinguish music from noise, so it feels weird and dissatisfying when the rhythm goes wrong. When unpausing music or playing not from the start, we don't start where the rhythm starts and it will feel weird for a few beats.