Sure, I had that feeling for most of my adolescence and young adulthood. It still pops up here n there. If to know you more is to love you less, then that person really didn't love you. Instead, they loved an idea of you or even just the idea of someone. So I agree with the other part of that scene, "you should never settle for someone who just loves the idea of you."