Does this guy think this man is out of my league?

Don't worry, in most cases people are just trying to get their own backs. So he probably was just trying to think of something to say that would make him look good or feel comfortable. Not thinking about you really at all. So it's not personal.