The Definition of a sport via Google is " an activity involving physical exertion and skill in which an individual or team competes against another or others for entertainment." So with that being said let me shed a little light on why swimming is not a sport...
There's hardly any physical exertion involved in swimming. Besides the fact you are suspended in a substance simulating Zero Gravity and have to use every muscle in your body to propel yourself forward. It's just not that hard.
Anybody can make the Olympics. Seriously, ask any olympian and they’ll tell you they woke up one morning said “Hey, I’d like to make an olympic team in swimming.” and then next day they did it. That's how it works.
Elite swimmers have incredible social lives, stay out late, and have plenty of time to play video games. So, no you really don’t need to give anything up at all!
Since there’s only more than 400,000 registered swimmers in the U.S....it's really not big enough to considered a sport. I mean after all 2 people make the United States Olympic in each individual race team every four years so your chances are pretty good.