Snap out of it - no one cares, really. There are all sizes and shapes at the pool, in speedos, squares and jammers. Get the jammers and jump in. The only problem, mostly with the guys, is over-wearing a suit, shelf-life of the suit long past due. I have actually had to go up to a few guys and say, "dude - time to get a new suit, you are scaring children and their parents."
In any case, get the jammers and get a new pair every month or two depending on how often you swim. I find the Arena suits the most comfortable and longest lasting. Make sure to give your jams a quick soap wash and rinse to get the chlorine out and spin in the dryer after each swim.