Ronnier, chlorine is a poisonous chemical. You cannot develop an allergy to it. I would guess that your pool is probably over chlorinated and you are sensitive to it...like I am, in local council pool!
There may be underlying issues here, and maybe you should be discussing it with a doctor rather than an allergist? If it is any consolation, you do learn to tolerate it with more exposure. Not sure if that is a good thing though! I see a guy regularly swimming in chlorinated pool with no goggles on, with eyes open and it does not affect him. My eyes would be red for a week. :-)
Also, if you are drilling, and swimming slowly, you can ingest a bit of water, and have some up your nose, that can irritate delicate lining.