Your problem is that you are breathing much too late at the wrong point in the stroke. I'll wager that during your breathing, you are probably pulling your head up (clawing) for breath! - What you describe is NOT a balanced position.
There are two things that you need to correct this:
1.) Good balance
2.) Breath on the EXTENSION to air - NOT IN THE PULL PHASE!
(1) First check your overall balance in the water:
Important: I really suggest you do the following using a snorkel or over a short distance (Say 20m) during which you can hold your breath. Forget breathing until you can move effortlessly through the water - WITH NO KICKING!
Press down on your buoy, let go of your head and keep it inline with your spine. Use long rhythmic strokes keeping your arms extended (stroke should be almost a catch up in the front quadrant) with your core muscles driving you forward. Your hips should be riding the surface. When you have good balance you should be able to swim effortlessly using the rhythm of your core without kicking - Your legs will be close to, but beneath the surface. On each stroke try to reach with your lead hand toward the end of the pool - without introducing tension in your arm.
Now for the next bit:
2) You turn your head to air and breath as your lead hand extends:
Remember the extended stroke from above? now try to follow your extended arm to your breath. If you are properly balance you will feel like the water is supporting you. During the breath, you must NOT raise your head, simply turn it to the side as your lead hand extends to the catch. Your catch hand is not yet in the pull phase. After your breath then the pull starts!
This video should help. Just watch what Hackett does (there are some good examples of what happens when you have poor balance)! Notice how balanced Hackett is