Tomoy, I don't think there can be a consistent 'rule' for breathing in swimming or any other sport. It depends on your body's current needs. In ultra-distance running, which was my main game the idea was to always run at a conversational pace, ie a pace at which you could run, breath and hold a conversation without gasping. Clearly when running shorter distances within 5 beats of max. heart rate you are not going to be doing much talking but breathing deep and hard and you will probably only keep that up for 3 - 5 miles - if you are fit. I have transferred the same idea to my swimming with TI. I swim a mile or more almost every day and I take shallow breaths - no more breath than I need, unless someone begins to come up on my shoulder when I have a Pavlovian response and accelerate, which demands more air.
Where I have found it helpful to breath right out is on open-turns, during the underwater glide phase. This seems to help by keeping you down a bit longer at a reasonable speed and gives you a chance to clear the accumulated CO2 from the lungs; enabling a good intake of oxygen-rich air with the first couple of deep but not gasping breaths when resuming stroking.
Sorry about the length of this but I have the very bad habit of never using one word when nine will do . . .