Being relaxed is definitely a factor. I've found that when I'm swimming a longer distance than I'm used to swimming, I tend to need to breathe more frequently, but then as I become comfortable with swimming that distance, I don't need to breathe as often.
I suspect one factor is that adopting a more frequent breathing pattern gives me something to think about and thereby distracts me from my nervousness about swimming the longer distance. And as I become more accustomed to swimming that distance, I eventually reach the point where I no longer need the distraction. :-)