All things being equal, font crawl should be more efficient and less strenuous than breaststroke but all things aren't always equal.
I spent most of my swimming life (from 1947 to 2006) only able to swim breaststroke with any degree of ease. In 2006 I discovered Total Immersion and since then I can swim all four competitive strokes ( butterfly only after a fashion but I can swim it for 50m or so). I have swum several 1500m races, very slowly even for a man of my age, but nevertheless I have finished and lived to tell the tale.
A few years ago I could only have swum them breaststroke and my times would probably be even slower.
In my case it was mainly the flutter kick that was the obstacle and the first progress I made was when I read somewhere that the kick created more drag than propulsion for many unskilled swimmers and I began to swim freestyle without a kick.
Breathing has never been a real problem for me but there is still plenty of work to be done on it. I still find it difficult to breathe on the 'wrong' side.
I think that your fitness must be OK if you can easily swim a mile with breaststroke, therefore I conclude that your difficulty with freestyle is mainly to do with breathing and not fitness.
I suggest you do plenty of 25m and 50m repeats concentrating on easy breathing , making sure you're exhaling while your face is in the water and not holding your breath at all. Then move to 75m repeats and 100m repeats and soon it will be easy.