Breathing can be a challenge... so many factors psychological and physical.
I went through a phase where I suddenly couldn't breath despite not having any issues in the prior months as I was learning... still not totally sure what caused that, though I do have theories.
Sounds like you are exhaling effectively, which is the key action you can control while you are going through most of your stroke. As you suspect, when you exhale fully, inhaling comes naturally.
For me, it is critical that the breathing be fully integrated into the whole body role. I find that I start exhaling immediately after I take the breath but exhale with extra force as I lean into my catch.
Also, I don't know many people who can sustain consistent alternate side breathing for a long period. In the pool I switch sides every 3-4 strokes while in open water it depends much more on water conditions as well as how nervous I am. More nervous=shallower breathing=more time on each side.
I would try the following.
1) Do same side breathing for a couple of breaths on each side before switching
2) Start your swim trying to make it completely effortless no matter how slow you are going. Literally try to keep it the swim so light that you don't feel like you are exercising at all, and then slowly warm up.
3) Related to to 2, S-l-o-w d-o-w-n Try doing your stroke in super slow motion. This helps with overall mindfulness in every aspect of your stroke, but especially to be aware of breathing.
Have faith that it will come together. It is a wall that you will eventually fall through. (I don't thing pushing will do that.) Persistence, and awareness will get you there.