I wear goggles and I had to test a lot to find some who are tight on my face.
I never tested a mask. I thought of, but then I found the goggles which I wear now.
I always wear a swim cap, even in the pool. It's only to be seen, I flip over the edges because I don't like it when the edges of the cap reach over my ears. So the lower strap is under the fliped over edge of the cap.
I never had problems of loosing the goggles not even when diving.