The suggestion that novaswimmer had seems to me like an obvious one, and I found it surprising that you had not considered it yourself. To understand what it means when you say you may have actually tried it, and didn't find it relaxing is puzzling to me. Obviously there is more going on than what I (and maybe novaswimmer too) can easily grasp.
In what way is closed eyes with goggles on different from closed eyes without goggles? The only thing I can think of is that without goggles you have really committed to not looking, and this commitment is what has given you this "freedom", or maybe it's the totality of getting information from water sensation, i.e. not augmented by vision, and this richness of mental focus from the water sensation is liberating. Could this be the reason?