I'll chime in since it's been a while since I've done so. Being 'follicle challenged' I can attest to the 'need' for head coverage while swimming in cold waters. Not doing so leaves me with the a sort of 'brain freeze' and a bit light headed afterwards. I'm sure however that proper acclimation would alleviate most of that until a certain temp. Wearing a cap at least gives me a 'comforting' feeling that I am warmer.
No fat insulation? perhaps but 'thick skulled' people need not worry : )
Although the temps here off of Long Island haven't yet dipped to those low temps as Terry et al mention, time in becomes important as I recently found in swimming my longest distance to date, a 10K. Water temp was 65F, not a prob at shorter distance, but add an extra hour plus in and it makes all the difference so being prepped can be critical. I wore two standard latex caps and felt just fine afterwards, light shivering, no need for plugs with two caps but couldn't hurt especially if temps were lower (my partner wore a wetsuit and some sort of thermal shirt under that and still had the shivers). I'm sure continual swimming as the temps decrease will enable me to swim those distances in colder water. Naj mentions all the post swim necessities for comfortably keeping the core warm and the mind wanting more!
Naj, all I can say is one day, one day ...