I definitely experience this "phenomenon". In fact, practically every time I swim, the water feels different, depending on so many factors : the water temperature, the number of other people in the lane, the number of other people in the entire pool, the percentage of those other people who are children oblivious to anyone else around them, the time of day I'm swimming, the angle of the sunlight shining down on the lanes, which Speedos I'm wearing..... you name it. Some days, I swear, someone has poured jello into the water and let it partially set. So, yeah, I know exactly what you're talking about. You are not imagining it.
I think not only is the water temperature a big factor, but also the amount or concentration of chemicals in the water. Some days the water feels cool and clean and clear, and I can zip through my swim; other days it feels too warm, and I can almost taste the chemicals in my mouth, and then I feel like I'm slogging through slime. I am always amazed by how differently I might swim from one day to the next, and it's certainly not my stroke that's changing so dramatically.
I regularly swim in four very different pools - a 25-metre, 7-lane indoor pool with a graduated floor; a 25-metre, 5-lane indoor pool with a non-graduated floor (constant depth of about 4 feet); a 33-metre, 6-lane indoor pool with a graduated floor; and a 33-yard (from before New Zealand went metric), 5-lane outdoor pool with a graduated floor. I swim completely differently in each pool, and like I just said, it ain't my stroke that's changing! So it's got to be the environment in which I swim that is the cause.