This is a great topic. I'm fortunate that the rec pool that I use in San Francisco is A.) right around the corner from my house and B.) is wide enough that the three lanes are very wide.
Still, this is no excuse not to be polite. I ususally get in the slow lane to practice a slow steady glide in whole stroke or drills. I practice at 6am too so most folks are still sleeping and it is not as crowded. However, when I swim in the slow lane I have a mix of more elites who warm up there and true novices and folks who obvioulsy taught themselves to swim.
Whenever I get in I usually let those in front of me get at least a 15mm head start and then I push off. If I need a longer break and someone seems to be waiting behind me I always let them know that they are more than welcomed to go ahead of me. And like NY said, when I do take a break I move to the far right (in the pool only though not politics!)