It depends on which clip of Shinji you refer too.
Of course there are those multi million hits 9 strokes or so clips. He would almost go backward in rough conditions swimming in this manner.
But there are other clips, which are a bit more rare as they didn't receive that many hits where he swims closer to 1.0 (60rpm), and at that rate, though it's dangerously low for rough conditions, you can move forward.
Conditions don't need to be *that* rough to slow you down, sometimes small waves, when they hit you from the front are difficult to deal with. This video is a good example: