I have been using a tempo trainer for 4 years now. Absolutely no unreliability issues. I have had 2. One was without a replaceable battery. I stopped using it because I was afraid it would die just when I needed it. (Now thinking about it, it would be no worse than what I'm doing now with the more advanced one. I still would need to go home that day and replace the battery before it would work again). But the battery seems to be lasting for ever.
I have broken the clip that holds the TT to my goggles -- maybe I was too rough clipping and unclipping (to change the timing values), but my swim store had a whole bag full of replacement clips for me to buy from.
It works, it is reliable, and it does the trick for less cost than an MP3 player, and it is specially designed for the task, so it works easily in use.
If you bought one with a replaceable battery and it was dead when you bought it, it's annoying, but at worst, the fix is the cost of a coin cell.