The Apple iPod range has all-but decimated the wider MP3 player market, but there are still reasons why you might want to shun Apple and its admittedly-pretty-good Classic and touch devices. They won't play lossless files other than Apple's own ALAC format, for one, and you can often save a bit of cash by going for a Sony, Cowon or iRiver player.
Then there's the iTunes question. With an Apple device, you're pretty much tethered to Apple's music software, where most other MP3 players let you drag and drop files onto their internal memory. Some also have microSD slots, making it cheap and easy to expand the internal memory well beyond its previous limit.
If you're a sound quality nut, there are also much better sound engines out there than Apple's. Its players let you choose from a handful of EQ options as standard, but that doesn't compare to the subtle and powerful skills of the Cowon BBE engine. Then again, the Cowon J3 won't let you play Angry Birds.