The majority of the time I listen to business and educational podcasts while I run, some days I run in the silence of the outdoors.
Whatever you decide, always be sure you can stay in touch with the world around you. You should be able to hear the cars coming behind you or in front of you by at least a half-mile in fast-moving traffic.
For space savings and weather resilience, I use an iPod Shuffle with over the earbuds. The Shuffle is a simplistic audio player with no moving parts beyond the power, selection, and volume buttons. It has stayed in my pocket in some very wet and cold conditions without any damage. I doubt Apple would say it is waterproof, but it has dealt well so far.
If running in windy conditions pay attention to how much impact the wind noise is having on your audio clarity. Similarly when running in snow or ice where each step can increase the noise level.