Internet Archive has thousands of digital copies of old 78 RPM vinyl records available for your listening pleasure.
They also have an extensive collection of presidential and World War II recordings. With a little research, you can find audio files to supplement almost any subject area.
Most files are available to download so you can listen to them while jogging or away from an Internet connection. I also enjoyed hearing all the little scratches and audio imperfections inherent in a vinyl recording.