Cosmic-ray interactions with interstellar gas and low-energy photons make the Milky Way galaxy a gamma-ray source that covers the whole sky. The low instrumental background, large effective area, and long mission life of the Energetic Gamma-Ray Experiment Telescope (EGRET) on the Compton Gamma Ray Observatory provided the first opportunity for comprehensive study of this diffuse gamma-ray emission. Here I discuss what has been learned from EGRET about the cosmic rays and interstellar gas in the Milky Way, including the apparent excess of GeV emission from the interstellar medium, the potential Galactic halo at GeV energies, and the implications of EGRET observations for the nature of dark matter in the Milky Way. For many gamma-ray astronomers, the emission of the Milky Way is primarily a background to be removed in the study of point sources. I also discuss the potential for diffuse emission to confuse or contaminate the EGRET source catalogs, and conclude with the prospects for advances with the Gamma-ray Large Area Space Telescope (GLAST).