Although the universe is mainly empty space, there is the occasional hydrogen atom scattered out there. Now sure, that's not bad, cause on Earth you crash into Oxygen and Nitrogen particles every time you move.
But then you're not moving at 0.9999c or whatever. Raindrops are enough to damage missiles travelling at Mach 11 (around 3740m/s). So, travelling over 80,000 times faster than that, a hydrogen particle may just be enough to cause serious damage to an object.
But, by the time we've invented an engine that goes that fast, we'll probably have invented force fields and the like.