RE: robots.txt?
if you dont have a robots.txt it doesn't mean you wont get spidered. If your site is linked to and/or you submit it to search engine it will get spidered.
Robots.txt just tells the spiders which parts to not index, such as private areas etc
|