# All robots will spider the domain User-agent: * # Disallow directory /services/ Disallow: /services/ # Disallow directory /inc/ Disallow: /inc/ Disallow: /database/ Disallow: /emailText/ Allow: /