robots.txt

URL:

http://www.ucs.mun.ca/robots.txt 🔗

Replay URL:

http://webarchive.bac-lac.gc.ca:8080/wayback/20061114040018/http://www.ucs.mun.ca/robots.txt 🔗

Resource Name:

robots.txt

Host:

ucs.mun.ca

Collection:

NGOs

Collection Id:

4666

Crawl Date:

2014-09-23T00:32:47Z

Source File:

ARCHIVEIT-4666-NONE-3595-20140922235312197-00000-wbgrp-crawl058.us.archive.org-6441.warc.gz

General Content Type:

text

Content Language:

en

Length:

622

Content:

User-agent: * Disallow: /cgi-bin Disallow: /old Disallow: /OLD Disallow: /test Disallow: /test3 Disallow: /RCS Disallow: /webinator # was /mun, which caused /mun* directories to be excluded from webinator # changed on 2001-JUL-16 Disallow: /mun.dontuse Disallow: /dispatch.cgi Disallow: /webstats Disallow: /cc/tsg User-agent: Googlebot-Image Disallow: /research/2003report/people/newfaculty/images/shelly_reuter.jpg User-agent: Googlebot-Image Disallow: /research/2003report/people/newfaculty/images/shelley_reuter_thumb.jpg User-agent: Googlebot-Image Disallow: /marcomm/gazette/2002-2003/jan23/resources/reuter.jpg