robots.txt
- URL:
- http://www.ucs.mun.ca/robots.txt 🔗
- Replay URL:
- https://web.archive.org/web/20140923003247/http://www.ucs.mun.ca/robots.txt 🔗
- Resource Name:
- robots.txt
- Host:
- ucs.mun.ca
- Collection:
- NGOs
- Collection Id:
- 4666
- Crawl Date:
- 2014-09-23T00:32:47Z
- Source File:
- ARCHIVEIT-4666-NONE-3595-20140922235312197-00000-wbgrp-crawl058.us.archive.org-6441.warc.gz
- General Content Type:
- text
- Content Language:
- en
- Length:
- 622
- Content:
- User-agent: * Disallow: /cgi-bin Disallow: /old Disallow: /OLD Disallow: /test Disallow: /test3 Disallow: /RCS Disallow: /webinator # was /mun, which caused /mun* directories to be excluded from webinator # changed on 2001-JUL-16 Disallow: /mun.dontuse Disallow: /dispatch.cgi Disallow: /webstats Disallow: /cc/tsg User-agent: Googlebot-Image Disallow: /research/2003report/people/newfaculty/images/shelly_reuter.jpg User-agent: Googlebot-Image Disallow: /research/2003report/people/newfaculty/images/shelley_reuter_thumb.jpg User-agent: Googlebot-Image Disallow: /marcomm/gazette/2002-2003/jan23/resources/reuter.jpg