# $Id: robots.txt,v 1.18 2003/08/20 15:42:41 krusch Exp $ # # This is a file retrieved by webwalkers a.k.a. spiders that # conform to a defacto standard. # See # # Comments to the webmaster should be posted at # # Format is: # User-agent: # Disallow: | # Flag Date By Reason # $l1- 19950130 epc finally understood what the file was for! # $L2= 19960909 epc fixed url since mak moved to Webcrawler... # $L3= 19970811 epc drop /Stretch # $L4= 19991102 krusch fixed User-agent capitalization and contact info # $L5= 20010327 krusch Updated disallow rules # ----------------------------------------------------------------------------- User-agent: * Disallow: /kr/common Disallow: /kr/cgi-bin Disallow: /kr/board Disallow: /kr/jsp Disallow: /kr/i/ Disallow: /kr/img Disallow: /kr/include Disallow: /kr/email Disallow: /kr/community Disallow: /kr/data Disallow: /kr/membersslsample Disallow: /kr/sso Disallow: /kr/bbs Disallow: /developerworks/kr/servlet Disallow: /developerworks/servlet