# 1-The first paragraph specifies that the robot called 'webcrawler' has nothing disallowed: it may go anywhere.

# 2-The second paragraph indicates that the robot called 'lycra' has all relative URLs starting with '/' disallowed. 
# Because all relative URL's on a server start with '/', this means the entire site is closed off.

# 3-The third paragraph indicates that all other robots should not visit
# URLs starting with /tmp or /log. Note the '*' is a special token,
# meaning "any other User-agent"; you cannot use wildcard patterns
# or regular expressions in either User-agent or Disallow lines.

User-agent: webcrawler
Disallow:

User-agent: lycra
Disallow: /

User-agent: *
Disallow: /tmp
Disallow: /logs