Overview
The page says: “There is nothing on this page.” The source is also clean — no referenced files this time.
There is, however, a comment in the source:
Hints
Hint 1 — How does Google find pages?
Hint 1 — How does Google find pages?
Search engine crawlers discover pages by following links. Website owners can instruct crawlers to skip certain paths using a well-known standard file in the web root. What is that file called, and what does it contain?
Hint 2 — What's in robots.txt?
Hint 2 — What's in robots.txt?
Navigate to
/robots.txt. The file uses Disallow: directives to tell crawlers which paths to avoid. These paths are hidden from search engines — but not from you. Whatever is listed there is worth visiting.Solution
Full walkthrough
Full walkthrough
1
Read robots.txt
Navigate to
http://natas3.natas.labs.overthewire.org/robots.txt:2
Browse the disallowed directory
Navigate to
http://natas3.natas.labs.overthewire.org/s3cr3t/.Directory listing is enabled, exposing users.txt.3
Read users.txt
Open
users.txt to retrieve the password:robots.txt is publicly readable by design — it’s meant for crawlers, but any human can open it. Listing a sensitive path under Disallow only hides it from polite bots; it advertises it to anyone looking for hidden endpoints.