Angirash.com

Robots.txt is a plain txt file (HTML not allowed) that has been used by various search engines now a days. This file tells search engines that what to index on a certain website and what not to. Different tags are available for robots.txt (which I am explaining in detail in this post) that allows user [...]