lbt/I2P_Website
1
0
Fork 0
forked from I2P_Developers/i2p.www
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Updated robots.txt to disallow auto-generated index pages

Browse Source
This commit is contained in:
str4d
2013-01-04 23:26:22 +00:00
parent a6a39fcb36
commit f65b4cca28
1 changed files with 11 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
i2p2www/static/robots.txt
View File

@@ -1,4 +1,12 @@
# Disallow auto-generated index pages
# allow everything for everyone... at least for now
User-agent: * User-agent: *
Disallow: Disallow: /*/blog/$
User-agent: *
Disallow: /*/blog/page/
User-agent: *
Disallow: /*/meetings/$
User-agent: *
Disallow: /*/meetings/page/