perl-WWW-RobotRules
Database of robots.txt-derived permissions
This module parses /robots.txt files as specified in "A Standard for Robot Exclusion", at <https://www.robotstxt.org/robotstxt.html>. Webmasters can use the /robots.txt file to forbid conforming robots from accessing parts of their web site.
Releases Overview
| Release | Stable | Testing |
|---|---|---|
| Fedora Rawhide | 6.02-30.fc35 | - |
| Fedora 35 | 6.02-30.fc35 | - |
| Fedora 34 | 6.02-28.fc34 | - |
Package Info
- Upstream: https://metacpan.org/release/WWW-RobotRules
- License(s): GPL+ or Artistic
- Maintainers: mspacek, jplesnik
You can contact the maintainers of this package via email at
perl-WWW-RobotRules dash maintainers at fedoraproject dot org.
Sources on Pagure