About Robots (parent)

Parent module for robots exclusion protocol library.

Project Modules

This project has declared the following modules:

Name Description
Robots (core) Support for Robots Exclusion Protocol, including parsing and matching against robot.txt directives.
Robots (integration tests) Integration tests for robots exclusion protocol library.
Robots (command-line interface) Command-line interface for robots exclusion protocol library.