Boston Media Domain

Controlling what content is blocked from being found in search engines is crucial for many websites. Fortunately, the major search engines and other well-behaved robots observe the Robots Exclusion Protocol (REP), which has evolved organically since the early 1990’s to provide a set of controls over what parts of a web site search engines robots [...]

[Read more →]