• green: OK
  • s11: OK
  • orange: OK

On Monday afternoon two automated crawlers (programs that read websites page by page) were going much faster than any search engine does. One was paging deep into a news archive; the other kept looping through an events calendar that never ends.

We set a polite pace for the first and kept the second out of the calendar pages, while leaving the rest of both websites open to them. The server was back to its normal workload within minutes, and search engines weren't affected.