SEO is evolving. Your crawler should too.

In the meantime, check out our blog