Stop Confusing Your Crawlers: Three Files, Three Jobs (And Why I Kept Breaking Mine)
I spent an embarrassing amount of time last year debugging why my blog posts weren't appearing in Google search results. I had a beautiful sitemap. I had a robots.txt file. I thought I was doing everything right. Turns out, I was conflating three completely different concerns int...