Lucas Ng's blog

Lucas Ng is an online marketer, a digital warrior. Trained by ancient blackhat and whitehat Jedi in the arts of SEO, PPC, CRO, web analytics, social media and more.

Googlebot interprets some pattern matching

I don’t want to list every file that I want to block. Can I use pattern matching?

Yes, Googlebot interprets some pattern matching. This is an extension of the standard, so not all bots may follow it.