Say hi to bots and crawlers with Kriti.blog.
All you have to do is … nothing. Unless you want customization.
Suggesting crawlers on how to index a page
The default meta
tag for crawlers is <meta name="robots" content="index, dofollow">
. You don’t have to add it every crawler will adhere to this specification.
But what if noindex, nofollow
is required for a particular page? Add following to your frontmatter
params:
crawlers:
robots: noindex, nofollow
Great! what about googlebot
?
params:
crawlers:
googlebot: notranslate, ...
Same goes for any other bot, add its name under params.crawlers
and corresponding directive.
You don’t have to do it for every page
It can be tiresome to add it to every page, so the alternative is to add it site wide from dashboard.
- Go to project dashboard
- Navigate to
Publish
section - Expand the
Meta Data
sub section - Add your required data