Directing the Googlebot
While setting up PyBlosxom there were a few things I wanted to be able to do. The most important was being able to direct bots around my site, more specifically, the googlebot. I did some research and found a few sites that explain how the googlebot works and how you can guide it though your site.
I found Scribbling.net’s article, “Help the Googlebot understand your web site” which describes how the googlebot should index a blog. Basically, you want google to index your posts, not your main page. You do this so people can find the actual post about a topic, not your main page that has most likely changed since googlebot last indexed your site. They show that you can use metatags telling bots when and when not to index a page.
To do this using PyBlosxom you can use the comments plugin and “comment-story” flavour file with the meta tag telling googlebot to index this page and the regular “story” flavour file telling it not to index the page. Out of the box, the comments plugin would display comments any time you viewed a page with one post. This is a problem when using the calender and categories plugins because it would show the comments when viewing categories or dates with only one post, even though you were not viewing the actual post. We do not want this because it means that we will be telling google to index directories, not post pages. To fix this I modified the comments plugin so that it will only show comments when viewing an actual post. Here is my modified comments plugin for anyone interested in doing this with their blog.