Bot-Allow-Content WordPress plugin

5 Comments

This is the homepage of the Bot-Allow-Content plugin for WordPress, which lets user defined bots to index the full content through RSS and Atom feeds that normally serve only summaries. Please note that this plugin is discontinued and will not be developed any further.

This WordPress plugin allows pre-defined bots to index the full content of your posts through the RSS and Atom feeds. Obviously, it could be useful only to those who serve summaries of their posts.

My personal opinion about the all-time-classic question "Summaries or full content in the feed?" is absolutely: summaries. A publisher shouldn’t force a reader download the whole articles or blog posts, but rather provide an excerpt of his work and let the reader decide if he is interested in reading the full content. I tend to easily unsubscribe from feeds that serve lengthy content. On the other hand, this approach has one big disadvantage. Not only the human readers receive the summaries, but also the various blog search engine bots. This means that only these summaries get indexed by them, reducing the possibilities that your post would be returned in a blog search engine’s results, after searching for terms that do not exist in the excerpt, but somewhere in the rest of the article.

So, I decided to write this small plugin, which lets a pre-defined list of bots read the full posted content through the feeds. It assumes that you serve summaries in your feeds, although this setting has no impact on the plugin’s functionality. You do not have to supply the full bot’s USER_AGENT string, but just a part of it.

Installation

  1. Copy bot-allow-content.php to your /wp-content/plugins/ directory.
  2. Activate the plugin through the administration panel.

This plugin has effect on the Atom (1.0 and 0.3 are supported), the RDF and the RSS 2.0 feeds. The RSS 0.92 feed serves only summaries.

Configuration

From version 0.4, this plugin provides a configuration page inside the administration panel. Go to:

Administration Panel->Options->Syndication Extra

Follow the instructions from there.

This plugin is Atom 1.0 friendly. For those of you who have hacked the default wp-atom.php file in order to conform with the Atom 1.0 Specification, there is a configuration option within the source code. It has not been added to the configuration page for obvious reasons.

Find the following in the bot-allow-content.php file and set it to TRUE

$IsAtom1 = FALSE;

You can find some guidelines on how to make your Atom feed compliant with the Atom 1.0 specification in my Design the perfect Atom feed for WordPress article.

Note: You can safely de-activate the plugin or switch to serving full content. This way all user agents will comply with your default wordpress settings for syndicated content.

License

This project is released under the terms of the GNU General Public License version 2 or later.

Downloads, Issue Tracking, Support

This project has been discontinued.

The latest public releases of the Bot-Allow-Content plugin are available at the Bot-Allow-Content development web site.

This plugin is not available on the WordPress Plugin Repository.

This project is no longer supported. Submitting support requests, issue reports or feature requests is not possible at this time.

Changelog

* Wed Nov 01 2006 – v0.6
– When the options where modified in the administration panel, a confirmation was asked. This behaviour has been corrected and the options are saved immediately.
* Wed Oct 04 2006 – v0.5
– Plugin information update
* Sat Jan 15 2006
– Updated to version 0.4
– Addition: A configuration page has been added in the Admin Panel. No editing of the source code is needed.
– Bugfix: Added a check if the list of bots is empty. The absense of it did not produce any errors though.
* Wed Jan 4 2006
– Updated to version 0.3
– The plugin was completely re-written, so that no modifications are required to be done to core wp files, but supports only WP 2 or newer.
* Tue Jan 3 2006
– Initial release v0.2

Development Status

This plugin has been discontinued.

The concept for the development of it was the facilitation of the indexing of the full content of the blog through the feeds, while the feeds normally contained only summaries. In short, this plugin had been developed during an era in which RSS and Atom feeds were used as sitemaps. Later, the use of XML Sitemaps became standard practice, which rendered the use of feeds as sitemaps much less ideal, which in turn rendered the further development of this plugin not important for the developer. So, the development stopped.

Bot-Allow-Content WordPress plugin by George Notaras is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Copyright © 2006 - Some Rights Reserved

George Notaras avatar

About George Notaras

George Notaras is the editor of the G-Loaded Journal, a technical blog about Free and Open-Source Software. George, among other things, is an enthusiast self-taught GNU/Linux system administrator. He has created this web site to share the IT knowledge and experience he has gained over the years with other people. George primarily uses CentOS and Fedora. He has also developed some open-source software projects in his spare time.

Related Articles

5 responses on “Bot-Allow-Content WordPress plugin”

  1. Cary Permalink →

    Hi, I’d really, really like to make use of this plugin, but I’m using WP 1.5 and the directions for how to modify the code are written for WP 2 users :(

    Can you explain where in the Wp 1.5 “wp-commentsrss2.php” file I’m supposed to make the change?

    Thanks so much.

  2. George Notaras Post authorPermalink →

    Hi,
    I originally wrote the 0.2 plugin for WP 2, so the line numbers differ by 1 line. :-) Here follow the exact line numbers:
    For wp-atom.php and wp-rss2.php in WordPress 1.5.X, just make the modifications that are written in the info section of the plugin version 0.2.
    The lines to modify are:
    For wp-atom.php modify line 39
    For wp-rss2.php modify line 41

    It makes no sense to implement this in the wp-commentsrss2.php file, since the comments feed always serves full content.

    Thanks for your feedback.

  3. Cary Permalink →

    Thanks for the clarification!

    Actually, I just did the upgrade to WP 2.0.1, so now I can use it as is :)

    Thanks again, and great idea for a plugin…any suggestions on other bots I might want to allow in? I just switched to a partial feed because people kept misusing it, but I’m not really all that familiar with all the bots out there.

  4. George Notaras Post authorPermalink →

    There are many bots that index blogs through feeds. At the moment, I have only set those 4 bots that I mention in the example. When I have the time, I’ll dig into the server logs and create a list.

    All the best,
    George

  5. George Notaras Post authorPermalink →

    It has been reported that this plugin as of version 0.6 does not work in WordPress 2.1.
    I will investigate this as soon as possible. Thank you.

    This plugin has been discontinued.