Allow Google Into Password Protected Areas
I have many clients that have password protected portions of their website. They love that they can charge users to access content but still allow Google into their website to index it. How does the PHP script know if it's Google accessing the site? By checking the user agent, of course.
$allow_inside = ($is_logged_in) || substr_count($_SERVER['HTTP_USER_AGENT'],'Googlebot');
The above code checks to see if the user is logged in OR if the user agent is "Googlebot." So simple!
The one disclaimer I give is that this isn't the most secure way to do things. Firefox has a plugin that allows the user to set their user agent at will. This is a risk that the customer is willing to take.
You sure as hell wouldn't want to protect sensitive information in this fashion. Exclusive news articles? Sure, why not? The exposure on Google is likely worth a few hacks getting in. I also use this for the website's built-in search engine.