Disallow Robots Using Robots.txt
I develop customer websites on a publicly accessible web server so that my customers may check the progress of their website at any given time. I could use .htaccess to require username and password for the site but then I'm constantly needing to remind customers what their password is. My big concern is preventing search engines from finding their way to my development server. Luckily I can add a robots.txt file to my development server websites that will prevent search engines from indexing them.
The above directive prevents the search engines from indexing any pages or files on the website. Say, however, that you simply want to keep search engines out of the folder that contains your administrative control panel. You'd code:
Or if you wanted to allow in all spiders except Google's GoogleBot, you'd code:
What would you prevent the search engines from seeing?
One of the worst kept secrets about AJAX on the web is that the underlying API for it,
XMLHttpRequest, wasn't really made for what we've been using it for. We've done well to create elegant APIs around XHR but we know we can do better. Our effort to...
I was inspired when I first saw Addy Osmani's original ShineTime blog post. The hover sheen effect is simple but awesome. When I started my blog redesign, I really wanted to use a sheen effect with my logo. Using two HTML elements and...
The goal of CSS is to allow styling of content and structure within a web page. We all know that, right? As CSS revisions arrive, we're provided more opportunity to control. One of the little known styling option available within the browser is text selection styling.