robots.txt Rerouting on Development Servers
Every website should have a robots.txt file. Some bots hit sites so often that they slow down performance, other bots simply aren't desirable. robots.txt files can also be used to communicate sitemap location and limit request rate. It's important that the correct robots.txt file is served on development servers though, and that file is usually much different than your production robots.txt file. Here's a quick .htaccess snippet you can use to make that happen:
RewriteCond %{HTTP_HOST} devdomain
RewriteRule ^robots.txt$ robots-go-away.txt [L]
The robots-go-away.txt file most likely directs robots not to index anything, unless you want your dev server to be indexed for some reason (hint: you really don't want this).
![5 More HTML5 APIs You Didn’t Know Existed]()
The HTML5 revolution has provided us some awesome JavaScript and HTML APIs. Some are APIs we knew we've needed for years, others are cutting edge mobile and desktop helpers. Regardless of API strength or purpose, anything to help us better do our job is a...
![CSS Gradients]()
With CSS border-radius, I showed you how CSS can bridge the gap between design and development by adding rounded corners to elements. CSS gradients are another step in that direction. Now that CSS gradients are supported in Internet Explorer 8+, Firefox, Safari, and Chrome...
![MooTools Fun with Fx.Shake]()
Adding movement to your website is a great way to attract attention to specific elements that you want users to notice. Of course you could use Flash or an animated GIF to achieve the movement effect but graphics can be difficult to maintain. Enter...
![Create a Brilliant Sprited, CSS-Powered Firefox Animation]()
Mozilla recently formally announced Firefox OS and its partners at Mobile World Congress and I couldn't be more excited. Firefox OS is going to change the lives of people in developing countries, hopefully making a name for itself in the US as well. The...
Here’s an example showing how to include multiple development domains:
RewriteCond %{HTTP_HOST} ^localhost [OR] RewriteCond %{HTTP_HOST} ^example.dev [OR] RewriteCond %{HTTP_HOST} ^test.example.com [OR] RewriteCond %{HTTP_HOST} ^staging.example.com RewriteRule ^robots.txt$ robots-disallow.txt [L]use vagrant