robots.txt Rerouting on Development Servers
Every website should have a robots.txt file. Some bots hit sites so often that they slow down performance, other bots simply aren't desirable. robots.txt files can also be used to communicate sitemap location and limit request rate. It's important that the correct robots.txt file is served on development servers though, and that file is usually much different than your production robots.txt file. Here's a quick .htaccess snippet you can use to make that happen:
RewriteCond %{HTTP_HOST} devdomain
RewriteRule ^robots.txt$ robots-go-away.txt [L]
The robots-go-away.txt file most likely directs robots not to index anything, unless you want your dev server to be indexed for some reason (hint: you really don't want this).
![9 Mind-Blowing Canvas Demos]()
The <canvas> element has been a revelation for the visual experts among our ranks. Canvas provides the means for incredible and efficient animations with the added bonus of no Flash; these developers can flash their awesome JavaScript skills instead. Here are nine unbelievable canvas demos that...
![6 Things You Didn’t Know About Firefox OS]()
Firefox OS is all over the tech news and for good reason: Mozilla's finally given web developers the platform that they need to create apps the way they've been creating them for years -- with CSS, HTML, and JavaScript. Firefox OS has been rapidly improving...
![Create a Photo Stack Effect with Pure CSS Animations or MooTools]()
My favorite technological piece of Google Plus is its image upload and display handling. You can drag the images from your OS right into a browser's DIV element, the images upload right before your eyes, and the albums page displays a sexy photo deck animation...
![CSS Fixed Position Background Image]()
Backgrounds have become an integral part of creating a web 2.0-esque website since gradients have become all the rage. If you think gradient backgrounds are too cliche, maybe a fixed position background would work for you? It does provide a neat inherent effect by...
Here’s an example showing how to include multiple development domains:
RewriteCond %{HTTP_HOST} ^localhost [OR] RewriteCond %{HTTP_HOST} ^example.dev [OR] RewriteCond %{HTTP_HOST} ^test.example.com [OR] RewriteCond %{HTTP_HOST} ^staging.example.com RewriteRule ^robots.txt$ robots-disallow.txt [L]use vagrant