robots.txt Rerouting on Development Servers
Every website should have a robots.txt file. Some bots hit sites so often that they slow down performance, other bots simply aren't desirable. robots.txt files can also be used to communicate sitemap location and limit request rate. It's important that the correct robots.txt file is served on development servers though, and that file is usually much different than your production robots.txt file. Here's a quick .htaccess snippet you can use to make that happen:
RewriteCond %{HTTP_HOST} devdomain
RewriteRule ^robots.txt$ robots-go-away.txt [L]
The robots-go-away.txt file most likely directs robots not to index anything, unless you want your dev server to be indexed for some reason (hint: you really don't want this).
![6 Things You Didn’t Know About Firefox OS]()
Firefox OS is all over the tech news and for good reason: Mozilla's finally given web developers the platform that they need to create apps the way they've been creating them for years -- with CSS, HTML, and JavaScript. Firefox OS has been rapidly improving...
![CSS vs. JS Animation: Which is Faster?]()
How is it possible that JavaScript-based animation has secretly always been as fast — or faster — than CSS transitions? And, how is it possible that Adobe and Google consistently release media-rich mobile sites that rival the performance of native apps?
This article serves as a point-by-point...
![PHP / MooTools 1.2 Accordion Helper]()
The MooTools Accordion plugin seems to be the plugin that people seem to have the most problems with. It's an awesome plugin, so I can see why so many people want to use it, but I think that may be part of the problem.
![Introducing LazyLoad 2.0]()
While improvements in browsers means more cool APIs for us to play with, it also means we need to maintain existing code. With Firefox 4's release came news that my MooTools LazyLoad plugin was not intercepting image loading -- the images were loading regardless of...
Here’s an example showing how to include multiple development domains:
RewriteCond %{HTTP_HOST} ^localhost [OR] RewriteCond %{HTTP_HOST} ^example.dev [OR] RewriteCond %{HTTP_HOST} ^test.example.com [OR] RewriteCond %{HTTP_HOST} ^staging.example.com RewriteRule ^robots.txt$ robots-disallow.txt [L]use vagrant