robots.txt Rerouting on Development Servers
Every website should have a robots.txt file. Some bots hit sites so often that they slow down performance, other bots simply aren't desirable. robots.txt files can also be used to communicate sitemap location and limit request rate. It's important that the correct robots.txt file is served on development servers though, and that file is usually much different than your production robots.txt file. Here's a quick .htaccess snippet you can use to make that happen:
RewriteCond %{HTTP_HOST} devdomain
RewriteRule ^robots.txt$ robots-go-away.txt [L]
The robots-go-away.txt file most likely directs robots not to index anything, unless you want your dev server to be indexed for some reason (hint: you really don't want this).
![How to Create a Twitter Card]()
One of my favorite social APIs was the Open Graph API adopted by Facebook. Adding just a few META tags to each page allowed links to my article to be styled and presented the way I wanted them to, giving me a bit of control...
![Create a Sheen Logo Effect with CSS]()
I was inspired when I first saw Addy Osmani's original ShineTime blog post. The hover sheen effect is simple but awesome. When I started my blog redesign, I really wanted to use a sheen effect with my logo. Using two HTML elements and...
![Camera and Video Control with HTML5]()
Client-side APIs on mobile and desktop devices are quickly providing the same APIs. Of course our mobile devices got access to some of these APIs first, but those APIs are slowly making their way to the desktop. One of those APIs is the getUserMedia API...
![dwImageProtector Plugin for jQuery]()
I've always been curious about the jQuery JavaScript library. jQuery has captured the hearts of web designers and developers everywhere and I've always wondered why. I've been told it's easy, which is probably why designers were so quick to adopt it NOT that designers...
Here’s an example showing how to include multiple development domains:
RewriteCond %{HTTP_HOST} ^localhost [OR] RewriteCond %{HTTP_HOST} ^example.dev [OR] RewriteCond %{HTTP_HOST} ^test.example.com [OR] RewriteCond %{HTTP_HOST} ^staging.example.com RewriteRule ^robots.txt$ robots-disallow.txt [L]use vagrant