robots.txt Rerouting on Development Servers
Every website should have a robots.txt file. Some bots hit sites so often that they slow down performance, other bots simply aren't desirable. robots.txt files can also be used to communicate sitemap location and limit request rate. It's important that the correct robots.txt file is served on development servers though, and that file is usually much different than your production robots.txt file. Here's a quick .htaccess snippet you can use to make that happen:
RewriteCond %{HTTP_HOST} devdomain
RewriteRule ^robots.txt$ robots-go-away.txt [L]
The robots-go-away.txt file most likely directs robots not to index anything, unless you want your dev server to be indexed for some reason (hint: you really don't want this).
![fetch API]()
One of the worst kept secrets about AJAX on the web is that the underlying API for it, XMLHttpRequest, wasn't really made for what we've been using it for. We've done well to create elegant APIs around XHR but we know we can do better. Our effort to...
![5 Ways that CSS and JavaScript Interact That You May Not Know About]()
CSS and JavaScript: the lines seemingly get blurred by each browser release. They have always done a very different job but in the end they are both front-end technologies so they need do need to work closely. We have our .js files and our .css, but...
![AJAX Page Loads Using MooTools Fx.Explode]()
Note: All credit for Fx.Explode goes to Jan Kassens.
One of the awesome pieces of code in MooTools Core Developer Jan Kassens' sandbox is his Fx.Explode functionality. When you click on any of the designated Fx.Explode elements, the elements "explode" off of the...
![NSFW Blocker Using MooTools and CSS]()
One of my guilty pleasures is scoping out the latest celebrity gossip from PerezHilton.com, DListed.com, and JoBlo.com. Unfortunately, these sites occasionally post NSFW pictures which makes checking these sites on lunch a huge gamble -- a trip to HR's office could be just a click away.
Since...
Here’s an example showing how to include multiple development domains:
RewriteCond %{HTTP_HOST} ^localhost [OR] RewriteCond %{HTTP_HOST} ^example.dev [OR] RewriteCond %{HTTP_HOST} ^test.example.com [OR] RewriteCond %{HTTP_HOST} ^staging.example.com RewriteRule ^robots.txt$ robots-disallow.txt [L]use vagrant