Disallow Robots Using Robots.txt
I develop customer websites on a publicly accessible web server so that my customers may check the progress of their website at any given time. I could use .htaccess to require username and password for the site but then I'm constantly needing to remind customers what their password is. My big concern is preventing search engines from finding their way to my development server. Luckily I can add a robots.txt file to my development server websites that will prevent search engines from indexing them.
The above directive prevents the search engines from indexing any pages or files on the website. Say, however, that you simply want to keep search engines out of the folder that contains your administrative control panel. You'd code:
Or if you wanted to allow in all spiders except Google's GoogleBot, you'd code:
What would you prevent the search engines from seeing?
Today we get to play amazing games on our super powered game consoles, PCs, VR headsets, and even mobile devices. While I enjoy playing new games these days, I do long for the retro gaming systems I had when I was a kid: the original Nintendo...
For quite a long time now websites with the so called "parallax" effect have been really popular.
In case you have not heard of this effect, it basically includes different layers of images that are moving in different directions or with different speed. This leads to a...
We've always been able to create links with protocols other than the usual HTTP, like mailto, skype, irc ,and more; they're an excellent convenience to visitors. With mobile phone browsers having become infinitely more usable, we can now extend that convenience to phone numbers:
Chris Coyier authored a post titled Better Pull Quotes: Don't Repeat Markup a while back. In his post he created great-looking pull quotes without repeating any content -- instead he uses jQuery to dynamically create the pull quotes. The following is the...