O'Reilly

Retrieve Headers with cURL

By on  

We all know the cURL is incredibly useful.  We can retrieve remote content with curl, post to a remote URL, and perform hundreds of other tasks.  One simple task that can be completed is simply retrieving basic response headers.  To test the robot indexing prevention header I added to the Mozilla Developer Network, I used one simple cURL command to grab all headers from an address.

The Shell

The cURL command is short and sweet:

curl -I davidwalsh.name

Said command provides a list that looks similar to:

HTTP/1.1 200 OK
Date: Fri, 14 Sep 2012 21:51:17 GMT
Server: Apache/2.2.3 (CentOS)
Last-Modified: Fri, 14 Sep 2012 21:51:00 GMT
Accept-Ranges: bytes
Content-Length: 10910
Cache-Control: max-age=1, private, must-revalidate
Expires: Fri, 14 Sep 2012 22:51:00 GMT
Vary: Accept-Encoding,Cookie
X-Powered-By: W3 Total Cache/0.9.2.4
X-Pingback: http://davidwalsh.name/xmlrpc.php
Pragma: public
Connection: close
Content-Type: text/html; charset=UTF-8

This command is helpful when ensuring a given header has been correctly set within your programming, as well as seeing where a given short URL may redirect to:

$ curl -I bit.ly/Q8f9o

HTTP/1.1 301 Moved
Server: nginx
Date: Fri, 14 Sep 2012 21:53:14 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: _bit=5053a74a-0011d-0688d-311cf10a;domain=.bit.ly;expires=Wed Mar 13 21:53:14 2013;path=/; HttpOnly
Cache-control: private; max-age=90
Location: http://davidwalsh.name/
MIME-Version: 1.0
Content-Length: 115

It's also useful to see the server name, expires information and more.  I also appreciate that it's a clean list and no other information is pushed into the response.  If you get some time, cURL out to different popular domains and see what headers they send -- you could be surprised!

Track.js Error Reporting

Upcoming Events

Recent Features

  • How I Stopped WordPress Comment Spam

    I love almost every part of being a tech blogger:  learning, preaching, bantering, researching.  The one part about blogging that I absolutely loathe:  dealing with SPAM comments.  For the past two years, my blog has registered 8,000+ SPAM comments per day.  PER DAY.  Bloating my database...

  • Create a CSS Flipping Animation

    CSS animations are a lot of fun; the beauty of them is that through many simple properties, you can create anything from an elegant fade in to a WTF-Pixar-would-be-proud effect. One CSS effect somewhere in between is the CSS flip effect, whereby there's...

Incredible Demos

  • WebKit-Specific Style:  -webkit-appearance

    I was recently scoping out the horrid source code of the Google homepage when I noticed the "Google Search" and "I'm Feeling Lucky" buttons had a style definition I hadn't seen before:  -webkit-appearance.  The value assigned to the style was "push-button."  They are buttons so that...

  • MooTools TextOverlap Plugin

    Developers everywhere seem to be looking for different ways to make use of JavaScript libraries. Some creations are extremely practical, others aren't. This one may be more on the "aren't" side but used correctly, my TextOverlap plugin could add another interesting design element...

Discussion

  1. Thanks for this great tips for cURL.
    I did not know before that i can get header with cURL

  2. Alex

    I’m using Charles (on desktop), it can show both the request and response headers. Handy even when not trying to cheat for water on farmville 2 :P

    I highly recommend it to other devs as it has many cool features (such as throttling, charts and request/response manipulation).

  3. That’s a useful tip – thanks! I’ve always done lynx -head dump but that’s a lot quicker and easier.

    One thing to be aware of is that it makes a HEAD request. I would think it normally will return the same header values as a GET or POST but still something worth thinking about depending what you are using it for.

  4. iamzesh

    It’s important to note that some servers are set to respond differently to HEADER requests than to GET requests. For example, a HEADER request returns a 200 OK while a GET request returns a 301 Moved Permanently… It can be quite confusing and unreliable depending on what you’re testing.

Wrap your code in <pre class="{language}"></pre> tags, link to a GitHub gist, JSFiddle fiddle, or CodePen pen to embed!

Recently on David Walsh Blog

  • Get Node.js Command Line Arguments with yargs

    Using command line arguments within Node.js apps is par for the course, especially when you're like me and you use JavaScript to code tasks (instead of bash scripts).  Node.js provides process.argv but that doesn't provide a key: value object like you'd expect: Bleh.  If you want to work with a...

  • OâReilly Velocity Conference â New York

    My favorite front-end conference has always been O'Reilly's Velocity Conference because the conference series has focused on one of the most undervalued parts of client side coding:  speed.  So often we're so excited that our JavaScript works that we forget that speed, efficiency, and performance are just as important. The next Velocity...

  • Free Download: Font Bundle Featuring 17 Incredible Typefaces

    The only thing we love more than a good font, is a good free font. So we’ve combed the Web for some of our favorite free fonts, and gathered them here in a single download. You’ll find a variety of useful typefaces, from highly geometric designs...

  • OâReilly Velocity Conference â Amsterdam

    My favorite front-end conference has always been O'Reilly's Velocity Conference because the conference series has focused on one of the most undervalued parts of client side coding:  speed.  So often we're so excited that our JavaScript works that we forget that speed, efficiency, and performance are just as important. The next Velocity...

  • CanIUse Command Line

    Every front-end developer should be well acquainted with CanIUse, the website that lets you view browser support for browser features.  When people criticize my blog posts for not detailing browser support for features within the post, I tell them to check CanIUse:  always up to date, unlike...