Popular:
 Building Resilient Systems on AWS: Learn how to design and implement a resilient, highly available, fault-tolerant infrastructure on AWS.

Retrieve Headers with cURL

By  on  

We all know the cURL is incredibly useful.  We can retrieve remote content with curl, post to a remote URL, and perform hundreds of other tasks.  One simple task that can be completed is simply retrieving basic response headers.  To test the robot indexing prevention header I added to the Mozilla Developer Network, I used one simple cURL command to grab all headers from an address.

The Shell

The cURL command is short and sweet:

curl -I davidwalsh.name

Said command provides a list that looks similar to:

HTTP/1.1 200 OK
Date: Fri, 14 Sep 2012 21:51:17 GMT
Server: Apache/2.2.3 (CentOS)
Last-Modified: Fri, 14 Sep 2012 21:51:00 GMT
Accept-Ranges: bytes
Content-Length: 10910
Cache-Control: max-age=1, private, must-revalidate
Expires: Fri, 14 Sep 2012 22:51:00 GMT
Vary: Accept-Encoding,Cookie
X-Powered-By: W3 Total Cache/0.9.2.4
X-Pingback: https://davidwalsh.name/xmlrpc.php
Pragma: public
Connection: close
Content-Type: text/html; charset=UTF-8

This command is helpful when ensuring a given header has been correctly set within your programming, as well as seeing where a given short URL may redirect to:

$ curl -I bit.ly/Q8f9o

HTTP/1.1 301 Moved
Server: nginx
Date: Fri, 14 Sep 2012 21:53:14 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: _bit=5053a74a-0011d-0688d-311cf10a;domain=.bit.ly;expires=Wed Mar 13 21:53:14 2013;path=/; HttpOnly
Cache-control: private; max-age=90
Location: https://davidwalsh.name/
MIME-Version: 1.0
Content-Length: 115

It's also useful to see the server name, expires information and more.  I also appreciate that it's a clean list and no other information is pushed into the response.  If you get some time, cURL out to different popular domains and see what headers they send -- you could be surprised!

Request Metrics real user monitoring
Request Metrics real user monitoring
Request Metrics real user monitoring
Request Metrics real user monitoring

Recent Features

  • By
    Welcome to My New Office

    My first professional web development was at a small print shop where I sat in a windowless cubical all day. I suffered that boxed in environment for almost five years before I was able to find a remote job where I worked from home. The first...

  • By
    An Interview with Eric Meyer

    Your early CSS books were instrumental in pushing my love for front end technologies. What was it about CSS that you fell in love with and drove you to write about it? At first blush, it was the simplicity of it as compared to the table-and-spacer...

Incredible Demos

  • By
    Using jQuery and MooTools Together

    There's yet another reason to master more than one JavaScript library: you can use some of them together! Since MooTools is prototype-based and jQuery is not, jQuery and MooTools may be used together on the same page. The XHTML and JavaScript jQuery is namespaced so the...

  • By
    Elegant Overflow with CSS Ellipsis

    Overflow with text is always a big issue, especially in a programmatic environment. There's always only so much space but variable content to add into that space. I was recently working on a table for displaying user information and noticed that longer strings were...

Discussion

  1. Maidul

    Thanks for this great tips for cURL.
    I did not know before that i can get header with cURL

  2. Alex

    I’m using Charles (on desktop), it can show both the request and response headers. Handy even when not trying to cheat for water on farmville 2 :P

    I highly recommend it to other devs as it has many cool features (such as throttling, charts and request/response manipulation).

  3. Chris Hope

    That’s a useful tip – thanks! I’ve always done lynx -head dump but that’s a lot quicker and easier.

    One thing to be aware of is that it makes a HEAD request. I would think it normally will return the same header values as a GET or POST but still something worth thinking about depending what you are using it for.

  4. iamzesh

    It’s important to note that some servers are set to respond differently to HEADER requests than to GET requests. For example, a HEADER request returns a 200 OK while a GET request returns a 301 Moved Permanently… It can be quite confusing and unreliable depending on what you’re testing.

  5. Shyam Chathuranga

    Connection header is set to closed, however when checking via Firebug or other tools such as Pingdom shows Keep-alive. Do you know why is that so?

    Thank you,
    Shyam

Wrap your code in <pre class="{language}"></pre> tags, link to a GitHub gist, JSFiddle fiddle, or CodePen pen to embed!