O'Reilly

Retrieve Headers with cURL

By on  

We all know the cURL is incredibly useful.  We can retrieve remote content with curl, post to a remote URL, and perform hundreds of other tasks.  One simple task that can be completed is simply retrieving basic response headers.  To test the robot indexing prevention header I added to the Mozilla Developer Network, I used one simple cURL command to grab all headers from an address.

The Shell

The cURL command is short and sweet:

curl -I davidwalsh.name

Said command provides a list that looks similar to:

HTTP/1.1 200 OK
Date: Fri, 14 Sep 2012 21:51:17 GMT
Server: Apache/2.2.3 (CentOS)
Last-Modified: Fri, 14 Sep 2012 21:51:00 GMT
Accept-Ranges: bytes
Content-Length: 10910
Cache-Control: max-age=1, private, must-revalidate
Expires: Fri, 14 Sep 2012 22:51:00 GMT
Vary: Accept-Encoding,Cookie
X-Powered-By: W3 Total Cache/0.9.2.4
X-Pingback: http://davidwalsh.name/xmlrpc.php
Pragma: public
Connection: close
Content-Type: text/html; charset=UTF-8

This command is helpful when ensuring a given header has been correctly set within your programming, as well as seeing where a given short URL may redirect to:

$ curl -I bit.ly/Q8f9o

HTTP/1.1 301 Moved
Server: nginx
Date: Fri, 14 Sep 2012 21:53:14 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: _bit=5053a74a-0011d-0688d-311cf10a;domain=.bit.ly;expires=Wed Mar 13 21:53:14 2013;path=/; HttpOnly
Cache-control: private; max-age=90
Location: http://davidwalsh.name/
MIME-Version: 1.0
Content-Length: 115

It's also useful to see the server name, expires information and more.  I also appreciate that it's a clean list and no other information is pushed into the response.  If you get some time, cURL out to different popular domains and see what headers they send -- you could be surprised!

Track.js Error Reporting

Recent Features

  • 5 Awesome New Mozilla Technologies You’ve Never Heard Of

    My trip to Mozilla Summit 2013 was incredible.  I've spent so much time focusing on my project that I had lost sight of all of the great work Mozillians were putting out.  MozSummit provided the perfect reminder of how brilliant my colleagues are and how much...

  • 5 More HTML5 APIs You Didnât Know Existed

    The HTML5 revolution has provided us some awesome JavaScript and HTML APIs.  Some are APIs we knew we've needed for years, others are cutting edge mobile and desktop helpers.  Regardless of API strength or purpose, anything to help us better do our job is a...

Incredible Demos

Discussion

  1. Thanks for this great tips for cURL.
    I did not know before that i can get header with cURL

  2. Alex

    I’m using Charles (on desktop), it can show both the request and response headers. Handy even when not trying to cheat for water on farmville 2 :P

    I highly recommend it to other devs as it has many cool features (such as throttling, charts and request/response manipulation).

  3. That’s a useful tip – thanks! I’ve always done “lynx -head dump” but that’s a lot quicker and easier.

    One thing to be aware of is that it makes a HEAD request. I would think it normally will return the same header values as a GET or POST but still something worth thinking about depending what you are using it for.

Wrap your code in <pre class="{language}"></pre> tags, link to a GitHub gist, JSFiddle fiddle, or CodePen pen to embed!

Recently on David Walsh Blog

  • Loading Static Templates for Intern Testing

    I use Intern by SitePen for all of my JavaScript functional testing.  Intern has loads of features other functional test frameworks don't and it's completely Promise-based -- something I got very used to when I used the Dojo Toolkit every day. Async test creation can...

  • Convert Video to mp3

    Let's all be honest for a moment:  we've all ... not paid for ... music.  Whether it was via a file sharing app like Kazaa or Napster, or it was downloading and seeding on bittorrent, or maybe even downloading a music video and ripping its audio,...

  • Sort git Branches by Date

    I'll be first person to admit I don't do as much git repository maintenance as I should.  I rarely delete branches which have been merged, so a git branch execution shows me a mile-long list of branches that likely aren't relevant.  The best way to find branches I've recently...

  • Best Tools and Resources for Web Professionals in 2015

    Looking for the right resources to help you satisfy the needs of your clients? On the lookout for the best tools to help you increase your revenue? Searching for the right software to help you improve your business? Well, then you’ve come to the right place....

  • JavaScript Polling

    Polling with JavaScript is one of those ugly but important functions within advanced front-end user experience and testing practices.  Sometimes there isn't the event you can hook into to signify that a given task is complete, so you need to get your hands dirty and simply poll for...