O'Reilly

Retrieve Headers with cURL

By on  

We all know the cURL is incredibly useful.  We can retrieve remote content with curl, post to a remote URL, and perform hundreds of other tasks.  One simple task that can be completed is simply retrieving basic response headers.  To test the robot indexing prevention header I added to the Mozilla Developer Network, I used one simple cURL command to grab all headers from an address.

The Shell

The cURL command is short and sweet:

curl -I davidwalsh.name

Said command provides a list that looks similar to:

HTTP/1.1 200 OK
Date: Fri, 14 Sep 2012 21:51:17 GMT
Server: Apache/2.2.3 (CentOS)
Last-Modified: Fri, 14 Sep 2012 21:51:00 GMT
Accept-Ranges: bytes
Content-Length: 10910
Cache-Control: max-age=1, private, must-revalidate
Expires: Fri, 14 Sep 2012 22:51:00 GMT
Vary: Accept-Encoding,Cookie
X-Powered-By: W3 Total Cache/0.9.2.4
X-Pingback: http://davidwalsh.name/xmlrpc.php
Pragma: public
Connection: close
Content-Type: text/html; charset=UTF-8

This command is helpful when ensuring a given header has been correctly set within your programming, as well as seeing where a given short URL may redirect to:

$ curl -I bit.ly/Q8f9o

HTTP/1.1 301 Moved
Server: nginx
Date: Fri, 14 Sep 2012 21:53:14 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: _bit=5053a74a-0011d-0688d-311cf10a;domain=.bit.ly;expires=Wed Mar 13 21:53:14 2013;path=/; HttpOnly
Cache-control: private; max-age=90
Location: http://davidwalsh.name/
MIME-Version: 1.0
Content-Length: 115

It's also useful to see the server name, expires information and more.  I also appreciate that it's a clean list and no other information is pushed into the response.  If you get some time, cURL out to different popular domains and see what headers they send -- you could be surprised!

Track.js Error Reporting

Recent Features

  • Responsive Images: The Ultimate Guide

    Chances are that any Web designers using our Ghostlab browser testing app, which allows seamless testing across all devices simultaneously, will have worked with responsive design in some shape or form. And as today’s websites and devices become ever more varied, a plethora of responsive images...

  • fetch API

    One of the worst kept secrets about AJAX on the web is that the underlying API for it, XMLHttpRequest, wasn't really made for what we've been using it for.  We've done well to create elegant APIs around XHR but we know we can do better.  Our effort to...

Incredible Demos

  • Introducing MooTools ScrollSidebar

    How many times are you putting together a HTML navigation block or utility block of elements that you wish could be seen everywhere on a page? I've created a solution that will seamlessly allow you to do so: ScrollSidebar. ScrollSidebar allows you...

  • MooTools & Printing – Creating a Links Table of Contents

    One detail we sometimes forget when considering print for websites is that the user cannot see the URLs of links when the page prints. While showing link URLs isn't always important, some websites could greatly benefit from doing so. This tutorial will show you...

Discussion

  1. Thanks for this great tips for cURL.
    I did not know before that i can get header with cURL

  2. Alex

    I’m using Charles (on desktop), it can show both the request and response headers. Handy even when not trying to cheat for water on farmville 2 :P

    I highly recommend it to other devs as it has many cool features (such as throttling, charts and request/response manipulation).

  3. That’s a useful tip – thanks! I’ve always done lynx -head dump but that’s a lot quicker and easier.

    One thing to be aware of is that it makes a HEAD request. I would think it normally will return the same header values as a GET or POST but still something worth thinking about depending what you are using it for.

Wrap your code in <pre class="{language}"></pre> tags, link to a GitHub gist, JSFiddle fiddle, or CodePen pen to embed!

Recently on David Walsh Blog

  • OSCON Portland:  Conference  Discount!

    O'Reilly puts on the best web industry conferences in the world.  These conferences include Fluent Conference, Velocity Conference, and the upcoming OSCON in Portland, Oregon from July 20-24.  Open Source Convention (OSCON) is a conference that focuses specifically on open source developers and the tools and possibilities...

  • Follow Redirects with cURL

    I love playing around with cURL. There's something about loading websites via command line that makes me feel like some type of smug hacker, just like tweeting from command line does. I recently cURL'd the Google homepage and saw the following: I found it weird that Google...

  • Developers Have WordPress, Amateurs Have Squarespace, Professional Designers Have the NEW Webydo!

    Web design platforms have traditionally come in one of two varieties. There are the solutions like WordPress and Drupal that are incredibly powerful, but an understanding of web development and coding is required to be able to use those platforms effectively. On the other side of the...

  • Chris Coyierâs Favorite CodePen Demos II

    Hey everyone! Before we get started, I just want to say it’s damn hard to pick this few favorites on CodePen. Not because, as a co-founder of CodePen, I feel like a dad picking which kid he likes best (RUDE). But because there is just so...

  • GSAP + SVG For Power Users: Motion Along A Path

    Now that the GreenSock API is picking up steam, there are many tutorials and Getting Started guides out there to provide good introductions to the library, not to mention GreenSock’s own Forum and Documentation. This article isn’t intended for beginners, but rather a...