Popular:
 Building Resilient Systems on AWS: Learn how to design and implement a resilient, highly available, fault-tolerant infrastructure on AWS.

Follow Redirects with cURL

By  on  

I love playing around with cURL. There's something about loading websites via command line that makes me feel like some type of smug hacker, just like tweeting from command line does.

I recently cURL'd the Google homepage and saw the following:

curl google.com

#<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
#<TITLE>301 Moved</TITLE></HEAD><BODY>
#<H1>301 Moved</H1>
#The document has moved
#<A HREF="http://www.google.com/">here</A>.
#</BODY></HTML>

I found it weird that Google does the initial redirect but I still want to get the source of the Google homepage with cURL, as with any site that may do a redirect without you noticing. Luckily it's just a single flag:

curl -L google.com

#<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en">...

The -L flag instructs cURL to follow any redirect so that you reach the eventual endpoint. Those tiny redirects are just noise anyways, right?

Request Metrics real user monitoring
Request Metrics real user monitoring
Request Metrics real user monitoring
Request Metrics real user monitoring

Recent Features

  • By
    5 Awesome New Mozilla Technologies You&#8217;ve Never Heard Of

    My trip to Mozilla Summit 2013 was incredible.  I've spent so much time focusing on my project that I had lost sight of all of the great work Mozillians were putting out.  MozSummit provided the perfect reminder of how brilliant my colleagues are and how much...

  • By
    Chris Coyier&#8217;s Favorite CodePen Demos

    David asked me if I'd be up for a guest post picking out some of my favorite Pens from CodePen. A daunting task! There are so many! I managed to pick a few though that have blown me away over the past few months. If you...

Incredible Demos

  • By
    Spatial Navigation

    Spatial navigation is the ability to navigate to focusable elements based on their position in a given space.  Spatial navigation is a must when your site or app must respond to arrow keys, a perfect example being a television with directional pad remote.  Firefox OS TV apps are simply...

  • By
    WebSocket and Socket.IO

    My favorite web technology is quickly becoming the WebSocket API. WebSocket provides a welcomed alternative to the AJAX technologies we've been making use of over the past few years. This new API provides a method to push messages from client to server efficiently...

Discussion

  1. Dan

    Is there a way to do this outside of the command line in PHP?

    • H.Gerber

      The curl-lib in PHP offers an option for that:

      $handle = curl_init();
...
curl_setopt($handle, CURLOPT_FOLLOWLOCATION, true);
...
curl_exec($handle);
...
    • Jonny

      And if the url is using/forcing https, set this option before executing the curl:

      curl_setopt($handle, CURLOPT_SSL_VERIFYPEER, false);
    • Robert Munro

      Please don’t do that, unless you really don’t care about the content of the download. It essentially says “Ignore the SSL errors if it’s broken – trust the same as you would an unencrypted URL.”

      The data will still be encrypted, but could be coming from a man-in-the-middle, not from where you thing it’s coming from.

  2. HGerber

    The implementation of curl in PHP has an option for that.

    See the following link:
    http://stackoverflow.com/questions/3519939/make-curl-follow-redirects

  3. A

    if you liked curl, you’ll love lynx..

  4. Anthony F Worm

    I hit an issue with not following redirects myself, but this was a download from a GitHub release download that was redirecting to a S3 bucket. Broke a CLI out in the wild…

    In this case, I’m not sure it’s noise (to get at your closing question). I was intentionally not following redirects, as one should not expect this to be happening with GitHub downloads…

    I have a support ticket in to see what’s up with this very recent change. Maybe it was a DevOops?

Wrap your code in <pre class="{language}"></pre> tags, link to a GitHub gist, JSFiddle fiddle, or CodePen pen to embed!