Building Resilient Systems on AWS: Learn how to design and implement a resilient, highly available, fault-tolerant infrastructure on AWS.

Follow Redirects with cURL

By David Walsh on July 1, 2015

I love playing around with cURL. There's something about loading websites via command line that makes me feel like some type of smug hacker, just like tweeting from command line does.

I recently cURL'd the Google homepage and saw the following:

curl google.com

#<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
#<TITLE>301 Moved</TITLE></HEAD><BODY>
#<H1>301 Moved</H1>
#The document has moved
#<A HREF="http://www.google.com/">here</A>.
#</BODY></HTML>

I found it weird that Google does the initial redirect but I still want to get the source of the Google homepage with cURL, as with any site that may do a redirect without you noticing. Luckily it's just a single flag:

curl -L google.com

#<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en">...

The -L flag instructs cURL to follow any redirect so that you reach the eventual endpoint. Those tiny redirects are just noise anyways, right?

Recent Features

By David WalshNovember 2, 2015
JavaScript Promise API
While synchronous code is easier to follow and debug, async is generally better for performance and flexibility. Why "hold up the show" when you can trigger numerous requests at once and then handle them when each is ready? Promises are becoming a big part of the JavaScript world...
By David WalshAugust 15, 2011
Send Text Messages with PHP
Kids these days, I tell ya. All they care about is the technology. The video games. The bottled water. Oh, and the texting, always the texting. Back in my day, all we had was...OK, I had all of these things too. But I still don't get...

Incredible Demos

By David WalshApril 25, 2011
Background Animations Using MooTools
One of the sweet effects made easy by JavaScript frameworks like MooTools and jQuery is animation. I ran across this great jQuery tutorial that walks you through animating a background image of a page. Here's a quick MooTools code snippet that...
By David WalshNovember 21, 2011
Page Visibility API
One event that's always been lacking within the document is a signal for when the user is looking at a given tab, or another tab. When does the user switch off our site to look at something else? When do they come back?

Discussion

Dan
Is there a way to do this outside of the command line in PHP?

H.Gerber

The curl-lib in PHP offers an option for that:

$handle = curl_init();
...
curl_setopt($handle, CURLOPT_FOLLOWLOCATION, true);
...
curl_exec($handle);
...

Jonny
And if the url is using/forcing https, set this option before executing the curl:
```
curl_setopt($handle, CURLOPT_SSL_VERIFYPEER, false);
```
Robert Munro
Please don’t do that, unless you really don’t care about the content of the download. It essentially says “Ignore the SSL errors if it’s broken – trust the same as you would an unencrypted URL.”

The data will still be encrypted, but could be coming from a man-in-the-middle, not from where you thing it’s coming from.

HGerber
The implementation of curl in PHP has an option for that.

See the following link:
http://stackoverflow.com/questions/3519939/make-curl-follow-redirects
A
if you liked curl, you’ll love lynx..
Anthony F Worm
I hit an issue with not following redirects myself, but this was a download from a GitHub release download that was redirecting to a S3 bucket. Broke a CLI out in the wild…

In this case, I’m not sure it’s noise (to get at your closing question). I was intentionally not following redirects, as one should not expect this to be happening with GitHub downloads…

I have a support ticket in to see what’s up with this very recent change. Maybe it was a DevOops?

Follow Redirects with cURL

Recent Features

JavaScript Promise API

Send Text Messages with PHP

Incredible Demos

Background Animations Using MooTools

Page Visibility API

Discussion