O'Reilly

Download A File and wget

By on  

Using the cURL package isn't the only way to download a file. You can also use the wget command to download any URL.

The Bash Script

wget http://davidwalsh.name/ -O davidwalsh.name.txt

I'm guessing that wget and cURL aren't the only ways to download a file, but these are two easy ways to do so.

Also, look forward to more posts featuring wget -- it has a plethora of uses!

Track.js Error Reporting

Recent Features

  • Introducing MooTools Templated

    One major problem with creating UI components with the MooTools JavaScript framework is that there isn't a great way of allowing customization of template and ease of node creation. As of today, there are two ways of creating: new Element Madness The first way to create UI-driven...

  • Write Better JavaScript with Promises

    You've probably heard the talk around the water cooler about how promises are the future. All of the cool kids are using them, but you don't see what makes them so special. Can't you just use a callback? What's the big deal? In this article, we'll...

Incredible Demos

  • MooTools Kwicks Plugin

    I wrote a post titled Get Slick with MooTools Kwicks ages ago. The post was quite popular and the effect has been used often. Looking back now, the original code doesn't look as clean as it could. I've revised the original...

  • Facebook Sliders With Mootools and CSS

    One of the great parts of being a developer that uses Facebook is that I can get some great ideas for progressive website enhancement. Facebook incorporates many advanced JavaScript and AJAX features: photo loads by left and right arrow, dropdown menus, modal windows, and...

Discussion

  1. Ahh yes, wget and cURL are soon becoming my most used utilities. Very great programs.

    It is a little trickier but you can also download flash videos on sites that allow you to buffer, like YouTube. I havn’t yet been able to download flash videos that only allow streaming, like hulu.

    Using wget to Download Youtube Videos

  2. xpix

    Why would you use something like that to download a page?

  3. @xpix,
    You would use something like that to download a page for processing. Through a script.

    For example if you wanted to automatically get the latest ISO of a Linux distro you could set up a cron job to check for updates every single day and download the latest version through wget.

    You could use wget to download a stats page, run a regular expression and print out certain pieces.

    Another example, you can use it to interact with almost all of Google’s services.

    wget --post-file template_entry.xml --header "Authorization: GoogleLogin auth=ABCDEFG" --header "Content-Type: application/atom+xml" "http://picasaweb.google.com/data/feed/api/user/brad.gushue"");

    The above code would post a new album to your picasa web albums.

    The possibilities are limitless.

  4. Not Me

    So what does Cygwin have to do with any of these posts?

  5. @Anon: I use Cygwin on my Windows computer. Obviously, you don’t need Cygwin if you run Linux.

  6. Not Me

    Or the Mac OS, or any flavor of Unix, or ….

  7. xpix

    Thanks for the answer Mark.

    I am building quite simple application and I never felt the need to to do any of that.

  8. I’ve used both wget and curl for downloading jobs, never found any difference between the two.

  9. TeMc

    @Not Me:

    Not though that Mac OS X (atleast 10.5 Leopard, and maybe 10.6) does not support the wget command by default.

    You have to install
    * XCode Developer Tools (do not untick “UNIX” in the install) version 3.1.4 (for Leopard) or 3.2 (for Snow Leopard)
    * MacPorts 1.8.2 (as of writing) for Leopard and another version for Snow Leopard
    * and then install the wget-command into that.

    Tho the above will open a lot of possibilities, it’s not unfair to say that it’s a pain to do all that to get to 1 specific applicatoin :-(

    (for allya Googlers:
    – Get a free account at the Apple Developer Connection and download the xCode Developer Tools (if you’re using Leopard, you’ll notice the version on the ADC-site is by default 3.2 for Snow Leoaprd, read here ( http://www.mac-forums.com/forums/os-x-darwin-development/165766-xcode-leopard.html#post901636 ) to know how to get the old Leopard version
    – Then http://macports.org/ should get you going towards MacPorts and http://wget.darwinports.com/

    PS: DarwinPorts is the old name of MacPorts. Some tools still refer to DarwinPorts.

    TeMc

  10. Michael

    If a URL forces you to download a CSV file, how do we use PHP to download that onto a folder in the sever?

Wrap your code in <pre class="{language}"></pre> tags, link to a GitHub gist, JSFiddle fiddle, or CodePen pen to embed!

Recently on David Walsh Blog

  • Chris Coyierâs Favorite CodePen Demos II

    Hey everyone! Before we get started, I just want to say it’s damn hard to pick this few favorites on CodePen. Not because, as a co-founder of CodePen, I feel like a dad picking which kid he likes best (RUDE). But because there is just so...

  • GSAP + SVG For Power Users: Motion Along A Path

    Now that the GreenSock API is picking up steam, there are many tutorials and Getting Started guides out there to provide good introductions to the library, not to mention GreenSock’s own Forum and Documentation. This article isn’t intended for beginners, but rather a...

  • Copy a Directory from Command Line

    Copying a directory for the sake of backup is something I do often, especially when I'm trying to figure out why something isn't working when I use an external library.  I'll copy the directory structure as a backup, mess around with the original source until I find a solution,...

  • Hotjar &#8211; All-in-one Analytics and Feedback

    Website analytics are a massive business -- the more data you can collect with regard to your users' behaviors on your site, the more you can increase and maximise conversion...and increased conversion is always good.  Sometimes increase conversion means more money, improved user experience, viewer retention,...

  • Crafting a 3D React Carousel

    There is something in me that is amazed but beautiful 3D interfaces. And it doesn’t matter whether they’re functional like Gyroscope features menu, technology demonstrators like the amazing periodic table demo from famous or they’re artistic representation pushing the limits of...