How to Use wget, the Ultimate Command Line Downloading Tool
Newer isnt always better, and the command is proof. First released back in , this application is still one of the best download managers on the planet. Whether you want to download a single file, an entire folder, or even mirror an entire website, wget lets you do it with just a few keystrokes.
Of course, theres a reason not everyone uses wget: its a command line application, and as such takes a bit of time for beginners to learn. Here are the basics, so you can get started.
How to Install wget
Before you can use wget, you need to install it. How to do so varies depending on your computer:
Once youve installed wget, you can start using it immediately from the command line. Lets download some files!
Download a Single File
Lets start with something simple. Copy the URL for a file youd like to download in your browser.
Now head back to the Terminal and type followed by the pasted URL. The file will download, and youll see progress in realtime as it does.
RELATED:How to Manage Files from the Linux Terminal: 11 Commands You Need to Know
Note that the file will download to your Terminals current folder, so youll want to to a different folder if you want it stored elsewhere. If youre not sure what that means, check out our guide to managing files from the command line. The article mentions Linux, but the concepts are the same on macOS systems, and Windows systems running Bash.
Continue an Incomplete Download
If, for whatever reason, you stopped a download before it could finish, dont worry: wget can pick up right where it left off. Just use this command:
The key here is , which is an option in command line parlance. This particular option tells wget that youd like to continue an existing download.
Mirror an Entire Website
If you want to download an entire website, wget can do the job.
By default, this will download everything on the site rushbrookrathbone.co.uk, but youre probably going to want to use a few more options for a usable mirror.
- changes links inside each downloaded page so that they point to each other, not the web.
- downloads things like style sheets, so pages will look correct offline.
- stops wget from downloading parent sites. So if you want to download rushbrookrathbone.co.uk, you wont end up with the parent page.
Combine these options to taste, and youll end up with a copy of any website that you can browse on your computer.
Note that mirroring an entire website on the modern Internet is going to take up a massive amount of space, so limit this to small sites unless you have near-unlimited storage.
Download an Entire Directory
If youre browsing an FTP server and find an entire folder youd like to download, just run:
The in this case tells wget you want a recursive download. You can also include if you want to avoid downloading folders and files above the current level.
Download a List of Files at Once
If you cant find an entire folder of the downloads you want, wget can still help. Just put all of the download URLs into a single TXT file.
then point wget to that document with the option. Like this:
Do this and your computer will download all files listed in the text document, which is handy if you want to leave a bunch of downloads running overnight.
A Few More Tricks
We could go on: wget offers a lot of options. But this tutorial is just intended to give you a launching off point. To learn more about what wget can do, type in the terminal and read what comes up. Youll learn a lot.
Having said that, here are a few other options I think are neat:
- If you want your download to run in the background, just include the option .
- If you want wget to keep trying to download even if there is a error, use the option . That will try to download 10 times; you can use whatever number you like.
- If you want to manage your bandwidth, the option will cap your download speed at KB/s. Change the number to change the rate.
Theres a lot more to learn here. You can look into downloading PHP source, or setting up an automated downloader, if you want to get more advanced.
Justin Pot has been writing about technology for over a decade, with work appearing in Digital Trends, The Next Web, Lifehacker, MakeUseOf, and the Zapier Blog. He also runs the Hillsboro Signal, a volunteer-driven local news outlet he founded.
Read Full Bio »
-
-