how to... batch download and rename

Last Updated: 2015-04-12

The other day I wanted to download a bunch of pdf files - the only problem was each of it was linked from a seperate site, but fortunately following a pattern. So i had to generate a list of links for wget to download and used awk and rename to properly name the files. The following is just a note of a couple of commands so i don't have to remember them in case i need to do this again ;)

open csv in spreadsheet, copy row and edit-paste special... to transform them into a column. join the cells with


use xarg to give the list to wget:

xargs wget < linklist.txt

and finally cut the files (remove all after & and before =) and add an extension:

rename 's/&.*//g' filename* 
rename 's/.*=//g' filename*
ls | xarg -t -i mv {} {}.pdf
Not yet rated