how to... batch download and rename

Last Updated: 2025-01-02

The other day I wanted to download a bunch of pdf files - the only problem was each of it was linked from a seperate site, but fortunately following a pattern. So i had to generate a list of links for wget to download and used awk and rename to properly name the files. The following is just a note of a couple of commands so i don't have to remember them in case i need to do this again ;)

open csv in spreadsheet, copy row and edit-paste special... to transform them into a column. join the cells with 

=a1&a2&a3.

use xarg to give the list to wget: 

xargs wget < linklist.txt

and finally cut the files (remove all after & and before =) and add an extension: 

rename 's/&.*//g' filename* 
rename 's/.*=//g' filename*
ls | xarg -t -i mv {} {}.pdf
Not yet rated

94 Comments

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

-1 OR 2+602-602-1=0+0+0+1 --: on 2025-02-10

1

-1 OR 2+624-624-1=0+0+0+1: on 2025-02-10

1

-1' OR 2+583-583-1=0+0+0+1 --: on 2025-02-10

1

-1' OR 2+643-643-1=0+0+0+1 or 'goqUTiIx'=': on 2025-02-10

1

-1" OR 2+97-97-1=0+0+0+1 --: on 2025-02-10

1

if(now()=sysdate(),sleep(15),0): on 2025-02-10

1

mGZaLUfS0'XOR(if(now()=sysdate(),sleep(15),0))XOR'Z: on 2025-02-10

1

mGZaLUfS0"XOR(if(now()=sysdate(),sleep(15),0))XOR"Z: on 2025-02-10

1

(select(0)from(select(sleep(15)))v)/*'+(select(0)from(select(sleep(15)))v)+'"+(select(0)from(select(sleep(15)))v)+"*/: on 2025-02-10

1

mGZaLUfS-1 waitfor delay '0:0:15' --: on 2025-02-10

1

mGZaLUfS88gaAod4'; waitfor delay '0:0:15' --: on 2025-02-10

1

mGZaLUfSdxtMyXIO' OR 748=(SELECT 748 FROM PG_SLEEP(15))--: on 2025-02-10

1

mGZaLUfSM6WWJBFx') OR 525=(SELECT 525 FROM PG_SLEEP(15))--: on 2025-02-10

1

mGZaLUfSxMjn8FRP')) OR 126=(SELECT 126 FROM PG_SLEEP(15))--: on 2025-02-10

1

mGZaLUfS'||DBMS_PIPE.RECEIVE_MESSAGE(CHR(98)||CHR(98)||CHR(98),15)||': on 2025-02-10

1

mGZaLUfS'": on 2025-02-10

1

mGZaLUfS????%2527%2522\'\": on 2025-02-10

1

@@7DCGs: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

1

mGZaLUfS: on 2025-02-10

-1 OR 2+516-516-1=0+0+0+1 --

mGZaLUfS: on 2025-02-10

-1 OR 2+876-876-1=0+0+0+1

mGZaLUfS: on 2025-02-10

-1' OR 2+233-233-1=0+0+0+1 --

mGZaLUfS: on 2025-02-10

-1' OR 2+39-39-1=0+0+0+1 or 'P2zHnyjd'='

mGZaLUfS: on 2025-02-10

-1" OR 2+175-175-1=0+0+0+1 --

mGZaLUfS: on 2025-02-10

1*if(now()=sysdate(),sleep(15),0)

mGZaLUfS: on 2025-02-10

10'XOR(1*if(now()=sysdate(),sleep(15),0))XOR'Z

mGZaLUfS: on 2025-02-10

10"XOR(1*if(now()=sysdate(),sleep(15),0))XOR"Z

mGZaLUfS: on 2025-02-10

(select(0)from(select(sleep(15)))v)/*'+(select(0)from(select(sleep(15)))v)+'"+(select(0)from(select(sleep(15)))v)+"*/

mGZaLUfS: on 2025-02-10

1-1; waitfor delay '0:0:15' --

mGZaLUfS: on 2025-02-10

1-1); waitfor delay '0:0:15' --

mGZaLUfS: on 2025-02-10

1-1 waitfor delay '0:0:15' --

mGZaLUfS: on 2025-02-10

1U8UlMuih'; waitfor delay '0:0:15' --

mGZaLUfS: on 2025-02-10

1-1 OR 82=(SELECT 82 FROM PG_SLEEP(15))--

mGZaLUfS: on 2025-02-10

1-1) OR 463=(SELECT 463 FROM PG_SLEEP(15))--

mGZaLUfS: on 2025-02-10

1-1)) OR 895=(SELECT 895 FROM PG_SLEEP(15))--

mGZaLUfS: on 2025-02-10

1zJJU33kj' OR 931=(SELECT 931 FROM PG_SLEEP(15))--

mGZaLUfS: on 2025-02-10

1AR9PLSDX') OR 75=(SELECT 75 FROM PG_SLEEP(15))--

mGZaLUfS: on 2025-02-10

17qcdiSQk')) OR 878=(SELECT 878 FROM PG_SLEEP(15))--

mGZaLUfS: on 2025-02-10

1*DBMS_PIPE.RECEIVE_MESSAGE(CHR(99)||CHR(99)||CHR(99),15)

mGZaLUfS: on 2025-02-10

1'||DBMS_PIPE.RECEIVE_MESSAGE(CHR(98)||CHR(98)||CHR(98),15)||'

mGZaLUfS: on 2025-02-10

1'"

mGZaLUfS: on 2025-02-10

1????%2527%2522'"

mGZaLUfS: on 2025-02-10

@@e6Ule

Add a comment:

*Required fields