Post by james masonHere is an example: https://i.imgur.com/z1HLDB3.gif
I can tediously right click on each of the downloaded files, but
when there are scores of downloads, it would be nicer if there was
a better way to simply generate the list of all files downloaded.
How can we generate a list of all files downloaded that includes
the full path where each file came from?
How to get a text list of all files downloaded in Firefox?
OK, now it's my turn.
"Download history is now stored in the Places database,
in the same file as browsing history. The downloads.sqlite
file was removed in Firefox 26. The file "downloads.json"
is used for paused downloads."
So that suggests places.sqlite has the info.
If you were thinking they were stored in a
plaintext file, no, the Firefox devs have a
sense of humor. The stuff uses databases.
Go to Synaptic Package Manager and install "sqlite3".
That should add a new application to your setup.
Being database stuff, don't expect the manual page
or the --help stuff, to explain things :-) I just
stumble around until I get something.
Exit Firefox. Take a *copy* of places.sqlite
to your work directory. Look in ~/.firefox for
things like that.
sqlite3 places.sqlite .dump > places.txt
Now, I'll pick out some specific lines.
First, the "moz_anno" definitions. This suggests to
me, that an item with "3,4,5" entries, is a download.
INSERT INTO "moz_anno_attributes" VALUES(1,'bookmarkProperties/description');
INSERT INTO "moz_anno_attributes" VALUES(2,'Places/SmartBookmark');
INSERT INTO "moz_anno_attributes" VALUES(3,'downloads/destinationFileURI'); <---
INSERT INTO "moz_anno_attributes" VALUES(4,'downloads/destinationFileName'); <---
INSERT INTO "moz_anno_attributes" VALUES(5,'downloads/metaData'); <---
INSERT INTO "moz_anno_attributes" VALUES(6,'places/excludeFromBackup');
INSERT INTO "moz_anno_attributes" VALUES(7,'PlacesOrganizer/OrganizerFolder');
INSERT INTO "moz_anno_attributes" VALUES(8,'PlacesOrganizer/OrganizerQuery');
INSERT INTO "moz_anno_attributes" VALUES(9,'URIProperties/characterSet');
But the whole thing is not in those items. So I find an example.
Item #23 has a 3, a 4, and a 5 entry. And because I know I downloaded
bootstrap.py from the mozilla site, this wasn't just an accident that
I found that one.
INSERT INTO "moz_annos" VALUES(1,23,3,NULL,'file:///home/paul/Downloads/bootstrap.py' ...);
INSERT INTO "moz_annos" VALUES(2,23,4,NULL,'bootstrap.py' ...);
INSERT INTO "moz_annos" VALUES(3,23,5,NULL,'{"state":1,"endTime":1452731744606,"fileSize":5493}' ...);
Now I look for the places entry, which corresponds to 23.
INSERT INTO "moz_places" VALUES(23,'https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py','bootstrap.py','gro.allizom.gh.' ...);
*******
OK, so I whipped together this GAWK program.
********** Start of "firefoxdownlinks.awk" **********
# firefoxdownlinks.awk
#
# The four lines of interest come from
# sqlite3 places.sqlite .dump > places.txt
#
# With file in hand, now run
#
# gawk -f firefoxdownlinks.awk places.txt | tee yourdownloads.txt
#
# bitmap variable keeps track whether the four necessary lines are present
# 15 = perfect score, print out the result
# 8421
# 8 = moz_places detected
# 4 = field 5 of moz_annos detected
# 2 = field 4 of moz_annos detected
# 1 = field 3 of moz_annos detected
#
# The four lines of interest from
# sqlite3 places.sqlite .dump > places.txt
#
# INSERT INTO "moz_places" VALUES(23,'https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py','bootstrap.py','gro.allizom.gh.' ...);
# INSERT INTO "moz_annos" VALUES(1,23,3,NULL,'file:///home/paul/Downloads/bootstrap.py' ...);
# INSERT INTO "moz_annos" VALUES(2,23,4,NULL,'bootstrap.py' ...);
# INSERT INTO "moz_annos" VALUES(3,23,5,NULL,'{"state":1,"endTime":1452731744606,"fileSize":5493}' ...);
#
# Want to keep second field of first line
# want to print fifth field of second line
# timestamp is on many of the lines, it is tenth field of first line
# timestamp is in microseconds and needs to be divided by one million
#
# I have no idea whether strftime does a good job or not.
# Check your actual download time, against a test run of this program.
# Both the raw timestamp and the text are being printed out.
#
# Output looks like this:
#
# https://hg.mozilla.org/mozilla-central/raw-file/default/python/mozboot/bin/bootstrap.py
# file:///home/paul/Downloads/bootstrap.py
# 1452731744295000 Wed Jan 13 19:35:44 Eastern Standard Time 2016
#
# The entries are *not* sorted by time. Sorting costs extra... :-)
BEGIN {
FS=","
}
/INSERT INTO "moz_annos" VALUES/ {
if ($3 == 3) {
bitmap[ $2 ] += 1
three[ $2 ] = $5
}
if ($3 == 4) {
bitmap[ $2 ] += 2
}
if ($3 == 5) {
bitmap[ $2 ] += 4
}
}
/INSERT INTO "moz_places" VALUES/ {
num_of_link = $1
gsub( /INSERT INTO "moz_places" VALUES\(/ , "", num_of_link )
bitmap[ num_of_link ] += 8
places[ num_of_link ] = $2
timestamp[ num_of_link ] = $10
}
END {
for (i in bitmap) {
if ( bitmap[ i ] == 15 ) {
gsub( /'/, "", places[ i ] )
print places[ i ]
gsub( /'/, "", three[ i ] )
print three[ i ]
print timestamp[ i ] " " strftime("%a %b %d %H:%M:%S %Z %Y", timestamp[ i ]/1000000 )
print " "
}
}
}
********** End of "firefoxdownlinks.awk" **********
Have fun,
Paul
