Author Topic: Suggestions on an "efficient" way to date ~ 10,000 scanned photos?  (Read 325 times)

Bob_K

  • Newbie
  • *
  • Posts: 3
Newbie, just got into exiftool and figured out how to bulk change selected (mostly global) metadata by single file or recursive directory with some wildcard use. Well that's a start, but now I was hoping to find a "simple" way to work with a selected subset of files in a directory, not readily definable by wildcards.

To explain what I want to do at a higher level: I'm digitizing some 400+ rolls of film (~ 10,000 photos) from 1975 to 2003, about 150 rolls done so far. I have them time-ordered negative by negative in the file name, but only grossly ordered by datetimeoriginal, e.g., all the 1975 pics have been given a dto of Jan 1 1975 00:01:00 incremented by 1 minute for each successive photo - just enough to get them in the right year and sort by date order.

I'd eventually like to give them all (approximate) actual dates. Lots of ways to do that but none that I've found so far that are (IMHO) efficient enough to contemplate for such a big job. My basic notion is to have some sort of continuously repeating script where I CTRL-click select a contiguous group of day "x" pictures by eyeball, set/increment a date, write it to the selected files, and keep repeating that cycle for day y and on and on with as little mouse mileage / keyboard tapping as possible. (And it would be nice if it would auto-increment the time a minute or so file-by file as well.)

This is in a Windows 10 environment. I reckon perhaps a bash script on Ubuntu for Windows at the very least. (I last used bash scripting circa 1995 but it's like riding a bicycle right? Not!) In any event I don't recollect ever passing a "CTRL-click" file list to it. Python or Perl probably better, but sorry zero experience here on those (yet). (I was an engineer, not a programmer.)

Hoping someone can steer me in a useful direction on this. Seems like there should have been other "film archivers" that have faced the same issue. Maybe I'm missing something obvious?

I have tried out well over a dozen promising-sounding utilities out there but they all seem to be "do one cycle and start completely over".

I've also mused on using a massive Excel spreadsheet via CSV input and output with Exif tool, but seems really clunky and not sure whether I can get thumbnails into it to do the eyeball "day x photos" selection.

Phil Harvey

  • ExifTool Author
  • Administrator
  • ExifTool Freak
  • *****
  • Posts: 14759
    • ExifTool Home Page
Re: Suggestions on an "efficient" way to date ~ 10,000 scanned photos?
« Reply #1 on: March 13, 2019, 07:15:57 AM »
If you want to point and click, then you need something other than ExifTool.
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

StarGeek

  • Global Moderator
  • ExifTool Freak
  • *****
  • Posts: 2516
Re: Suggestions on an "efficient" way to date ~ 10,000 scanned photos?
« Reply #2 on: March 13, 2019, 01:27:17 PM »
While I don't have any advice to offer as to any program or script to do exactly what you want, I will offer what I do that works for me.

I use a DAM (Digital asset management) to keep track of my images.  Actually, 2 versions of the same program, iMatch 3 and 5.  With either of them I can select a group of images and copy the file paths to the clipboard.  I can then either paste that list into the command line to run exiftool on those files or if the number of files is large, paste it into a temp file and use the -@ (Argfile) option to process them. (Nothing except exiftool ever writes to my files, they never write metadata the way I like it.)

There are freely available programs out there that you could do something similar, select files and get the filepaths on the clipboard.  You can even automate a lot of that with some simple Autohotkey/Autoit3 scripts.  You can even do it with just windows and a program like PathCopyCopy by setting Windows to display thumbnails, selecting the files you want, and using PathCopyCopy to right click->copy paths to clipboard.

Some DAMs may even let you directly run exiftool from within them.  iMatch5 has that ability.
Troubleshooting hints:
* When posting, include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).
* Double all percent signs (%) in a Windows batch file.
* If your GPS coords are negative, make sure and set the GpsLatitudeRef and GpsLongitudeRef tags correctly.

Bob_K

  • Newbie
  • *
  • Posts: 3
Re: Suggestions on an "efficient" way to date ~ 10,000 scanned photos?
« Reply #3 on: March 14, 2019, 02:53:53 AM »
Thanks for your input. I will look into iMatch that StarGeek suggested. However, I believe that a day of further contemplation has brought me to a fairly straight forward solution that I've "prototyped" and works (basically). The key was resolving that in lieu of inventing something new, a 2 stage solution was in order. Stage 1 being pure Windows "drag & drop" to put photos into YYY-MM-DD subdirectories using file explorer. Stage 2 being the use exiftool to parse the YYYY-MM-DD from the subdir name and write DateTimeOriginal from that.

In Stage 1 I'm using a BAT file to create subdirectories under year YYYY for every day of the year, 365 of them e.g. 1978-01-01 to 1978-12-31. Then it's basically just a brain-strain exercise with file explorer to select time-adjacent groups of pics from the upper YYYY folder and drag them into the "best guess" day subdir until I empty the YYYY dir. Do while YYYY is < 2004 (digital era) or ambition gives out.

In Stage 2, I'm using a trivial variation of a nifty command Phil provided in one of the posts further down in these Newbies posts:
      exiftool "-datetimeoriginal<${directory;s/.*\/(.*)/$1/} 00:00:00" -r C:\Users\rober\Pictures\EXIF_tryout\1978_Daily

Worked like a champ to set datetimeoriginal in every photo file in the sandbox directory I'm experimenting in. (I currently don't see any reason to do alldates.)

One improvement I would like is a way to increment the time between each file - say something like 5 minutes - in order to definitely preserve proper sorting by date. I would appreciate some specific help on that. Perhaps some more Perl regex wizardry embedded in the command line? (Related - I'm also wondering if I can rely on the photo files being processed/tagged in "Sort by Name" Ascending order? I believe there's an option that I maybe should add to ensure that?)

There's going to be a Stage 3 as well, which will be to merge the modified files back into my "master files" which is where I'm doing ACDSee edits to them. I guess I didn't mention that I plan to perform Stage 1 and 2 on a copy of those. There's some potential exiftool relevance there which I won't dwell on beyond saying that it may be desirable to copy the modified time tags from copy to master files (by the same name) rather than the whole file. I'm pretty certain that's in exiftool's repertoire.

By the way, my photo file (JPG) naming convention is: YYYY-MM-RRR-NN where MM is the month of the first pic on the roll, RRR is a numeric sequential roll number (from 001 in 1972) and NN is a sequential counter "more or less" = to film negative number. I want to keep a trace path back to my "hard" files of original negatives. Since nearly all my film photos were taken with a single camera, and I have (or am retroactively) naming rolls sequentially, that gives me a single thread "sort by name" timeline (with just a few exceptions).

Bob_K

  • Newbie
  • *
  • Posts: 3
Re: Suggestions on an "efficient" way to date ~ 10,000 scanned photos?
« Reply #4 on: March 14, 2019, 04:29:26 AM »
Relative to my desire to have the directory name-derived datetimeoriginal tag increment from file to file - I found a command line for that in the forum that works for me. (After I figured out I had to change the single quotes to double quotes)

C:\Users\rober>exiftool "-datetimeoriginal+<0:$filesequence" -r C:\Users\rober\Pictures\EXIF_tryout\1978_Daily

Result is it added 1, 2, 3, 4, etc  minutes to each file after the 1st across the entire main YYYY directory.

I would prefer it to reset to 0 at the beginning of each subdirectory (day). As is, it will roll time over into the next day if I ever run it on a directory with more than 1440 (24 hrs x 60 min) files in it. I think I can live with that as long as I'm careful to run it on only a single year at a time, and run it only once per that year. A more refined solution would be welcome. 

Partial Results:

C:\Users\rober>exiftool -datetimeoriginal -r C:\Users\rober\Pictures\EXIF_tryout\1978_Daily
======== C:/Users/rober/Pictures/EXIF_tryout/1978_Daily/1978-01-01/1978-02-P52-01.JPG
Date/Time Original              : 1978:01:01 00:00:00
======== C:/Users/rober/Pictures/EXIF_tryout/1978_Daily/1978-01-02/1978-02-P52-01.JPG
Date/Time Original              : 1978:01:02 00:01:00
======== C:/Users/rober/Pictures/EXIF_tryout/1978_Daily/1978-01-02/1978-02-P52-02.JPG
Date/Time Original              : 1978:01:02 00:02:00
======== C:/Users/rober/Pictures/EXIF_tryout/1978_Daily/1978-01-02/1978-02-P52-03.JPG
Date/Time Original              : 1978:01:02 00:03:00
======== C:/Users/rober/Pictures/EXIF_tryout/1978_Daily/1978-01-02/1978-02-P52-04.JPG
Date/Time Original              : 1978:01:02 00:04:00
======== C:/Users/rober/Pictures/EXIF_tryout/1978_Daily/1978-01-02/1978-02-P52-05.JPG
Date/Time Original              : 1978:01:02 00:05:00
======== C:/Users/rober/Pictures/EXIF_tryout/1978_Daily/1978-01-02/1978-02-P52-06.JPG
Date/Time Original              : 1978:01:02 00:06:00
======== C:/Users/rober/Pictures/EXIF_tryout/1978_Daily/1978-01-02/1978-02-P52-07.JPG
Date/Time Original              : 1978:01:02 00:07:00
======== C:/Users/rober/Pictures/EXIF_tryout/1978_Daily/1978-01-02/1978-02-P52-08.JPG
Date/Time Original              : 1978:01:02 00:08:00
======== C:/Users/rober/Pictures/EXIF_tryout/1978_Daily/1978-01-02/1978-02-P52-09.JPG

Phil Harvey

  • ExifTool Author
  • Administrator
  • ExifTool Freak
  • *****
  • Posts: 14759
    • ExifTool Home Page
Re: Suggestions on an "efficient" way to date ~ 10,000 scanned photos?
« Reply #5 on: March 14, 2019, 07:15:23 AM »
Hi Bob,

In Stage 2, I'm using a trivial variation of a nifty command Phil provided in one of the posts further down in these Newbies posts:
      exiftool "-datetimeoriginal<${directory;s/.*\/(.*)/$1/} 00:00:00" -r C:\Users\rober\Pictures\EXIF_tryout\1978_Daily

Worked like a champ to set datetimeoriginal in every photo file in the sandbox directory I'm experimenting in. (I currently don't see any reason to do alldates.)

One improvement I would like is a way to increment the time between each file - say something like 5 minutes

You can do this after you run the above command:

exiftool "-datetimeoriginal+<0:${filesequence;$_*=5}" DIR

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

StarGeek

  • Global Moderator
  • ExifTool Freak
  • *****
  • Posts: 2516
Re: Suggestions on an "efficient" way to date ~ 10,000 scanned photos?
« Reply #6 on: March 14, 2019, 11:36:56 AM »
Thanks for your input. I will look into iMatch that StarGeek suggested.

iMatch is expensive, but there are low cost/free alternatives out there if you feel like looking into it.  But iMatch is very good at what it does and it uses exiftool on the back end to read/write metadata.  But you mention ACDSee, which should cover you. 

Quote
(Related - I'm also wondering if I can rely on the photo files being processed/tagged in "Sort by Name" Ascending order? I believe there's an option that I maybe should add to ensure that?)

For the most part, file names do get passed to exiftool in alphabetical order for most file systems, in my experience.  Here's a post by Phil on the issue.  But on Windows, if you have a drive pool setup, such as StableBit DrivePool, then you can't depend upon the order.  In those cases you can use the -FileOrder option with Filename as the argument.

Quote
By the way, my photo file (JPG) naming convention is: YYYY-MM-RRR-NN where MM is the month of the first pic on the roll, RRR is a numeric sequential roll number (from 001 in 1972) and NN is a sequential counter "more or less" = to film negative number. I want to keep a trace path back to my "hard" files of original negatives.

Good system, IMO.  I'm only done a limited amount scanning the piles of negatives from my families old photo collection, but I match the images to the glassine envelopes (left over from my stamp collecting days) that hold the negatives.  I did scan four carousels of slides and did the same, matching the file to the carousel number/slot number. 
Troubleshooting hints:
* When posting, include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).
* Double all percent signs (%) in a Windows batch file.
* If your GPS coords are negative, make sure and set the GpsLatitudeRef and GpsLongitudeRef tags correctly.