General Category > Blogs

Linux programs and Bash scripts to consolidate photos and begin a slideshow

(1/3) > >>

Dragon:
Earlier this week, a backup drive of mine crashed... it doesn't seem to be recoverable without sending it to a specialist lab with a cleanroom. (I didn't know what a "cleanroom" was before this week, so obviously this isn't something that I'd have access to at work.) Anyway, I'm pretty sure that I've lost very little overall, if anything. It was one of 3 backup drives, but it failed the day after I had run an rsync process to copy over some files that were still missing from one of the 3.

Of course, I noticed that on my personal computer, I didn't have all the family photos that were available on the storage drive, even though I have plenty of space for them. To remedy that, I thought that now would be a good time to consolidate some photos, since they had been collected up in all different locations over the years. My personal computer is running Ubuntu Desktop 16.04 currently, while several other computers in the house over the years have primarily been Windows computers. Fortunately, two primary folder names have been used, regardless of which OS they were on - "Photos" and "Pictures".

To begin, I decided to create a text file containing all the paths with either of those names. Since I have my storage drive connected to my laptop under /mnt/toshiba, I opened up the command line and started the search there.


--- Code: ---find /mnt/toshiba -name Photos >> DirectoryList-Photos.txt
find /mnt/toshiba -name Pictures >> DirectoryList-Pictures.txt

--- End code ---

After creating those two files and verifying that the directories listed, I went on to creating the Bash script which would read through the files and, then use the paths listed on each line to copy everything to my Pictures directory on my laptop. One thing that I noticed before starting this part though, is that files with duplicate names are going to get overwritten. Since I want to due some purging of duplicates anyway, I'm not too concerned with that, but I am still going to tackle each list separately at least for partial separation between running my backups. Here is my Bash script:


--- Code: ---#!/bin/bash
# Copy files from the Photos and Pictures directories of the Toshiba backup to my Sony Vaio HD.

SOURCES=/home/dragon/DirectoryList-Photos.txt
DESTDIR=/home/dragon/Pictures/

while read LINE; do
[ -z "$LINE" ] && continue # skip blank lines
rsync -avh --exclude='.*' "$LINE/" "$DESTDIR" --log-file=/home/dragon/rsync-toshiba-to-vaio.log
done < $SOURCES

--- End code ---

Once that was done, I was able to jump into my Pictures directory and start up my slideshow with these commands:


--- Code: ---cd ~/Pictures/
eog --slide-show ./

--- End code ---

Dragon:
I found another Linux program for browsing images which looks pretty nice too... it's called gThumb Image Viewer.


--- Code: ---sudo apt-get install gthumb
--- End code ---

It has a Presentation mode, for running a slideshow, an Organization option, for grouping photos by date or other criteria (without moving them to new locations), and even a feature to Find Duplicates. Since I'm still moving files over to my laptop from my storage drive, I haven't tried all these features out yet, but I do intend to check them out soon.

One thing that I have already noticed about gThumb is that it loads up pretty quickly, much faster than the Nautilus File Manager when viewing a directory with hundreds of photos. On the flip side, it appears to have a delay detecting new files. Since I'm in the middle of copying 11 GB of images from one drive to another, I've noticed that several directories came up with a message of simply "(Empty)" even though I could navigate to the same directory through Nautilus and find plenty of files.

Dragon:
Since my local hard drive had gotten filed up quickly, I had to change where my staging directory was going to be at. I have a 2TB drive mounted at /media/dragon/BluePortableDrive/ with a folder in there called PhotoStage. All of my files were copied there, then gthumb was used to remove the duplicate files which I did after copying smaller batches of files to the PhotoStage directory. Now that I've had some time to get all the files copied into a single directory, it's time for me to put them back to my storage drive with my consolidated location.


--- Code: ---rsync -avh "/media/dragon/BluePortableDrive/PhotoStage/" "/mnt/toshiba/Photo/"
--- End code ---

In order to handle the same process simply with our other Windows computers in the house, I've looked around for some "rsync for Windows" programs, but the ones that I found were just too cumbersome for what I wanted. Options like DeltaCopy and cwRsync were a couple that I noticed, but ultimately xcopy seemed to be the answer I was looking for. After using the Windows Map Network Drive to connect "V:" to my PhotoStage folder, I ran this simple command from Windows Command Line to start copying all of the files from my PhotoStage to the current directory of that PC:


--- Code: ---xcopy V:\ *.* /e
--- End code ---

Check out http://www.computerhope.com/xcopyhlp.htm to see more about the options and examples for using Windows xcopy.

Since I have another computer on our network that was able to connect to the root of our storage drive using "T:" having the slash at the end of the directory path to Photo caused a problem finding the path, so that xcopy command was just slightly different:


--- Code: ---xcopy T:\Photo *.* /e
--- End code ---

I've also found that robocopy is an updated version of xcopy that is also built into Windows. Syntax is slightly different, but I'm trying this another computer to copy the same files:


--- Code: ---robocopy T:\Photo C:\Users\Public\Pictures /e
--- End code ---

Since robocopy shows much more, such as percentage of file transfer, if it's a new file, and file size, just with this simple command, this is probably going to be the best method for setting up my automated tasks.  I also noticed that robocopy was able to complete the task more quickly, possibly because it was a different path, but even after twice as much time, the xcopy process hadn't completed and appeared to be hung up on transferring one specific video file. Looking into the new directory, I see that the subdirectories hadn't been copied at all, which was location of a majority of photos, so I just cancelled the task and ran robocopy on that computer instead also.

Dragon:
It had been a while since I've done any photo purging since we've been using Google Photos to store new pictures, but I wanted to get back to my old storage drives. I was going through some old photos recently and gthumb was being slow. My wife came in and was making some suggestions to be helpful and we talked some about what I was looking to do. My primary concern being that it needed to group photos using the folder structure of the native operating system rather than relying solely on database records that wouldn't be recognized on another computer without the same program.

While I continued to do some cleanup, my wife discovered a program called Digikam, which is an open source photo management tool that has a Duplicate finder along with other tools. I got it installed and although it still took a while to search through all the photos that I have, the Duplicates feature is very nice. Preview images of the photos along with the path to the photos are visible during the process. I have seen in other programs where you could select something like "Delete all except one" which I have NOT yet seen in Digikam, but aside from that, I've found it to be quite user friendly.

Changing the Album that a photo was in and renaming the photos does so on the native file structure. There is also a SQLite Database that stores information locally about the images. Options to update metadata on the actual files does exist, although it seems to be discouraged since it slows things down. Considering that I wanted to keep things useful outside of this application as much as possible, having a little slower response in order to have the metadata stored on the actual photo files seems like a good trade, although I haven't yet tried updating any metadata there.

Dragon:
In the process of moving photos from one location to another, especially uploading to Google Photos, I've come across some corrupt video files. Lots of times we have videos of things that could have been photos, but not to digress from my purpose of this post, I found a nice little way to extract photos from some AVI files even though Google isn't able to upload the file in video format. In these cases, I had some videos which were able to play a little after opening the file locally, but would error out before getting to the end.

This is for use in Linux. This command will take MVI_1271.AVI, extract an image every 1 second and store it in a file called MVI_1271-01.jpeg, then MVI_1271-02.jpeg, MVI_1271-03.jpeg, and so on. Just replace MVI_1271.AVI with the video file you have, and the destination at the end of the line. Also, %2d at the end will form 2-digit suffixes while %3d will form 3-digit suffixes on your extracted image files.


--- Code: ---ffmpeg -i MVI_1271.AVI -r 1 -f image2 MVI_1271-%2d.jpeg
--- End code ---

I didn't have ffmpeg installed to start with, but it was simple to install it:


--- Code: ---sudo apt install ffmpeg
--- End code ---

Navigation

[0] Message Index

[#] Next page