Shots for and from the videopodcast "Meet the Gimp!".

Linux workflow/renaming

Joseph   December 05, 2007, 12:22 AM

I'm using Ubuntu 7.10 and we're averaging around 6,000-7,000 pictures a year. I'm interested in how everyone handles 1.) download, 2.) backup, and 3.) archiving their photos.
Here is how I am doing it presently;
1.) I just plug in the USB and drag the files to a temporary folder.
2.) First I rename the photos using KRename (I use the JPEG EXIF data to give me the date and time in the format YYYYMMDD-HH:MM:SS am/pm), then I burn a DVD of all the photos for backup. problem a.) the time is in 12 hour format so the pictures are out of order, problem b.) the name has a colon in it which FAT32 filesystems (like USB sticks) don't like.
3.) Then I place the files in a folder according to Year and a sub folder according to Month.

Does anyone use F-Spot? I tried a PERL script to rename the files, but if any were taken the same second the duplicates would disappear (get deleted). I do a manual regex replacement of the colon now, but I'm looking for something more "automagic"...

 
Peter Russell   December 05, 2007, 03:30 AM

Hi Joseph

I suggest you have a look at "digikam" You can install it through ubuntu's "synaptic package manager". It can handle renaming, tags, and most other things you can to throw at it.

Regards Pete

 
60mls   December 05, 2007, 09:24 AM

Joseph

I use f-spot at the moment and I'm quiet liking the experience. When I was a KDE user I used digikam and it took me a little while to get use to F-Spot. F-Spot also supports tags and file handling along with rudimentary editing and exporting to flickr and 23hq.

An applet (not sure what at the moment) pops up when I plug in the camera - it creates a directory based on date/time and plonks the files into that directory. I then do a search in F-Spot to load them into the catalogue.

Photos are stored on an external raid drive (http://www.thecus.com/products_over.php?cid=10&pid=3) with mirroring enabled as a safety feature in case one of the HDs packs it in. I periodically tar the files onto another storage device.

Cheers
Ed

 
Joseph   December 05, 2007, 12:15 PM

digiKam looks like a move in the right direction: using "Tools->Batch Process->Rename Images..." I can put this format on the filenames: "%Y%m%d-%H-%M-%S"! That works for the USB sticks and it remembers that I want that format in the dialog. I'll play around some more, thanks for the tips.

 
Ziggen   December 05, 2007, 12:54 PM

picasa is also possible

 
Rolf Steinort   December 05, 2007, 09:36 PM

This is a great thread. Please continue it until there is a usable manuscript. ;-)

Definitely a show theme - and one where I have no solution. Up to now I have no better system than Year/Month/day folders. I need to look into these tools - give me some advice!

 
amigapc   December 05, 2007, 11:24 PM

Picasa is my choice!

 
cosmonautirussi   December 05, 2007, 11:46 PM

mine is digikam... (I really love it's possibilities!)

 
Joseph   December 06, 2007, 12:47 AM

I liked the way that Picasa works, but I've already crashed it three times (maybe too many pictures to handle). I think may use wine?

 
kristjan.siimson   December 06, 2007, 05:16 AM

I do not have much to say, but here is my two cents to the topic:

1) Picasa does indeed use Wine. Google hired a team of programmers to make Wine handle Picasa.

2) Mentioning workflow, I have recently improved my productivity by making a good use of virtual desktops. I have changed the numbers into text, so I know where things are at. Not anything to say about photography though.

 
Peter Russell   December 06, 2007, 05:32 AM

I use the format for cataloging my photos as Rolf suggested. Using folders like this "original/2007/05_23" in combination with "digikam" Then I backed up using "synctoy" in windows. Since I'm almost exclusively using linux now has anyone found anything as good as synctoy for linux?

Regards Pete

 
kristjan.siimson   December 06, 2007, 05:59 AM
 
Joseph   December 06, 2007, 11:59 AM

I'm thinking about using this for backup: http://rsync.net/

 
mitu   December 06, 2007, 01:13 PM

Hi all,

In my laptop, I use UbuntuStudio[1] distro, based on Ubuntu 7.10. It is designed for audio/video and graphics editing.

I also have windows xp installed on another partition since i need to use it for my work sometimes.

And I have another partition formatted as vfat where i keep my photos/documents/music and so. Since it is formatted as vfat it is accessable both from winxp and linux.

I use f-spot to transfer the pictures via usb and to keep them organized. f-spot is configured to store the pics in a directory which is actually above mentioned vfat partition.

f-spot is not yet mature and functional enough for me. One of the main reasons that I use is is that it stores the pictures in year/month/day hierarchy, it has tag support, in the context menu I have options to open the photo with gimp/ufraw etc. and it supports displaying raw images (i use canon .cr2)

But i never use f-spot to touch my photos (we have gimp : ), instead i use it as an organiser and honestly i will change it whenever i find a replacement.

I also recommend you to use GPL tool gphoto2 [2] to transfer your files from the camera. gphoto2 supports more than 900 cameras at the moment.

gphoto2 is a console tool (there might be some graphical interfaces but the core is gphoto2 and i like to use it in scripts, i'll explain how)

simple usage from the command line:

AUTO DETECT:
$ gphoto2 --auto-detect
Model Port
----------------------------------------------------------
PTP/IP Camera usb:

CHECK ABILITIES:
$ gphoto2 -a
Abilities for camera : PTP/IP Camera
Serial port support : no
USB support : no
Capture choices :Capture notsupported by the driver
Configuration support : yes
Delete selected files on camera : yes
Delete all files on camera : no
File preview (thumbnail) support : yes
File upload support : yes

LIST FILES:
$ gphoto2 -L
There is no file in folder '/'.
There is no file in folder '/store_00010001'.
There is no file in folder '/store_00010001/DCIM'.
There are 15 files in folder '/store_00010001/DCIM/100CANON'.
#1 IMG_8906.JPG 2501 KB 3888x2592 image/jpeg
#2 IMG_8907.JPG 2450 KB 3888x2592 image/jpeg

and GET ALL FILES:
$ gphoto2 -P
Downloading 'IMG_8906.JPG' from folder /store_00010001/DCIM/100CANON'...
Saving file as IMG_8906.JPG
Downloading 'IMG_8907.JPG' from folder /store_00010001/DCIM/100CANON'...
Saving file as IMG_8907.JPG

this is the basic usage, but much more functionality can be added by a simple shell script:

8<---------------8<------------8<---------------
#!/bin/sh

# Photo transfer script using gphoto2
# by humit@tr.net Thu Dec 6 15:09:15 UTC 2007

# Base dir for the pics
base="/tmp/pics"

# Year/month and date
year=`date +%Y`
month=`date +%m`
day=`date +%d`

#create dir i.e /tmp/pics/2007/12/06
mkdir -p $base/$year/$month/$date

#chdir transfer directory
cd $base/$year/$month/$date

# run gphoto2 and get all files into current dir
gphoto2 -P

echo Your photos are transferred into $base/$year/$month/$date
8<---------------8<------------8<---------------

[1] http://ubuntustudio.org
[2] http://www.gphoto.org/

 
mitu   December 06, 2007, 02:50 PM

I wrote a little script to download the files from your camera and rename them in format YYYYMMDD-HH-MM-SS i checked it with my EOS 400d for both jpg and raw files and it seems to work well. you will need gphoto2 [1] and exiv2 [2] programs to run the script.

READ the script before running. and first test it with a test base dir. /tmp/pics could be a good place. Hope it works for you.

[1] http://www.gphoto.org/
[2] http://www.exiv2.org/
-----------------------------------------------------------------------
#!/bin/sh

# Photo transfer&rename script using gphoto2
# by humit@tr.net Thu Dec 6 16:46:59 EET 2007

#-- C O N F I G U R A T I O N --#

exifprog="exiv2" # COMMAND TO READ EXIF DATA
base="/tmp/pics" # BASE DIR FOR FILES


#-- PROCEDURES --#

exphrase="Image timestamp" # date string in exiv2
# SET Year/month and date
year=`date +%Y`
month=`date +%m`
day=`date +%d`

mydir="$base/$year/$month/$date"
#create mydir i.e /pics/2007/12/06
mkdir -p $mydir

#chdir transfer directory
cd $mydir

# run gphoto2 and get all files into current dir
gphoto2 -P

echo Your photos are transferred into $base/$year/$month/$date

for i in `ls -1 $mydir`
do
# get exif data and extract creation date
datum=`$exifprog $i |grep "$exphrase"|awk '{print $4}'|sed s/://g`
timum=`$exifprog $i |grep "$exphrase"|awk '{print $5}'|sed s/:/-/g`
extum=`echo $i|awk -F. '{print $2}'`
mv $i $datum"-"$timum.$extum
done
-----------------------------------------------------------------------

 
Peter Russell   December 06, 2007, 06:10 PM

I did try unison and the interface etc looks quite good. However it was choking on my fat and ntfs partitions. Saying something about not being able to change permissions then fails.(huh!! I don't want it to change permissions).

Anyway I tried rsync using the Grsync interface and this works well backing up from fat disk to ntfs disk. I turned off the preserve options etc; to avoid getting errors and it seemed to work well. Because my files were not in Linux native partition I turned on "always checksum" in the "advanced options" for some extra precaution.

Hope this helps.

Thanks regards Pete

PS for Ubuntu users all these programs can be installed in seconds using "synaptic package manager"

 
Joseph   December 07, 2007, 11:08 AM

Mitu, thanks for the script! You may need to add a counter at the end of the filename the way that digiKam does because I sometimes take multiple pictures per second. When this happens, your script will overwrite these files. I tested it with a few sample snap-shots; I had 9 pictures on the camera and only five in the folder.

I like the two tools you used though - this could come in very handy!

 
Joseph   December 07, 2007, 11:58 AM

I'm sold on digiKam now;
Under the camera menu - I can download all the files and while is does it; have it rename them in my format (there are other formats as well), have it generate the directory structure as it downloads, have it rotate the image according to the EXIV camera data...

 
mitu   December 07, 2007, 12:52 PM

Hi,

it is good that you solved the issue using digiKam, but i'd like to address the issue you pointed out. I couldn't test it but the final for loop could be modified to avoid data loss due to same file name:

samecnt=0
for i in `ls -1 $mydir`
do
# get exif data and extract creation date
datum=`$exifprog $i |grep "$exphrase"|awk '{print $4}'|sed s/://g`
timum=`$exifprog $i |grep "$exphrase"|awk '{print $5}'|sed s/:/-/g`
extum=`echo $i|awk -F. '{print $2}'`
newname=$datum"-"$timum.$extum

# see if the same filename exists
ls -1|grep -q $newname
same=$?

if [ $same -eq 0 ]
then
mv $i $datum"-"$timum"-"$samecnt.$extum
let "samecnt=$samecnt+1"
else
mv $i $newname
fi
done

 
Joseph   December 17, 2007, 01:04 AM

I was out of town so I just now got the chance to try the changes to your script. When I ran it the first time I got the following error:
"./bin/mitu-script: 52: let: not found"

So I changed the first line of the program from "#!/bin/sh" to "#!/bin/bash" and the script worked as needed! The files taken within one second were appended with the iterative counter.

Very nice scripting!

 
Jason Fuchs   December 17, 2007, 01:58 AM

jhead can perform some of this as well. I like jhead because it can also automatically rotate my images based on the exif orientation data (not all cameras support this..) You can also rename the files to their exif time/date as well. I use jhead -autorot -n *, after downloading, this could be added to your script as well..

 
mitu   December 25, 2007, 10:39 AM

Hi all,

Thanks Jason for his tip about jhead. It help to make the script much more shorter : )

One drawback is that it cannot get exif data from raw files, but for the files shot within same seconds it works like a charm. I've tested it on my pictures that have been taken for a stopmotion and it worked very well.

resulting filenames: (YYYYMMDD-HHMMSS)
20071107-130944a.jpg
20071107-130944.jpg
20071107-130945a.jpg
20071107-130945b.jpg
20071107-130945.jpg
...

I didn't use the autorotation parameter, but of course you can add it if you wish to use the script.

--------------------------------------------------
#!/bin/bash

# Photo transfer&rename script using gphoto2 & jhead
# by humit@tr.net Dec 2007

#-- C O N F I G U R A T I O N --#
exifprog="jhead" # COMMAND TO READ EXIF DATA
base="/tmp/pics" # BASE DIR FOR FILES

#-- PROCEDURES --#
# SET Year/month and date
year=`date +%Y`
month=`date +%m`
day=`date +%d`

mydir="$base/$year/$month/$date"
mkdir -p $mydir

#chdir transfer directory
cd $mydir

# run gphoto2 and get all files into current dir
gphoto2 -P

echo Renaming files according to exif data
$exifprog -n%Y%m%d-%H%M%S *

echo Your files has been transferred into $base/$year/$month/$date
echo and renamed in YYYYMMDD-HHMMSS format.

--------------------------------------------------

have fun.

 
imitis   January 02, 2008, 09:59 PM

I like this script but I'd like that it makes folder not of day i launch it but makes folder with that date the picture was taken.for exapmle.... ghpoto dowloads pictures than rename them using date then move to appropiate folder....hope you unerstood me..not too good at english and programming :D

thanx

 
mitu   January 03, 2008, 10:49 AM

Hi,

here another version of the script (by the way i name it 'sompic' for now to sound like "sort my pic"). It has two parameters -drm and -jrm

if you call it as "sompic -drm" it downloads the files from camera, places them under your base_dir/temp dir, then renames the jpeg files as YYYYMMDD-HHMMSS from exif data. Then according to the file name, moves the files under base_dir/YYYY/MM/DD tree.

Or if you have a bunch of jpeg files laying around you can go into that folder and just type "sompic -jrm" and it will rename those files again to exif date, then move'em into base_dir/YYYY/MM/DD tree.

Hope you like it ;)

-----------------------------------------------------------------------
#!/bin/bash

# Photo transfer&rename script using gphoto2 & jhead
# by humit@tr.net Dec 2007 - 2008

#-- C O N F I G U R A T I O N --#

exifprog="jhead" # COMMAND TO READ EXIF DATA
base="/tmp/pics" # BASE DIR FOR FILES

#-- PROCEDURES --#


move_it(){
year=`echo $i|cut -c 1,2,3,4`
month=`echo $i|cut -c 5,6`
day=`echo $i|cut -c 7,8`
mydir=$base/$year/$month/$day
mkdir -p $mydir
mv -i $i $mydir
}

# Download all files to a temporary folder
get_it(){
mkdir -p $base/temp
cd $base/temp
gphoto2 -P
jhead -n"%Y%m%d-%H%M%S" *
do_it
}

do_it(){
for i in `ls -1 *.jpg`
do
move_it $i
done
}


case `echo $1` in
"-drm") get_it;;
"-jrm") do_it;;
*) echo "Usage: $0 [-drm] [-jrm]"
echo "-drm Download, rename & move files in camera"
echo "-jrm Just rename & move files in current folder";;
esac

 
Joseph   January 03, 2008, 10:55 AM

Very nice work, mitu! This is why I like linux & open source: take a few general purpose tools and string them together & you have a powerful tool! Now if I could just convince some people that they don't have to be afraid of the command line...

 
mitu   January 03, 2008, 12:19 PM

the power of command line comes from the simplicity. But people think that a GUI is much easier to use. For me, typing the command "cd /usr/local/pics; sompic -jmr" is much more easier than starting some gui and clicking here and there to select the source/destination folders and naming scheme etc... for simple tasks, we need simple solutions ;)

Anyway maybe later this code can be converted into some simple X GUI program when i have time ;)

 
Jotty23   January 03, 2008, 01:34 PM

Hi,

first of all I wish everybody a happy new year!

I followed this threat but my system is quite a different one: it does not interrest me, when a picture was taken. I'm interrested in what it shows me.

So I import all photos from the storage card into an folder called "bremen" if the photos are made in my hometown. In there are some more folders like "winter", "@night" and so on.

Yes, I have to do all the work manually without using a script, but I shoot about 50 pics a month and not as much as some other people around here.

For backup-reasons I make a copy on my server HD everytime I've done some major changes.

One problem I haven't solved up to now: After converting RAW-files to jpg the exif-datas are lost. But I think there was a script posted here to read them from the raw files and to write them into the jpg files. I will have to search this posting. ;-)

 
mitu   January 03, 2008, 01:49 PM

Hi,

can I ask you how you convert your raw images into jpeg? In my case, I open raw files with gimp, and it launches the ufraw program, when i finish the ufraw, the gimp has all the exif info into the new file.

also, the program to read exif data from raw files is called exiv2 [1]

[1] http://home.arcor.de/ahuggel/exiv2/index.html

 
To participate in this conversation, you'll need to join the group




About 23

About 23
What is 23 and who's behind the service?
Just In
Discover the world from a different angle.
Here's a crop of the latest photos from the around the world.
Search
Search photos from users using 23
Help / Discussion
Get help or share your ideas to make 23 better
23 Blog / 23 on Twitter
Messages and observations from Team 23
Terms of use
What can 23 be used for and what isn't allowed
More services from 23
We also help people use photo sharing in their professional lives
RSS Feed
Subscribe to these photos in an RSS reader
  • Basque (ES)
  • Bulgarian (BG)
  • Chinese (CN)
  • Chinese (TW)
  • Danish (DK)
  • Dutch (NL)
  • English (US)
  • French (FR)
  • Galician (ES)
  • German (DE)
  • Italian (IT)
  • Norwegian (NO)
  • Polish (PL)
  • Portuguese (PT)
  • Russian (RU)
  • Spanish (ES)
  • Swedish (SE)

Popular photos right now