Digital Photography and Linux: my Workflow

Este artículo también está disponible en Español: Fotografía digital y Linux: flujo de trabajo.

In this article, I would like to share with you the workflow I have developed, to handle the photos I make using Linux and free software exclusively.

Preparing

I do not follow any specific routine, not even formatting the cards before I start to make photos (except those times when I go on a several day trip). Probably, the only aspect worth mentioning is that I always shot RAW, never JPEG.

Unloading

I use gPhoto2 to unload the photos: when I connect the camera to the computer (I have used a card reader in the past, but it has been doing weird things lately) a pop-up window appears, where I can select which photos will be downloaded, and then the program makes all the job. gPhoto2 downloads the pictures to a folder inside "/home/eperez/Pictures" named something like "2009-07-27--20:34:45" (using the date and time when the download is performed).

I use these folders as workspaces, and this way I have all the files belonging to a "session" inside the same folder; it is very convenient when I need to locate pending work quickly, and to know how long it has been waiting for me.

Browsing

Without modifications, Nautilus (GNOME's file manager) cannot show thumbnails for RAW files; and that is a serious problem for me, because handling the folders without seeing the pictures can be quite difficult. But there is guide at "GNOME integration of digital camera RAW format MINI HOWTO", which explains how to solve this issue.

Sometimes, I may need to export the photos quickly, perhaps because I want to show them to somebody, or send them by email. In this case, I just extract the embedded JPEG image that each RAW file contains. I have the following script (associated to Nautilus's context menu), which creates a folder named EXP, and exports there a reduced version of each RAW file:

#! /bin/sh

SELF=`basename $0`

DIR="$1"
PRE="$DIR"/"PRE"

echo "$SELF: `date`"

echo "$SELF: Options:"
echo "$SELF:  Input  = $DIR"
echo "$SELF:  Output = $PRE"

echo "$SELF: Creating output directory:"
mkdir "$PRE"

echo "$SELF: Extracting JPGs:"
ufraw-batch --out-type=jpeg --embedded-image --overwrite --out-path="$PRE" "$DIR"/IMG_????.CR2 "$DIR"/img_????.cr2

Sorting

I tend to experiment and play a lot, and I frequently get home with several panos and HDRs in the memory card. I prefer to keep each batch of photographs inside an different folder, to handle them separately. I have another script (associated to Nautilus's context menu, too), that creates a new sub-folder (named "000", "001", ...) and moves the selected files there:

#! /bin/sh

SELF=`basename $0`

DIR="$1"
shift

echo "$SELF: `date`"

for i in 0 1 2 3 4 5 6 7 8 9; do
 for j in 0 1 2 3 4 5 6 7 8 9; do
  for k in 0 1 2 3 4 5 6 7 8 9; do
   SUB="$DIR"/$i$j$k
   if [ ! -e "$SUB" ]; then
    break 3
   fi
  done
 done
done

echo "$SELF: Options:"
echo "$SELF:  Directory  = $DIR"
echo "$SELF:  Subdir     = $SUB"

echo "$SELF: Creating output directory"
mkdir "$SUB"

echo "$SELF: Moving files"
while [ "$1" != "" ]; do
 FILE="$1"

 echo "$SELF:  Moving file '$FILE'"
 mv "$DIR"/"$FILE" "$SUB"/

 shift
done

Selecting

Now comes the selection of the photographs that I will keep: I delete the duplicates and all those that are of no use to me (out of focus, motion-blurred, ...). I use Geeqie, for this task, as I can quickly browse the pictures and delete the bad ones, or compare two similar photographs and keep the best one (very useful when I shot bursts, for example).

Processing

Once I have a clean folder, it is time for the processing. I have tried several packages, and the one that bests fits my needs is RawTherapee; the final result is really good, and it is very fast when dealing with lots of photographs, because you can copy-and-paste parameters from one file on to another. With my old computer I edited first all the files, and then made a batch export; but now I have a faster computer, and I can edit and export at the same time. Anyhow, the outcome will be a folder with all the files in JPEG format.

Before going on to the next step, I usually do a last review of the files: using Eye of GNOME I just open the folder as a sideshow and watch each file full-screen; basically, I try to find serious processing errors.

Retouching

Some photographs deserve more attention, perhaps because I need to correct a small flaw, or because I want to play a little more with them. I do not export those files to JPEG but to lossless TIFF. For the retouching, I mostly use GIMP, with some plug-ins and brushes I have collected.

Panoramas

Next come the panoramas. I use Hugin to do the stitching; but first I need to convert the RAW files to a format that the program can handle, like 16bit TIFF. In this case, I just need a direct conversion, without any adjustment, because I will do that after the stitching. The following script takes care of all the job; it creates a new folder named EXP, and makes the export there, using UFRaw:

#! /bin/sh

SELF=`basename $0`

DIR="$1"
EXP="$DIR"/"EXP"

echo "$SELF: `date`"

echo "$SELF: Options:"
echo "$SELF:  Input  = $DIR"
echo "$SELF:  Output = $EXP"

echo "$SELF: Creating output directory:"
mkdir "$EXP"

echo "$SELF: Extracting TIFFs:"
ufraw-batch --gamma=0.45 --linearity=0.10 --exposure=0.0 --saturation=1.0 --wb=camera --out-type=tiff --out-depth=16 --overwrite --out-path="$EXP" "$DIR"/IMG_????.CR2 "$DIR"/img_????.cr2

Those TIFF files can be opened with Hugin now; then I make the stitching and export the result as a new 16bit TIFF file; I open that file with RawTherapee, make the adjustments of color, curves, sharpness, etc., ..., and then I export the result as a final JPEG file.

HDRs

I explained my workflow with HDRs at "HDR and Linux": basically, I just need to execute a script (you can find about it at "Script to make HDRs with Linux") and then I can make the adjustments with GIMP using layers; finally, I export the result as a JPEG file.

HDR Panoramas

Sometimes I make HDR panoramas, taking several exposures at each camera location. With these, I fist make an independent panorama for each exposure, and later produce the HDR from these panoramas. This procedure is explained at "HDR night panorama from Barcelona".

Storing

Once I have all the photos from the same event processed, I store them using this schema:

• Inside my "Pictures" folder I have a folder for each year, and inside that another folder for each event; I use a sequential number and the name of the event, like "/home/eperez/Pictures/2009/19-Cerdanya".
• I store all the final JPEG files there, a folder with the RAW originals and the PP2 side-cars used by RawTherapee, and another folder with the XCF files from GIMP (on the retouched photos); I tend to discard all intermediate files used for panoramas and HDRs, because those take lots of space and are relatively easy to recreate.
• The JPEG files get renamed (using pyRenamer) to "nnn_mmm.jpg", where "nnn" is a sequential number different for each event, and "mmm" is the number of the file within the event; RAWs and the associated PP2s keep their original filenames, while XCFs have the same name than their corresponding RAW.

Finally, to be able to find the folder that contains each file, I keep another folder named "/home/eperez/Pictures/0000", where I make symbolic links to each event folder, using the global number of the event; for example, "/home/eperez/Pictures/0000/090" points to "/home/eperez/Pictures/2009/19-Cerdanya".

Cataloguing

Currently, I still do not tag or catalogue the photographs, because the volume of files that I have is not big enough to need it. But sometimes I take too much time finding a particular photo, so I have been trying some solutions lately: F-Spot, digiKam, and Picasa; while I still have not taken a decision, the last one seems to meet my needs better.

2009/09/21: Vincent Tassy has posted a nice alternative to my sorting step: he has a script that automatically detects bracketed files and moves them to a separate folder. Find it at Linux script to automatically sort bracketed images.