Digital Photography and Linux: my Workflow

Este artículo también está disponible en Español: Fotografía digital y Linux: flujo de trabajo.

In this article, I would like to share with you the workflow I have developed, to handle the photos I make using Linux and free software exclusively.

Preparing

I do not follow any specific routine, not even formatting the cards before I start to make photos (except those times when I go on a several day trip). Probably, the only aspect worth mentioning is that I always shot RAW, never JPEG.

Unloading

I use gPhoto2 to unload the photos: when I connect the camera to the computer (I have used a card reader in the past, but it has been doing weird things lately) a pop-up window appears, where I can select which photos will be downloaded, and then the program makes all the job. gPhoto2 downloads the pictures to a folder inside "/home/eperez/Pictures" named something like "2009-07-27--20:34:45" (using the date and time when the download is performed).

I use these folders as workspaces, and this way I have all the files belonging to a "session" inside the same folder; it is very convenient when I need to locate pending work quickly, and to know how long it has been waiting for me.

Browsing

Without modifications, Nautilus (GNOME's file manager) cannot show thumbnails for RAW files; and that is a serious problem for me, because handling the folders without seeing the pictures can be quite difficult. But there is guide at "GNOME integration of digital camera RAW format MINI HOWTO", which explains how to solve this issue.

Sometimes, I may need to export the photos quickly, perhaps because I want to show them to somebody, or send them by email. In this case, I just extract the embedded JPEG image that each RAW file contains. I have the following script (associated to Nautilus's context menu), which creates a folder named EXP, and exports there a reduced version of each RAW file:

#! /bin/sh

SELF=`basename $0`

DIR="$1"
PRE="$DIR"/"PRE"

echo "$SELF: `date`"

echo "$SELF: Options:"
echo "$SELF:  Input  = $DIR"
echo "$SELF:  Output = $PRE"

echo "$SELF: Creating output directory:"
mkdir "$PRE"

echo "$SELF: Extracting JPGs:"
ufraw-batch --out-type=jpeg --embedded-image --overwrite --out-path="$PRE" "$DIR"/IMG_????.CR2 "$DIR"/img_????.cr2

Sorting

I tend to experiment and play a lot, and I frequently get home with several panos and HDRs in the memory card. I prefer to keep each batch of photographs inside an different folder, to handle them separately. I have another script (associated to Nautilus's context menu, too), that creates a new sub-folder (named "000", "001", ...) and moves the selected files there:

#! /bin/sh

SELF=`basename $0`

DIR="$1"
shift

echo "$SELF: `date`"

for i in 0 1 2 3 4 5 6 7 8 9; do
 for j in 0 1 2 3 4 5 6 7 8 9; do
  for k in 0 1 2 3 4 5 6 7 8 9; do
   SUB="$DIR"/$i$j$k
   if [ ! -e "$SUB" ]; then
    break 3
   fi
  done
 done
done

echo "$SELF: Options:"
echo "$SELF:  Directory  = $DIR"
echo "$SELF:  Subdir     = $SUB"

echo "$SELF: Creating output directory"
mkdir "$SUB"

echo "$SELF: Moving files"
while [ "$1" != "" ]; do
 FILE="$1"

 echo "$SELF:  Moving file '$FILE'"
 mv "$DIR"/"$FILE" "$SUB"/

 shift
done

Selecting

Now comes the selection of the photographs that I will keep: I delete the duplicates and all those that are of no use to me (out of focus, motion-blurred, ...). I use Geeqie, for this task, as I can quickly browse the pictures and delete the bad ones, or compare two similar photographs and keep the best one (very useful when I shot bursts, for example).

Processing

Once I have a clean folder, it is time for the processing. I have tried several packages, and the one that bests fits my needs is RawTherapee; the final result is really good, and it is very fast when dealing with lots of photographs, because you can copy-and-paste parameters from one file on to another. With my old computer I edited first all the files, and then made a batch export; but now I have a faster computer, and I can edit and export at the same time. Anyhow, the outcome will be a folder with all the files in JPEG format.

Before going on to the next step, I usually do a last review of the files: using Eye of GNOME I just open the folder as a sideshow and watch each file full-screen; basically, I try to find serious processing errors.

Retouching

Some photographs deserve more attention, perhaps because I need to correct a small flaw, or because I want to play a little more with them. I do not export those files to JPEG but to lossless TIFF. For the retouching, I mostly use GIMP, with some plug-ins and brushes I have collected.

Panoramas

Next come the panoramas. I use Hugin to do the stitching; but first I need to convert the RAW files to a format that the program can handle, like 16bit TIFF. In this case, I just need a direct conversion, without any adjustment, because I will do that after the stitching. The following script takes care of all the job; it creates a new folder named EXP, and makes the export there, using UFRaw:

#! /bin/sh

SELF=`basename $0`

DIR="$1"
EXP="$DIR"/"EXP"

echo "$SELF: `date`"

echo "$SELF: Options:"
echo "$SELF:  Input  = $DIR"
echo "$SELF:  Output = $EXP"

echo "$SELF: Creating output directory:"
mkdir "$EXP"

echo "$SELF: Extracting TIFFs:"
ufraw-batch --gamma=0.45 --linearity=0.10 --exposure=0.0 --saturation=1.0 --wb=camera --out-type=tiff --out-depth=16 --overwrite --out-path="$EXP" "$DIR"/IMG_????.CR2 "$DIR"/img_????.cr2

Those TIFF files can be opened with Hugin now; then I make the stitching and export the result as a new 16bit TIFF file; I open that file with RawTherapee, make the adjustments of color, curves, sharpness, etc., ..., and then I export the result as a final JPEG file.

HDRs

I explained my workflow with HDRs at "HDR and Linux": basically, I just need to execute a script (you can find about it at "Script to make HDRs with Linux") and then I can make the adjustments with GIMP using layers; finally, I export the result as a JPEG file.

HDR Panoramas

Sometimes I make HDR panoramas, taking several exposures at each camera location. With these, I fist make an independent panorama for each exposure, and later produce the HDR from these panoramas. This procedure is explained at "HDR night panorama from Barcelona".

Storing

Once I have all the photos from the same event processed, I store them using this schema:

• Inside my "Pictures" folder I have a folder for each year, and inside that another folder for each event; I use a sequential number and the name of the event, like "/home/eperez/Pictures/2009/19-Cerdanya".
• I store all the final JPEG files there, a folder with the RAW originals and the PP2 side-cars used by RawTherapee, and another folder with the XCF files from GIMP (on the retouched photos); I tend to discard all intermediate files used for panoramas and HDRs, because those take lots of space and are relatively easy to recreate.
• The JPEG files get renamed (using pyRenamer) to "nnn_mmm.jpg", where "nnn" is a sequential number different for each event, and "mmm" is the number of the file within the event; RAWs and the associated PP2s keep their original filenames, while XCFs have the same name than their corresponding RAW.

Finally, to be able to find the folder that contains each file, I keep another folder named "/home/eperez/Pictures/0000", where I make symbolic links to each event folder, using the global number of the event; for example, "/home/eperez/Pictures/0000/090" points to "/home/eperez/Pictures/2009/19-Cerdanya".

Cataloguing

Currently, I still do not tag or catalogue the photographs, because the volume of files that I have is not big enough to need it. But sometimes I take too much time finding a particular photo, so I have been trying some solutions lately: F-Spot, digiKam, and Picasa; while I still have not taken a decision, the last one seems to meet my needs better.

2009/09/21: Vincent Tassy has posted a nice alternative to my sorting step: he has a script that automatically detects bracketed files and moves them to a separate folder. Find it at Linux script to automatically sort bracketed images.

Read more...

Aging a portrait with GIMP

Este artículo también está disponible en Español: Como envejecer un retrato.

There are countless articles available on Internet, explaining how to retouch a portrait and make somebody look younger; basically, what they do is soften the skin and remove wrinkles and other skin defects. In this article I will explain the method I employ to produce exactly the opposite result: aging a portrait. This process enhances the contrast of a photo and highlights the textures; but when applied to a portrait, the effect is that of aging the 'victim'.

Base material

This is the base image, as it left the camera with the standard settings:

Fist I make the conversion from RAW to TIFF, using RawTherapee. I try to mantain as much detail as possible, avoiding under and over exposed zones. The result does not differs a lot from the ofiginal image, I just adjusted the curves, and removed some noise:

Blue channel

Next, I open the file with GIMP. Retouching with portraits, I have noticed that the blue channel tends to be the most interesting one, because it is where the skin defects are more visible:

Consequently, my fist step is duplicate the blue channel and add it as a new layer: open the channels dialog and drag the blue channel over the image window; now I select overlay as the mode for the newly created layer:

High-pass filter

To enhance the contrasts even more, I use a high-pass filter to the new layer, using a relatively large radious (around 32 pixels):

This layer must have an overlay mode, too:

Selective lighting

Our image has the problem that some zones are too dark now; in this example, the tint from the glasses is clearly way too dark, the eyes can hardly be seen. To solve this, I add a new transparent layer, in overlay mode again, and paint with a white brush over the zones that I need to brighten:

The eyes are still a bit too dark for my tastes; and I usually light up the eyes on all my portraits, anyhow. I repeat the previous step: add a new transparent layer, in overlay mode, and paint with a white brush over the eyes:

Corrections and retouching

Now comes the retouching of the hair: I add a new transparent layer, this time in normal mode, and using the clone tool I delete those rebel hairs:

Conversion and exporting

Finally, I export the image to a TIFF file, and open it with RawTherapee again; after some cropping, desaturation, curves, and local contrast enhancement, the final result is:

Because of some strange reason I fail to understand, the main character in this portrait is not fully convinced with the ending results...

Read more...

Painting with light

Este artículo también está disponible en Español: Pintar con luz.

I had read on Internet about a photographic technique called 'painting with light', which can produce some amazing results. As I waited for the right opportunity to test it myself, a flower bouquet appeared at home, and presented itself as the perfect victim. A flashlight, a piece of paper, and some time later here is the result of my humble lighting experiment.

Preparing

First of all, we need a dark place with ample space. It must be dark because we are going to do long exposures, and we do not want to get the background into the final photo; and it must be large enough so we can work comfortably, and not illuminate any surrounding wall or piece of furniture. In this case, I waited until the night hours, closed the drapers at my dinning room, and moved the furniture away from the center of the room.

Then we need a continuous source of concentrated light. Sounds complex, but in fact we just need to get a normal flashlight and put a paper cone in front of it, in such a manner that we get a small beam of light. A LED flashlight would probably make the cone unnecessary, because those tend to have a concentrated beam of light, but I must confess I never tried one.

Finally, we put the camera on a tripod, place the flowers in front of it, and compose the shot.

Executing

We set the camera at the manual exposure mode, select a relatively closed diaphragm to get a good depth of field (like f/8), and put the sensibility down to ISO100. With the room lights still on, we focus on the zone that we are going to photograph, and then we put the camera in manual focus mode (later, while we make the photos the lights will be off, and the camera will not be able to focus automatically).

To control the shutter, I selected the bulb ("B") mode and used a remote trigger. But for this kind of experiment we just need a very long enough exposure, like 30 seconds; we just need to keep the shutter open long enough to lighten our flowers adequately.

Now that everything is ready, we turn the lights off, open the shutter, and then we use the flashlight to 'paint' one of the flowers. We must move the flashlight as much as possible, illuminating all the parts of the flower. And we need to move the flashlight around the flower too, so the light gets to it from different angles. Do this for several seconds, then close the shutter and check the result.

Probably, the result will not be good on the first try. Sometimes I spent too much time lighting one zone, and then the photo was overexposed; and sometimes I put the flashlight too close, and the paper cone could be seen on the photo. But with a some patience, we can achieve a good result, just keep trying.

Processing

All photos shown in this article are almost as I got them out of the camera. I always shot RAW, and obviously there where some adjustments made during the conversion to JPEG, but I mostly touched the contrast and saturation, nothing else. All the effect was produced during the shot, there are no post-processing tricks.

I worked on the following photo a bit more: with the camera fixed, I made three exposures, illuminating one flower on each exposure; later, I merged them together using layers.

Concluding

With little more that a normal flashlight and a paper cone, we can produce a fun lighting effect. Sometimes, there is no need to spend large quantities of money in costly lighting equipment: a little bit of imagination and some patience may be enough.

Read more...

Script to make HDRs with Linux

Este artículo también está disponible en Español: Un script para hacer HDR en Linux.

In my article HDR and Linux I explained my current work-flow to make HDRs using Linux. Still on that subject, in this article I show a script I wrote to make all that process automatic with a single call.

The source code for such script is:

#! /bin/bash

set -e

SELF=`basename $0`

DIR=.
ALIGN=N
ENFUSE=N
TONEMAP=N

echo "$SELF: `date`"

while [ "$1" != "" ]; do
 case "$1" in
  "-a")
   ALIGN=Y
   ;;
  "-e")
   ENFUSE=Y
   ;;
  "-t")
   TONEMAP=Y
   ;;
  "-d")
   DIR="$2"
   shift
   ;;
  *)
   echo "$SELF: Parameters"
   echo "$SELF:  -d      -> Directory"
   echo "$SELF:  -a      -> Align images"
   echo "$SELF:  -e      -> Create enfuse"
   echo "$SELF:  -t      -> Create and tonemap HDR"
   exit
   ;;
 esac

 shift
done

HDR="$DIR"/HDR
FILES=("$DIR"/*.[Cc][Rr]2)
COUNT=${#FILES[@]}

echo "$SELF: Options:"
echo "$SELF:  Directory  = $DIR"
echo "$SELF:  Output     = $HDR"
echo "$SELF:  Align      = $ALIGN"
echo "$SELF:  Enfuse     = $ENFUSE"
echo "$SELF:  Tonemap    = $TONEMAP"
echo "$SELF:  Files      = ${FILES[*]}"

if [ $COUNT -eq 0 ]; then
 echo "$SELF: No files found!!!"
 exit 1
fi

echo "$SELF: Creating output directory:"
mkdir "$HDR"

echo "$SELF: Parsing EXIF information:"
dcraw2hdrgen ${FILES[*]} > "$HDR"/pfs_raw.hdrgen

echo "$SELF: Generating TIFFs"
ufraw-batch --wb=camera --gamma=0.45 --linearity=0.10 --exposure=0.0 --saturation=1.0 --out-type=tiff --out-depth=16 --overwrite --out-path="$HDR" ${FILES[*]}

FILES=("$HDR"/*.tif)

if [ "$ALIGN" = "Y" -a $COUNT -gt 1 ]; then
 echo "$SELF: Aligning images"
 align_image_stack -a "$HDR"/AIS_ ${FILES[*]} > "$HDR"/align_image_stack.log
 rm -f ${FILES[*]}
 FILES=("$HDR"/AIS_*.tif)
fi

if [ "$ENFUSE" = "Y" ]; then
 if [ $COUNT -gt 1 ]; then
  echo "$SELF: Generating enfuse:"
  enfuse -o "$HDR"/enfuse.tif ${FILES[*]} > "$HDR"/enfuse.log
 else
  echo "$SELF: By-passing enfuse:"
  ln ${FILES[*]} "$HDR"/enfuse.tif
 fi
fi

if [ "$TONEMAP" = "Y" ]; then
 echo "$SELF: Parsing hdrgen"
 i=0
 > "$HDR"/pfs_tif.hdrgen
 cat "$HDR"/pfs_raw.hdrgen | while read LINE; do
  echo "${FILES[$i]} $LINE" | cut -d' ' -f1,3- >> "$HDR"/pfs_tif.hdrgen
  i=`expr $i + 1`
 done

 echo "$SELF: Generating HDR:"
 pfsinhdrgen "$HDR"/pfs_tif.hdrgen | pfsout "$HDR"/pfs.hdr

 echo "$SELF: Tone-mapping HDR:"

 echo "$SELF:  Operator = mantiuk06"
 pfsin "$HDR"/pfs.hdr | pfstmo_mantiuk06 -e 1 -s 1 | pfsgamma -g 2.2 | pfsout "$HDR"/pfstmo_mantiuk06.jpg

 echo "$SELF:  Operator = fattal02"
 pfsin "$HDR"/pfs.hdr | pfstmo_fattal02 -s 1 | pfsout "$HDR"/pfstmo_fattal02.jpg
fi

echo "$SELF: Cleanning:"
rm -f ${FILES[*]}

This script was made to be used with the RAW files generated by a Canon 400D camera, but I think it can be easily adapted to other cameras. Specifically, at "FILES=("$DIR"/*.[Cc][Rr]2)" we are selecting files with "CR2" extension, used by Canon (Nikon generates "NEF" files, by example); and the parameters "--gamma=0.45 --linearity=0.10" passed to "ufraw-batch" are specific of a 400D. I think that the rest should be generic.

First of all, we must download that code to our computer and save in into a file that we can execute. I suppose that the easiest way is to enter as the "root" superuser and create a file named "hdr.sh" inside the "/usr/local/bin/" directory. Then select and copy the code from this page, paste it into the file, and save it; finally, give execute permissions to everyone, with "chmod +x /usr/local/bin/hdr.sh" from the command line.

To use this script, we must first unload all the exposures from the same scene into an empty directory. Then execute the script with "hdr.sh -d _directory_ -a -e -t". We wait patiently for a while, and when the script ends there will be three new files inside our directory: "enfuse.tif", "pfstmo_mantiuk06.jpg", and "pfstmo_fattal02.jpg".

The "-a" parameter aligns the images; if the photos where made using a tripod, and we are completely sure that they are perfectly aligned, we can drop it and the script will be faster. With "-e" we indicate that an enfuse version must be generated, and we can drop it too; moreover, without "-t" the tone-mapped version will not be produced.

2009/07/27: I have updated this script, to enhance the tonemapping step, using parameters that produce a better result; it can also produce HDRs from a single shot.

2009/09/21: Vincent Tassy has posted an enhancement to this script; his version automatically creates the GIMP file at the end of the process (that is a nice addition), and can also do HDRs from JPEG files (I always shot RAW, never JPEG). Go there and grab his version, it is better than mine: Linux script to generate HDR images from bracketed images.

Read more...

Circuit to trigger a camera using a mobile phone, PDA, or MP3 player

Este artículo también está disponible en Español: Circuito para controlar la cámara desde un movil, PDA, o MP3.

On night photography, long exposures are the norm: sometimes several seconds, and sometimes several minutes; I also tend to employ HDR techniques, and that means that I often end making series of nine or even more photos all together. Spending half an hour (sometimes more) with the remote trigger on your hand can be quite boring, so I toyed with the idea of making this process authomatic... and I found a way to control the camera from a mobile phone, a PDA, or even an MP3 player.

The problem

One of my favorite 'recipes' stats with a one-second exposure, and then I double the time on each take, up to a final four-minute exposure; but, when the scene is specially dark, it may require more exposures, up to thirty seconds.

My problem is that cameras are generally not ready for this type of work: it is uncommon to find cameras capable of doing exposures longer than 30 seconds, and automated sequences of nine exposures are quite rare too. There are commercially available intervalometers, sure; but they are expensive, and I have never heard of one capable of doing this type of series.

The idea

So I began wandering on the idea of building one myself, as cheap as possible. My camera (a Canon 400D) has a specific connector for remote triggers, which can be used to release the shutter quite easily; in fact, there is a myriad of schemes on Internet to build your own trigger. And I came to the conclusion that the easiest way to make one would be to connect the camera to a mobile phone: this way I could control the timing by software, and have an almost infinite flexibility; all I needed was to make that connection possible.

And I figured I could use the earphone output from the mobile telephone: using the circuit shown here, it is possible to connect the audio output from the gadget to the remote trigger input from the camera. Basically, we just need to detect when there is a sound on the output, and simulate a press on the trigger release button while that sound lasts. Then, using a small JavaME application we could produce beeps of controlled length, and those beeps would control the shutter on the camera.

The circuit

WARNING: I must warn readers that I am not a professional of electronics; I cannot guarantee that this circuit will not fail, or that it will not break your camera, I cannot event guarantee that it will not produce alopecia: it works for me, and I tried to make it completely safe both for the camera and the mobile phone. But I cannot take any responsibility.

The output form the sound source enters through the connector at the left. C1 is just a decoupling capacitor. Then, the signal is amplified at Q1 and filtered by C2. IC1 is used as a comparator, and R4 is there to adjust the circuit to the signal level at the input (usually, it can be left at the intermediate position). D1 is an activity indicator LED, which should go off when the circuit releases the shutter. Finally, IC2 is an opto-isolator, and at the other side we have the connectors that go to the camera; in my case, I used a 2.5mm stereo JACK.

Conclusion

As can be seen, the circuit can be used with a 5v power source, because that is what I had available at the moment; but my intention is to adapt it to use a 9v battery: I do not expect having to change more that a couple of resistors. In fact, I think that none of the components has a critical value, I made this with what I had lying around. Soon I will publish the modified version, along with photos of the real construction of the device; I will also publish the software used to drive the camera.

Considering that the input to this circuit is the earphone output from the mobile phone, it could be used with almost any PDA, instead of the mobile phone. In fact, it could also be used with an MP3 player, if we previously record a sequence of beeps on a computer, and we put that 'song' into the player; then, reproducing that song with the player connected to the camera through this circuit, we would be triggering the camera at the rhythm of the sequence of beeps that we generated with our computer.

Links

• Homebrew Wired Remote for Canon EOS 350D Camera: Good instructions to build your own basic remote trigger.
• HDR-Jack - The ultra-compact interval and HDR trigger: A programmable trigger, using a micro-controller.
• Long-exposure night HDR photography with arduino (aka bulb mode bracketing): Another remote trigger project, with screen.

Read more...