Workflow workflow workflow.
A photo processing workflow is a process that you follow after you have taken the pictures. It should be straight forward and allow you spend time working on the photos, not the organization of the photos. My main goal was to create a workflow that was easy to reproduce, something I could do while traveling, and would leave me with files that I could identify and use years later with the minimal of work. All of this pre-processing is done before I actually do any editing or reviewing of pictures I’ve taken. That stage comes after doing this prep work.
Included below is the workflow I use to prep my photos for actual editing. I do a small number of things across both linux and windows boxes to prep my photos after taking them. I’ve attempted to convert all the tools used below for use in a single windows environment. Most of these tools are PERL programs, so installing a perl interpreter will allow you to run the original source. Links to all the tools are included at the bottom of the article.
A straight forward file system format was the first stage. So I keep all my images stored in individual directories, by date. A directory for every year, and a sub directory for every “shoot”. Where a shoot can be as short as a day, or as long as about a week. Within this primary shoot directory are additional subdirectories used to store different data. I create a “RAW” directory to store straight from the camera files, a “DNG” directory where I store Digital Negatives (DNG) versions ,a “JPG” directory for JPG images that come straight from the camera , a “GPS” directory to store gps tracking info, in GPX format for right now, and a “WEB” directory containing post processed ready for posting images for websites. In addition to these directories I will create a “notes.txt” file and include information about the shoot, timezones, mistakes, and general notes about the images. The end results is something that looks like this:
2009/ 2009/2009-06-02 Half Moon Bay/ 2009/2009-06-02 Half Moon Bay/DNG/ 2009/2009-06-02 Half Moon Bay/DNG/20090602-1405-32162.dng 2009/2009-06-02 Half Moon Bay/DNG/20090602-1412-32163.dng 2009/2009-06-02 Half Moon Bay/GPS/ 2009/2009-06-02 Half Moon Bay/GPS/20090602.gpx 2009/2009-06-02 Half Moon Bay/JPG/ 2009/2009-06-02 Half Moon Bay/RAW/ 2009/2009-06-02 Half Moon Bay/RAW/_MG_2162.cr2 2009/2009-06-02 Half Moon Bay/RAW/_MG_2163.cr2 2009/2009-06-02 Half Moon Bay/WEB/ 2009/2009-06-02 Half Moon Bay/WEB/20090602-1405-32162.jpg 2009/2009-06-02 Half Moon Bay/WEB/20090602-1412-32163.jpg 2009/2009-06-02 Half Moon Bay/WEB/index.html
I’ve chosen to use DNG files over RAW files, an open format for my negatives, that also supports metadata. I keep the originals in a separate directory, as opposed to embedding them, so that I have completely unaltered copies. Keeping both a DNG and a RAW doubles my storage requirements for pictures. With the current cost of storage being so low, I keep both. Keeping all your files organized is important in this situation, as having too many files in random locations is as good as not having them at all.
I also chose to use DNG so that all metadata gets embedded into the DNG file. As I add GPS info, keywords, or IPTC data, it all gets written back out into the file. This keeps all the custom data in the actual file, and not located in a XMP sidecar, or similar. I do this so that if in the future I need to switch software tools, the new tool should be able to read all the metadata out of the files directly.
So how do I do all this?
- Turn on the GPS logger, and then go out and shoot pictures.
- At the end of the day I create the primary directory I’m going to work in, with the format “YYYY/YYYY-MM-DD – Subject Info”, and create subdirectories in there for “RAW”, “DNG”, and “GPS” data.
- Download all the raw pictures from my camera, and copy them in the RAW directory. I do this with a simple USB card reader, and copy by hand. Sooner or later I’ll find something I like to automate this step.
- Convert all the RAW files to Adobe DNG format, using the GUI Abobe DNG converter, putting the resulting files into the DNG directory.
- Optionally download my GPS data, in GPX format, placing it in the GPS directory. I use a Qstarz BT1000P gps logger that tracks every 10 seconds. I leave my Camera set to GMT time and adjust everything in post processing. This helps solve the problem of remembering to update the camera clock when traveling.
- Use the gpsPhoto tool to insert basic location tags and gps coords. The command to do this:
gpsPhoto --dir . --gpsdir ../gps/ --timeoffset 0 --geoinfo geonames --overwrite-geotagged --city auto --sublocation auto --state auto --country autoThis will automatically take the GPX file from the GPS directory, write the latitude/longitude coordinates into the DNG, and then make best guesses at location, and write that into the IPTC section of the DNG. - Correct the time information in the photo to match the local time. I use the ExifTool program. In this example, I adjust for local California time from GMT in the camera with the following command:
exiftool -AllDates-=8:00 -overwrite_original *dng - Exiftool again to rename the file to something useful. I use a format that has the date and time in it:
exiftool "-filename<${CreateDate}-$filenumber.%e" -d "%Y%m%d-%H%M" *dngOn my Canon 20D, this produces files that look like this – “20090705-1202-327-2721.dng”. I run a small custom program which merges the last numbers into a single 5 digit number, and the file becomes – “20090705-1202-32721.dng”. This results in the Date-Time-Img# filename that is unique and “relatively” compact in length. Source/binary for this is provided below. - Append boilerplate metadata. Using Adobe Bridge, I apply a metadata template that has my contact and copyright data to all the DNG files. Menu item “Tools”, then “Replace Metadata”. I prebuilt a template to include my name, email address, and basic copyright.
After all this is done, we now have a directory filled with DNG files ready for processing. It takes about 10-20 minutes to do the above steps and I try to do it daily when traveling to make sure I have gps tags and locations on everything.
Later, anywhere from later that day, to weeks later, I will go through and in multiple passes cull down the pile of shots to 1/10 or similar number of shots that I want to do more changes to. This is done in simple two stage system. Initially scanning through all the pictures and marking interesting ones with one star in Bridge if they look good, or I want to edit them. This usually cuts out 50-75%, and I try to spend less than 10 seconds judging an image. The idea is to cut out all the obvious ones you are not going to spend any time on. After the first quick pass, I make a second pass just on the ones that are marked with one star. This pass is longer, and I’ll spend time playing around with an image to determine which shot I like. Photos that I decide to keep, I’ll mark with a second star. I may have done some minor work with camera raw at this point (cropping, basic levels).
Following this two pass method, I then go through and crop/level all the two star photos in camera raw or lightroom. Most of the time this is enough editing and I post the resulting images online. In my next blog entry I’ll update how I use “TravelGen” to create static travel log pages with my images.
Software:
ExifTool is the swiss army knife of metadata modifications. Download the zip file, unzip, and copy the executable into “c:\windows”.
gpsPhoto. This is optional and allows you to add GPS coordinates to your pictures. It needs a one line change to recognize DNG files. This change is included the binary version I compile. Win32 Executable that you can use if you are running on a windows environment.
ImageMagick. Download the the Q16-windows-dll version. Run the installer, and make sure to install the activestate libraries (this is unchecked by default, so will require you to select it). My experience is you need to reboot or the libraries don’t get recognized by other programs properly.
TravelGen. This is a tool I created that generates static web pages of thumbnails and text. It was originally developed so that I could build web pages with image thumbnails and have a travel log mixed in, and make it fast to load (completely static). It provides a number of simple customizations, and I still use it today. I’ll document this in my next Technical Tip page.