Copter Image Data Processing Workflow

Some thoughts on the data processing workflow with multicopter image data:

lr-profile-template

Depending on the original image data format you might want to setup a different workflow but so far and with DNG Raw files (or other Raw files from a Bayer pattern sesnsor) I believe that Adobe Lightroom creates the best solution to generate clean and artefact-free JPEGs for further processing in UAS specific photogrammetry software packages.
Adobe Lightroom comes in two different licensing models (in 2016/2017). One model is a standalone licensing where you pay once and can use the software until your operating system is not supporting this software anymore OR you updated to different digital cameras that are not supported anymore by the (than) outdated LR version. Usually with the next major LR update you just buy the update costs and move on to the next version. This is different with the second licensing mode that Adobe tries to push into the market for nearly all of the software solutions they offer – its called “Creative Cloud” and you basically pay monthly to use the software. Its a subscription based licensing model that ist very interesting for software firms as it ties you to one software solution and revenue ist created on a more predictable (monthly) basis.

Lightroom is however one of the few very fast and effective solutions to manage huge amounts of Raw image files and LR doesnt touch the original Raw file – it only demosaics the bayer sensor pattern and saves a description of the processing steps that are planned for the output and visualizes temporarily the result. To create the modified file you have to export the image Raw file to a external file format (Jpeg/Tiff/PSD etc.). LR also provides lens profiles for most of the common lenses and also covers some copter specific lens designs. Lens profiles correct vignetting, chromatic aberrations and lens distortion. (https://helpx.adobe.com/x-productkb/multi/lens-profile-support.html). Additionally specific CA effects can also be corrected manually. All corrections can be saved as profiles and applied while importing a copter dataset from a flight campaign – you can even export the profiles as ASCII files and edit and reimport later again.

lr-lensprofile-dji

Fig.: Lens profiles for DJI copter sensors/camera combinations in Lightroom 6.6.1

LR also manages gradual or localized edits, can be used to remove dust spots, has distortion control, basic panorama stitching and HDR functionality and integrates with online gallery database solution via plugins and exports to other software solutions for better processing f.e. to panorama software (Autopano Pro) or HDR data processing solutions and for other special processing tasks. LR also handles metadata (it doesnt show height within the GPS tag but it exports the height nicely to the JPEG), can be used to create a tagged image database and works with DNG smart previews that allow keeping a full archive in smaller DNG size for mobile editing of thousand of shots of  flight campaigns or large photo shootings (see figure below).

More about DNR Raw format ist avail. here: https://de.wikipedia.org/wiki/Digital_Negative and on Adobes site: https://helpx.adobe.com/de/photoshop/digital-negative.html

smartprevie-in-lrlr-gps-coordiantes

Fig.: Smart Preview (Mini DNG) icon and Metadata info showing the GPS coordinate.

LR wont touch your Raw file, it can write your processing description into an xml file or keep it within a single database file (that is backuped regularly if configured). More Info on the LR data base techniques is available on Adobe help pages https://helpx.adobe.com/lightroom/help/create-catalogs.html .

Some info here: The LR database – the “catalogue file” “stores a

  1. reference to where the photo is on your system,
  2. stores Instructions for how you want to process the photo,
  3. stores Metadata, such as ratings and keywords that you apply to photos to help you find or organize them.
  4. LR is “non-destructive” – you can always use and work with you unmodified Raw file.

Why should I use Raw files instead of JPEGs that the camera generated? Well its possible to use the in camera generated JPEGs but the acuity and resolution as well as noise control are much better handled when you do the JPEG processing with a dedicated RAW converter. The quality is just a totally different story. Raw data ist also non-interpolated data. If you ever want to come back to your Raw data archive some years in the future – its likely that you use a much more sophisticated Raw developer software – you will be able to extract more information from the Raw files that you capture NOW. This is also a strategical aspect. Some camera systems are also known to produce very bad JPEGs in-camera. Especially Sony isnt too famous for good JPEG rendering while Nikon usually gets out of a comparison with better grades. While LR doesnt tell you much about the demosaicing algorithm used when you process a Raw file to a JPEG there are some packages online that are much more transparent when it comes to demosaicing the bayer pattern: namely “RawTherapy” is very interesting here: http://rawtherapee.com/blog/features but it lacks the integrated (does it all) capabilities and the databased linkage.

Alternatives to LR are: Phase One Capture  One Pro and DxO Optics Pro. Unfortunately the real competitor “Aperture” isnt any longer maintained by Apple. Aperture was a serious alternative to Lightroom – “Fotos” the follow up App from Apple isnt a professional batch environment for processing images anymore. Camera specific Raw import software comes f.e. from Nikon (Capture NX/ViewNX) or Canon but it lacks the integrated database connection and is usually much slower  processing large amounts of Raw files.

Automated Lightroom workflow with the following data processing on the DNG copter files :

  1. Lightroom Preset processing (on import) (you can also export that profile and check values  as ASCII – it has an -.lrtemplate ending – compare with figure on top of this article):
    1. Metadata update with address, copyright, website info and contact info,
    2. Renaming to a intelligent naming convention: YourNameAbbrev-ProjectName-DATE-CameraID-ShotID.DNG
    3. lens Profile correction including chromatic aberration correction, distortion correction, vignetting,
    4. sharpening with a high masking setup,
    5. noise reduction (very conservative),
    6. contrast and micro contrast improvement,
  2. LR-compression to DNG HDR stacks (if needed and/or planned) (LR creates DNG HDRs!)
  3. LR-export to adjusted uncompressed 8 Bit JPEGs (or 16 Bit TIFs – usually JPEGs will be enough), hundreds of 16Bit TIFs quickly demonstrate how small you hard disc system is in reality.

Photogrammetric workflow:

  1. Photoscan data processing (photogrammetric workflow) – http://www.agisoft.com
    1. Processing to point cloud level (high accuracy alignment, dense point cloud calculation) (this usually takes verrry long if you use the “ultra high” quality option – but this is the only way to use the full resolution of your original image data – the “High” quality level down samples to 1/4 of the image size),
    2. optional Chunk calibration and point cloud merging (if you worked with more than one point cloud)
    3. optional LAS point cloud export and point cloud transformation in LASTools,
      1. (Mean terrain height can be offset or negative),
    4. Export to UTM projected and processed ortho data sets,
    5. DSM extrapolation using LAStoolss – export to 16Bit height raster model,
    6. DTM calculation using LAStoolss
  2. Multispectral/Point Cloud data processing in Trimble OBIA software “eCognition” and/or PCI Geomatica (specialized commercial Earth observation software packages).
  3. Point Cloud visualization QTerrainReader or Autodesk ReCap can do that nicely.
  4. Point Cloud Analysis – … (this one is tricky – stay tuned for more in this chapter – right now  I am working in an OBIA environment (5.), but there is much more in CAD world that hasnt been tested!)

LAStools are a very handy command line controlled software solution to work with a massive amount of points. https://rapidlasso.com/lastools/ Some of the usual commands include mapping to an interpolated DSM raster height file and conversion of z-values with offset correction (wrong height values – due to an uncalibrated copter IMU). LAStools only runs on a Windows CLI unfortunately:

lastools-blast

Fig.: LAStools bast2dem that imports and works on a las/laz point cloud and rasters a dem file.

lr-profile-page

Fig.: Lightroom Profiles (Presets) to batch-transform image files – can be integrated into the Raw file-import function easily. Highlighted here is the preset import function.

lr-importpreset

Fig.: LR import preset (Apply During Import options), Developer options, in LR6.6.1

mtk –