DCamProf - a digital camera profiling tool

Table of contents

News

2016-05-07
Version 0.10.5, changes: 2015-11-09
Version 0.10.4, changes: 2015-10-30
Version 0.10.3, changes: 2015-10-27
Version 0.10.2, changes:

2015-10-25
Version 0.10.1: hotfix for a rolloff bug in the tone reproduction operator causing oversaturated skies among other things.

2015-10-23
Version 0.10.0, changes:

2015-09-29
Version 0.9.13, changes: 2015-09-26
Version 0.9.12, changes: 2015-09-17
Version 0.9.11, changes: 2015-09-16
Version 0.9.10, changes: Known serious bug: I've become aware that there are some pretty severe issues with the neutral tone reproduction operator and indeed this new gamut compression feature around the sun in sunset pictures, due to that the transition into clipping does not work well. I'm looking into fixing this and will release a new version as soon as possible. Linear profiles are unaffected.

This bug has been introduced sometime in the last versions, so until I've fixed this I recommend using version DCamProf v0.9.7.

DCamProf is new software and may contain serious bugs like this that I have missed in my testing. If you discover a bug like this I'm very grateful if it's reported back to me so I become aware of it and can fix it.

2015-09-14
Version 0.9.9, changes:

2015-09-13
Version 0.9.8, changes: 2015-09-06
Version 0.9.7, changes: 2015-09-05
Version 0.9.6, changes: 2015-09-03
Version 0.9.5, changes: 2015-08-28
Version 0.9.4, changes: 2015-08-18
Version 0.9.3, changes: 2015-08-14
Version 0.9.2, changes: 2015-08-11
Version 0.9.1, changes: 2015-08-05
Version 0.9.0, changes: 2015-07-29
Version 0.8.4, changes: 2015-07-26
Version 0.8.3, changes: 2015-07-06
Version 0.8.2, changes: 2015-07-03
Version 0.8.1, changes: 2015-06-26
Version 0.8.0, changes: 2015-06-09
Version 0.7.4, changes: 2015-06-07
Version 0.7.3, a quick update due to some bad bugs introduced in 0.7.2: 2015-06-05
Version 0.7.2, changes: 2015-05-30
Version 0.7.1, changes: 2015-05-23
Version 0.7.0. Changes: 2015-05-18
Patch release 0.6.4. Fixed overall stats bug. Added possibility to show matrix result only on make-profile (-L), useful when doing repeated runs tuning weights for a matrix-only profile. Minor adjustment of matrix optimizer. Updated make-profile weighting documentation with some more tips.

2015-05-17
Patch release 0.6.3. Fixed a plot bug introduced in 0.6.2. I also adjusted whitepoint preservation handling, now the LUT (per default) excludes patches close to the whitepoint, as the matrix is whitepoint-preserving (neutral patches already optimized for as good as possible) it doesn't make sense to stretch there. Added sign to error tables on lightness and chroma so one can see if the color is too light or dark or too saturated or desaturated. Added two more error vector plots.

2015-05-16
Patch release 0.6.2. Added ICC support for test-profile, including transfer function reversal, and also some ICC plots. Now ICC support is as good as DCP support.

2015-05-13
Patch release 0.6.1. Fixed various ICC bugs, added possibility to provide transfer functions which means Capture One ICCs can now be generated. It's now also possible to make LUT ICCs. Test-profile for ICC is still missing, but otherwise ICC support should be complete. To read transfer functions a new dependency was added, libtiff. The native LUT now applies whitepoint preservation.

2015-05-11
Here's 0.6.0. It's now possible to make ICC profiles (make-icc), matrix-only for linear pipelines only to start with (-L flag must be enabled). Added possibility to add a tone curve when making a DCP. Extended profile format (can't use old profile.json files, sorry). Added icc2json and json2icc commands. Added -I parameter to make-target so you can separate RGB and XYZ illuminants also in that command. Increased parsing flexibility for Argyll-like files (various CGATS), should handle patchtool files better.

2015-05-05
Patch release 0.5.4. Fixed bug in RGB/XYZ levels when merging several targets, fixed a few Windows compile bugs. Now DCamProf supports emissive spectra in the target files (added SAMPLE_TYPE column, "R" or "E"). When merging targets the special patch "illuminant" and "white" (new) is now always kept even if there are nearby patches.

2015-05-03
Patch release 0.5.3. Added possibility to read Argyll SPECT files (produced by Argyll's illumread) as illuminants. Now it's possible to make dual-illuminant DCPs directly with the make-dcp command. When running the command without parameters there is now a full list of exif light sources and their temperatures, a useful list when choosing a suitable calibration illuminant. Added a new command txt2ti3 to convert raw text files with spectral data to .ti3 that make-target can read, useful when getting spectral data from various third-party sources.

2015-05-02
Patch release 0.5.2. A slight adjustment of observer remapping in make-dcp and some update of observer documentation.

2015-05-01
New patch release 0.5.1. I hadn't tested on Lightroom properly. It was shown that Lightroom cannot handle too high precision on matrix rationals and it doesn't like if the standard observer WP is different from 1931_2. I have now changed default observer to 1931_2 as it's an easier to use default, and made an automatic remapping in the make-dcp tool to handle the case when a different observer was used during profile creation.

2015-04-30
Here's the first release of DCamProf to the public (0.5.0). This is an early one, and while you can make camera profiles with it it's still in a "hackish" state, probably some bugs left and certainly slow. There's lots of silly loops over loops here and there. My excuse is that I've been focused to get things working first rather than get stuck optimizing for speed. As usual with these kind of projects it has taken far more time than I initially planned, but then it's far more feature-rich too!

What is DCamProf?

DCamProf is a free and open-source command line tool for generating camera profiles and perform tasks related to camera profiles and profiling.

To generate a camera profile you need either the camera spectral sensitivity functions (SSFs) or a measured target. DCamProf has no measurement functionality, but you can use the free and open-source Argyll CMS to get a .ti3 file with measurement data which DCamProf can read.

Here's a feature list:

Note that many features are related to camera SSFs, and indeed you get most out of DCamProf if you have SSFs available. You don't need them to make great profiles though, having them is more about flexibility and convenience than quality. You can then also learn many things of how cameras work by testing various scenarios, testing the efficiency of a specific target design, testing how a profile performs under a different illuminant etc.

The reason I started this software was that 1) Argyll can't do DCPs, and 2) I was not pleased with the commercially available alternatives for making own camera profiles. Too much hidden under the hood, too little control, and many indications that the quality of the finalized profiles was not that good. Then I added the SSF ability and the software grew to something more than just a profile maker, now you can say it's a camera color rendering simulator as well.

The software is quite technical, but if you can use Argyll you can use DCamProf. You can also find a separate tutorial of how to make profiles using DCamProf. It's supposed to complement the reference documentation found on this page.

Downloading and building DCamProf

Download the source code for DCamProf v0.10.5. It's developed on Linux and compiling it there is easy as all third parties should be available as standard packages in your Linux distribution. It should also be relatively easy to build on Mac OS X. On Windows it's easy to build using Cygwin, or Mingw. It won't work with Microsoft's own compilers though as they refuse to support the C99 standard, and I'm not in the mood of dropping that just to let Microsoft be lazy and ignore standards.

DCamProf uses OpenMP to make use of all your CPU cores in parallel. You can build it without OpenMP (remove the -fopenmp words from the Makefile) but some aspects of the program will then run much slower as it will use only one core. At the time of writing Mac OS X standard compiler clang doesn't support OpenMP (it's in the works though), unless you build your own clang from source.

An alternative to building it on your OS X or Windows platform is simply to install a Linux virtual box and run it there, make sure you give the virtual machine access to all cores. An added bonus is then easy access to Argyll, gnuplot, exiftool and other related tools.

Temporary link: I currently don't provide own builds, but might do so in the future. In this forum thread you can often find recent ready-made builds (be sure to jump to the end and search backwards).

How DCamProf models cameras

DCamProf looks on the perfect camera as a colorimetric camera, that is the SSFs matches the color matching functions for the CIE XYZ color space. No real camera is colorimetric so the goal of profiling is to make it perform as close as possible to one.

DCamProf assumes that the camera is linear, that is if you for example double the intensity of a certain spectrum the raw values will also double and there will be no change in their relation. This is indeed true for any normal digital camera today, with the possible exception of extreme underexposure and very close to clipping where there can be non-linear effects.

The linearity assumption leads to that the correction lookup table only needs to be indexed on chromaticity (that is saturation and hue, but not lightness), but the output still needs correction factors for all three dimensions as some colors can be rendered too dark or too light with a fixed factor throughout the full lightness range. That is DCamProf works with a LUT with 2D input and 3D output, commonly referred to as a 2.5D LUT.

DCamProf does allow you to apply a subjective look on top of the accurate colorimetric 2.5D profile. It will then use a full 3D LUT so you can make lightness-dependent adjustments, but the colorimetric part always stays 2.5D (well, except for some gamut compression of extreme colors, but that doesn't turn up in normal images).

2.5D vs 3D LUT

With a 2.5D LUT we assume that the same color in a darker shade will have the same shape of its spectrum, only scaled down. This is true if you render colors darker by reducing the camera exposure in a fixed condition. However, if we compare a dark and light color of the same hue and saturation in a printed media the spectrum shapes can differ because a typical print technology will alter the colorant mix (eg inks) depending on lightness. In some cases lightness is controlled by adding a spectrally flat white or black colorant, and in those cases spectrum shapes are retained, but that is not always the case.

This means that our linearity assumption breaks as the relative mix of camera raw values may differ slightly between dark and light colors and in this case a full 3D LUT could make a more exact correction. However, this only makes sense in highly controlled conditions when copying known media (such as printed photographs), that is when you're using the camera just like a flatbed scanner. The light source must be fixed, the camera exposure must be fixed, and the camera profile must be designed using a target made with the same materials as the objects you shoot.

As a 3D LUT only makes sense in this very narrow use case DCamProf supports only 2.5D (so far). If you really need a 3D LUT you can use Argyll, but you're then limited to ICC profiles. For strict reproduction work that may be a better approach.

Note that commercial raw converters often use 3D LUTs, not to achieve better colorimetric accuracy though but to make subjective "look" adjustments, which you also can do with DCamProf with its "look operator" functionality.

Basic workflow for making a DNG profile using a test target

  1. Get or make a physical test target. The classic Macbeth/X-Rite 24 patch color checker is a fine choice.
  2. Get or make a reference file for the test target, preferably containing reflectance spectra.
    • If you're using that 24 patch color checker like most will be doing, the reference file with reflectance spectra assembled by BabelColor is the second best choice if you can't measure it yourself. This data should be valid for the nowadays more popular "ColorChecker Passport" product too.
    • The above BabelColor CGATS text file example needs some conversion to be used with Argyll. To save you some time I've done it for you, look for "cc24_ref.cie" in the DCamProf distribution.
    • Note tha X-Rite changed their formula slightly in ColorCheckers produced November 2014 and later. The bundled cc24_ref.cie is for targets produced before November 2014.
  3. Shoot your test target under the desired light source. Store in raw format.
  4. Convert the raw file to a 16 bit linear TIFF without white balancing.
    • You can use a recent version of RawTherapee (export for profiling, with disabled white balance), or DCRaw
      • dcraw -v -r 1 1 1 1 -o 0 -H 0 -T -W -g 1 1 <rawfile>
    • If you have an odd camera format and you want to use the profile in Adobe's products it may be safer to convert to DNG first using Adobe's DNG converter, as raw decoding of proprietary formats may differ a little concerning application of black levels, white levels and calibration data.
    • Crop so only the target is visible, and rotate if needed. Argyll is very sensitive to target orientation. If you use some image editor to do this make sure that the full 16 bit range is kept, that is don't use 8 bit Gimp. If you use RawTherapee you can crop and rotate in there.
  5. Use Argyll scanin command to generate a .ti3 file.
    • It needs the target reference file, test target layout file and raw image as 16 bit TIFF as input.
    • scanin -v -dipn rawfile.tif ColorChecker.cht cc24_ref.cie
    • The scanin command will generate a diag.tif which shows patch matching (look at it to see that it matched) and a rawfile.ti3 file which contains the raw values read from rawfile.tif together with reference data from the cc24_ref.cie file.
  6. Use DCamProf to make a profile from Argyll's rawfile.ti3 target file.
    • dcamprof make-profile -g cc24-layout.json rawfile.ti3 profile.json
    • The above command doesn't specify any illuminants, which means that the profile will be made for D50 and the rawfile.ti3 must contain reflectance spectra (it will if the example cc24_ref.cie is used) or have it's XYZ values related to D50. To change calibration illuminant use the -i parameter, and if the .ti3 lacks reflectance spectra specify its XYZ illuminant using -I.
    • Per default DCamProf will automatically relax the LUT a bit, that is prioritize smoothness over accuracy. How this is done can be precisely manually controlled if you desire, see the documentation for the make-profile command for an in-depth explanation.
    • The target layout is provided in a JSON file cc24-layout.json. If the target contains both black and white patches glare will be modeled and reduced, and if the target contains several white patches (not the CC24, but for example a ColorChecker SG) it will be flat-field corrected.
  7. Convert the native format profile to a DCP.
    • dcamprof make-dcp -n "Camera manufacturer and model" -d "My Profile" profile.json profile.dcp
    • For many raw converters the camera manufacturer and model must exactly match what the raw converter is expecting. For example if using Adobe Lightroom the name must match the name Lightroom is using for it's own DCPs.
    • The description tag (set by -d, "My Profile" in this example) will be the one shown in the profile select box in for example Adobe Lightroom.
    • The above example makes a colorimetric profile without a curve. If you want to embed a tone-curve and make use of DCamProf's neutral tone reproduction (which really makes more sense for a general purpose profile), you can do like this:
      • dcamprof make-dcp -n "Camera manufacturer and model" -d "My Profile" -t acr profile.json profile.dcp
  8. Optionally use DCamProf's dcp2json and json2dcp commands to do any manual edits of the DCP file, such as changing profile name and copyright.
  9. The DNG profile is now ready to use in your raw converter.

Basic workflow for making an ICC profile using a test target

Making an ICC profile is almost the same as a DNG profile. Actually you can follow the exact same workflow and run the make-icc command at the end instead of the make-dcp, the native profile format can be converted to both types. However, some raw converters using ICC profiles apply some sort of pre-processing such as a curve before the ICC profile is applied which much be taken into account. Capture One is one such raw converter.

The steps that are the same as in the DNG profile case are only briefly described here, so look there if you need further details.

  1. Get a physical target with reference file, and shoot a raw file in desired light.
  2. Export to a tiff for profiling in the raw converter you will be using.
    • How you do it varies depending on raw converter, look in its documentation or search on the 'net.
      • Capture One: Select ICC profile: "Phase One Effects: No Color Correction", Select Curve: "Linear Response" (or similar), rotate/crop to show only the target. Then Export variants, "16 bit TIFF", "Embed camera profile". Note: you probably don't want to use "Linear Scientific" as it is a special mode which disables highlight reconstruction. Of course there should be no clipping in the profiling TIFF, but as the profile should be used with the same curve when finished the "Linear Response" is better for all-around use.
    • White balance setting does not matter, unless you intend to let the profile correct a camera preset (not a common use case, see make-icc reference documentation for more information).
    • If you can choose a curve, choose linear.
      • DCamProf must somehow be able to calculate the linear data. If the exported TIFF contains a transfer function tag, like from Capture One, the curve choice doesn't really matter as it can be linearized anyway. I still recommend using a linear curve during profiling, for precision reasons.
      • If the exported TIFF doesn't contain a transfer function tag, you should export with a linear curve, unless if you can find out the transfer function in some other way. Note that "linear" doesn't always mean exactly linear so you may need to find out the transfer function anyway somehow.
  3. Use Argyll scanin command to generate a .ti3 file.
    • scanin -v -dipn target.tif ColorChecker.cht cc24_ref.cie
  4. If the raw converter applies a pre-processing curve (like Capture One), re-process the .ti3 file to get linear RGB data.
    • dcamprof make-target -X -f target.tif -p target.ti3 new-target.ti3
    • DCamProf gets the pre-process curve from the transfer function tag in the profiling tiff. You can also extract it separately with the dcamprof tiff-tf command, but as the make-target command can handle the tiff directly it's generally not needed.
    • If pre-processing curve cannot be had from the tiff, you need to supply it in a json file (still -f parameter), see the provided data example for formatting.
  5. Use DCamProf to make a native profile from the linear .ti3 file.
    • dcamprof make-profile -g cc24-layout.json new-target.ti3 profile.json
  6. Convert the native format profile to an ICC profile.
    • If pre-processing curve was used it must be given:
      • dcamprof make-icc -n "Camera manufacturer and model" -f target.tif profile.json profile.icc
    • ...otherwise the -f parameter is skipped:
      • dcamprof make-icc -n "Camera manufacturer and model" profile.json profile.icc
    • If you want to embed a tone curve and make use DCamProf's neutral tone reproduction you do like this (both with and without pre-processing shown):
      • dcamprof make-icc -n "Camera manufacturer and model" -f target.tif -t acr profile.json profile.icc
      • dcamprof make-icc -n "Camera manufacturer and model" -t acr profile.json profile.icc
      • The "-t acr" will apply Adobe's standard film curve, you can also design your own, or import via the tiff-tf command.
      • The curve will be applied to the profile's LUT.
      • If you're using Capture One, read the section about Capture One and curves.
  7. The ICC profile is now ready to use in your raw converter.
Note that some ICC raw converters do additional processing than just a curve and white-balance, they may for example do some sort of pre-matrixing. If the stripped profiling tiff looks much more saturated than a corresponding tiff from a DCRaw or DNG profiling workflow it's likely that some pre-matrixing has been applied. As you profile based on the raw converter's own profiling TIFF this doesn't matter, except that the native format profile generated in the process will not be compatible with any other raw converter.

There are also ICC raw converters that do no specific pre-processing, that is provide the ICC profile with "pure" raw input just like to a DNG profile, meaning that you can use the same native profile produced in a DNG workflow and make an ICC profile too. DxO Optics is one such raw converter.

Capture One and curves

If your raw converter allows choosing curve separately, as Capture One does, it's actually against the color rendition principle of DCamProf. With DCamProf the profile itself applies the curve, and if you want different curves you simply render different profiles, one for each curve. The reason why this is the case is because tone curves can fundamentally affect color rendition, as described in the "tone curves and camera profiles" section. That Capture One doesn't alter ICC profile when the curve is switched I'd say is broken color science, however due to the mild shape of their curves and that it's applied before the ICC profile the color appearance is not affected that much. Actually they have a mixed approach, some of the curve is applied separately before the ICC and some is applied by the profile's LUT. This mixed approach makes the color appearance more stable between the different curves than it otherwise would have been, but it also makes the result with "Linear Response" far from actual linear.

In any case, you can assume that their bundled profiles have been optimized for the default curve and the others will provide somewhat sub-optimal color, although the difference is not huge.

If you like your DCamProf profile to work the same way, you do this way:

  1. Make a profile with the linear response, that is everything in the standard workflow until you've run the make-profile command.
  2. Export a profiling TIFF one with the "Linear Response" (let's call it linear.tif) and the other with the desired curve, usually "Auto" or "Film Standard" (let's call it curve.tif).
  3. Extract the actual shape of the curve from the TIFF files, using the tiff-tf command:
    • dcamprof tiff-tf -f linear.tif curve.tif tone-curve.json
    • It may warn that the curves don't end at 1.0, don't worry about that. Capture One does that, probably related to having some margin for highlight reconstruction or white balance adjustments. The tiff-tf command will automatically compensate.
  4. Make a preliminary ICC profile adapted for the curve's pre-processing, and also apply the tone curve.
    • dcamprof make-icc -n "Camera manufacturer and model" -f curve.tif -t tone-curve.json profile.json preliminary-profile.icc
    • as the pre-processing curve and tone curve are the same, they will cancel them out and the profile is linear in terms of contrast, but the neutral tone reproduction operator will have done its work with the appropriate curve, so the profile becomes optimized for the desired curve.
  5. Load the preliminary profile and compare with the bundled. Probably the DCamProf profile has lower contrast because the native profile applies some extra contrast in the LUT. Use the curve tool inside Capture One to match the contrast. An exact match is not necessary, what you find most pleasing is the best for you.
  6. Transfer the coordinates from the curve tool to a JSON file DCamProf can read, let's call it modifier-curve.json. Find an example below.
  7. Re-run make-icc with the extra curve cascaded.
    • dcamprof make-icc -n "Camera manufacturer and model" -f curve.tif -t tone-curve.json -t modifier-curve.json profile.json profile.icc
  8. The profile is now ready to use. The look will be optimized for the chosen curve, but it will look okay too for "Linear Response", just like Capture One's bundled profiles. Of course, there will be the residual curve left (the modifier-curve) when you use the profile in "Linear Response" mode, the same way it is for native profiles.
The modifier curve is suitably designed with the curve tool inside Capture One. Load your first profile generated in the workflow above, and then edit a curve to your liking. Then you copy the handle values into a text file with the JSON tone curve format, like this: 
{
    "CurveType": "Spline",
    "CurveHandles": [
      [ 0,0 ],
      [ 14,8 ],
      [ 27,20 ],
      [ 115, 118 ],
      [ 229, 233 ],
      [ 255, 255 ]
    ],
    "CurveMax": 255,
    "CurveGamma": 1.8
}
Capture One uses 0-255 as their range in the curve, and the curve works with gamma 1.8.

Basic workflow for making a profile from camera SSFs

If you have the camera's spectral sensitivity functions you can skip the target shooting process.
  1. Format your camera's SSF data into a JSON file that DCamProf can read.
    • Use the distributed examples as a guide for how the JSON file should be formatted.
    • If you don't have the equipment or knowledge to measure your camera's SSFs, you can look in the SSF links section and see if you're lucky and can find your camera in one of the sources.
  2. Generate a "virtual" target with your desired spectral data.
    • You can use DCamProf's built-in spectral database or use its ability to generate spectra, or import from some other spectral source (see provided import_spectra.txt for formatting, it's the Argyll .ti3 format, but you can use a subset).
    • Here's a basic example when we just generate a color checker from the built-in spectral database target:
      • dcamprof make-target -c ssf.json -p cc24 target.ti3
    • The resulting target.ti3 contains reflectance spectra for all patches, plus XYZ reference values and RGB values for the camera rendered using the SSFs found in ssf.json.
  3. Make the profile
    • dcamprof make-profile -c ssf.json target.ti3 profile.json
    • We don't really need to provide the camera's SSF again (ssf.json) as the target file already contains rendered RGB and XYZ values, but it's a good habit since then the RGB (and XYZ) values will be regenerated from spectra each time which is convenient and reduces the risk of making mistakes.
    • If the SSFs are of high quality you will typically get a considerably better match with this than if you have shot a test target. This means that there is often less need of weighting and LUT relaxation when rendering the profile.
  4. Convert the native profile to a DNG profile or an ICC profile.
    • See the description for the basic workflow using a test target for more details.
In this example workflow we keep the illuminants at default, D50. As we let the spectral information follow through in the workflow we can change calibration illuminant late in the process, when making the profile: 
  dcamprof make-profile -c ssf.json -i StdA target.ti3 profile.json
Note that as SSFs are generally measured from real raw data without pre-processing, profiles generated from SSFs won't work for ICC raw converters that does pre-processing before applying the ICC.

Choosing test target

Due to natural limitations of camera profiling precision it's quite hard to improve on the classic 24 patch Macbeth color checker when it comes to making profiles for all-around use. It's more important to have a good reference measurement of the test target than to have many patches. If you don't believe me please feel free to make your own experiments with DCamProf; by using camera SSFs you can simulate profiling with both few and many patches and compare target matching between them.

DCamProf allows you to use any target you like though, you can even print your own and use a spectrometer and Argyll to get reference values. Although darker repeats of colors does not hurt there's not much gain from it as the LUT is 2.5D, so an IT-8 style target layout (many patches are just repeats in darker shades) does not make that much sense.

Dark patches are problematic as they are more sensitive to glare and noise (both in camera and spectrometer measurement), so an ideal target has as light colors as possible for a given chromaticity.

The profiling process requires at least one white (or neutral gray) patch. It's no problem if it's slightly off-white though. It should preferably also contain one black patch which should be the darkest patch in the target. This black patch is used to monitor glare. If feasible the "black" should be made as light as possible while darker than the darkest colored patch. If the black patch is significantly darker than the darkest colored patch we may detect a glare issue than in actuality only affects the black patch.

The white (and black) patches should preferably have a very flat spectral reflectance, as it makes glare monitoring more accurate.

Most targets have a gray scale step wedge which can be used for linearization. Digital cameras have linear sensors, but the linearity can be hurt by glare (and flare). Normally it's much better to reduce glare to a minimum during shooting than trying to linearize afterwards, as glare distortion is a more complex process than just affecting linearity.

(Semi-)glossy targets, such as X-Rite's ColorChecker SG, are extremely glare-prone and therefore hard to use. They cannot be shot outdoors, but must be shot indoor in a pitch-dark room with controlled light. Due to their difficulty during measurement the end result is often a worse profile than using a matte target. I recommend to first get good results with a matte target before starting to experiment with a semi-glossy. Those targets often receive bad reviews simply because the users have not minimized glare when shooting them.

A note about X-Rite targets: due to regulatory and compliance reasons the colors where changed sligthly in November 2014, so all targets produced in November 2014 and later has slightly different colors than those produced earlier. This means that if you don't measure your target yourself you need to make sure you have a reference file that matches the production date of your target. These things can happen also for other manufacturers and they may not always be announced.

If you have the camera's SSFs you can use the built-in spectral databases (or import your own) rather than shooting real test targets. In that case you will probably want to select spectral data that matches what you are going to shoot, for example reflectance spectra from nature if you are a landscape photographer.

The classic 24 patch Macbeth color checker, originally devised in the 1970's. Despite its age it still holds up well for designing profiles, thanks to relatively saturated colors with a relatively large spread. As seen in the u'v' chromaticity diagram (with locus, AdobeRGB and Pointer's gamut) there's still space to fill though, and some patches are occupying almost the same chromaticity coordinate which is not that useful when making 2.5D LUTs.

Making your own target

Using the make-testchart command you can make your own target. Here's a workflow, showing making a target for an A4 sheet, and using a Colormunki Photo for scanning the patches:
  1. Generate test patches in Argyll's .ti1 format:
    • dcamprof make-testchart -l 15 -d 14.5,12.3 -O -p 210 target.ti1
    • In the above example we specify the chart layout with -l, -d and -O parameters, so that white patches can be placed optimally for flatfield correction later on. The layout must match what Argyll's printtarg is going to generate.
  2. Generate a .tif for printing, a .cht file for chart recognition and .ti2 file for scanning using Argyll's printtarg command:
    • printtarg -v -S -iCM -h -r -T300 -p A4 target
    • It's important that we use the -r flag, otherwise Argyll will randomize the patch positions which can break flatfield correction.
    • In the above command we choose A4 sheet and Colormunki half-size patches, it will make 210 patches in one sheet, like we have generated. Even if you don't have a Colormunki instrument the generated patch size is nice for a test chart for photography.
  3. Print the tiff file on an OBA-free paper matte or semi-gloss, with color management disabled, that is the same way you would print a test chart for printer profiling.
  4. Measure the reflectance spectra of the patches (this will make a .ti3 file):
    • chartread -v -H -T0.4 target
  5. Convert the .ti3 to a reference .cie file to be used with Argyll's scanin later.
    • spec2cie -v -i D50 target.ti3 target.cie
  6. Now you have the printed test chart, a target.cht chart recognition file and a target.cie reference spectra file which can be used in the profiling workflows.
The quality of your own target will depend on the spectral qualities of your printer. A modern inkjet printer with several inks will have better spectral qualities than many other print technologies, but will still not be as good as the special print techniques used when commercial test targets are made. If you are curious about target performance you can use the SSF functionality of DCamProf to make simulations. Despite spectral limitations it seems that they perform at least as good as a CC24 or sometimes even better when it comes to making profiles that matches real colors.

Semi-gloss targets will get very high saturation patches, but those are difficult for the camera to match and it's hard to shoot those targets without glare issues, they may also be harder to measure accurately with the spectrometer if it has limited range (some consumer spectrometers start at 420nm) or issues with glare. Making a matte target may be better in practice, although you can't get deep violet colors in those.

Test target reference files

The foundation of profiling using test targets is that the profiling software knows what CIE XYZ coordinate each color patch corresponds to, or even better which reflectance spectrum each color patch has so the software can calculate the XYZ values internally.

Higher end test targets may be individually measured so you get a CGATS text file with reference values, and Argyll's scanin tool can use them directly. If you get a standard 24 patch Macbeth color checker you probably don't have an individual reference file and then a generic file like the one provided with DCamProf (cc24_ref.cie) will have to do. Having the reflectance spectra is strongly preferred over pre-calculated XYZ values, so do get that if you can. The problem with pre-calculated values and no spectra is that when changing illuminants the software cannot re-calculate XYZ from scratch using spectral data, but must rely on a chromatic adaptation transform which is less exact. It's also a higher risk for the user to mess up by forgetting to inform DCamProf of which illuminant the XYZ values are related to. If there's spectral data the reference values are always re-generated from scratch to fit the currently used illuminant, which is both exact and convenient.

If you have a spectrometer (usually designed for printer profiling) you can measure your target and generate your own reference file with spectra. Using Argyll you do like this:

  1. Create or find an Argyll .ti2 text file which contains the test target layout needed for the spectrometer scan. Note that Argyll is distributed with .ti2 files for many of the popular commercial test targets, the file is called ColorChecker.ti2 for the Macbeth 24.
  2. Scan the target with Argyll's chartread (exclude the .ti2 suffix, for most Argyll commands the suffix should be excluded):
    • chartread -v -H target
    • Note that some targets may have too small patches to be read successfully with your instrument. For example an X-Rite ColorChecker Passport cannot be read by an X-Rite Colormunki spectrometer.
  3. Convert the resulting .ti3 file (which contains complete spectra for each patch) to a new .ti3 file with reference CIE XYZ values with your desired illuminant.
    • spec2cie -v -i D65 target.ti3 reference.cie
    • In the above example "D65" was chosen, but you can also choose "A" or "D50", or any other supported by the spec2cie tool.
    • As the spectral data will be kept in the file it does not really matter what illuminant (or observer) you use, you can change that again when generating the profile with DCamProf. The described method is however also compatible with a standard Argyll workflow.
  4. The resulting reference.cie can now be used together with Argyll's scanin tool.
  5. If you don't have a usable .cht file for the chart layout (a more detailed layout information than needed for the spectrometer scan), you need to generate one. If you have generated your own chart using Argyll's printtarg you can add the -s (or -S) parameter to it to get the .cht file. If you haven't used printtarg it's unfortunately a bit of a headache to make your own .cht. You can use the scanin tool as a help for that (using the -g parameter), but it's quite messy with lots of manual edits. At the time of writing I have not tried doing it myself and as long as you're using a reasonable popular target there will be a .cht file distributed with Argyll, and if you make your own using Argyll you can make the .cht when calling printtarg.
It's probably better to measure your own target and get full spectral information than getting a typical pre-generated reference file with only XYZ values for some pre-defined illuminant. If it really is better depends on the precision of your instrument, the sample-to-sample variation of test targets and the quality of the provided reference file. It's not possible to really know what will be best, you can try both and see what you like the most. If there's some serious problem with the reference file it's usually noticed when making the profile, the LUT must make extreme stretches etc.

In some cases you could get the reference spectra in some format that Argyll can't read directly. Argyll is delivered with a few conversion tools to handle other common text formats, cb2ti3, kodak2ti3 and txt2ti3. You may be helped by making a dummy conversion using DCamProf, like this: dcamprof make-target -p input.txt -a "name" output.ti3, and sometimes you may have to do some manual edits in a text editor too to get it into a format Argyll accepts.

Shooting test targets

To consider: If you are simulating daylight using an artificial light source it's better to use a high temperature halogen lamp, such as a Solux lamp on overdrive, than a fluorescent. Using lower temperature halogen lamps and putting an 80A filter on the camera lens is also a way. Or just shoot outside in real daylight, but that only works with matte targets.

Avoid reflections from nearby colored surfaces that may distort the color of the light source. If shooting outdoor, shooting in an open space with someone holding up the test target in front away from the body is a good alternative.

I recommend to defocus very slightly so you won't capture any structure of the target patches surface and instead get fields of pure color. If your camera lacks anti-alias filter this also makes sure you get no color aliasing issues. Shoot at a typical quite small aperture, say f/8 if 135 full-frame.

Argyll's scanin is sensitive to perspective distortion, so try to shoot as straight on as possible, and correct any residual rotation/perspective in the raw conversion.

If you know what you are doing you can push the exposure a little extra to get optimal "expose to the right" (ETTR) and thus as low noise as possible. But be careful, clipped colors will be a disaster in terms of results. I use to exposure bracket a few shots and check the levels in the linear raw conversion to see that there is no clipping.

Uneven lighting is a common problem in camera profiling. The typical recommendation is to make sure you have even lighting (at least two lights if not shooting outdoor) and shoot the target small in the center (to minimize vignetting). However, if you employ DCamProf's flatfield correction (the testchart-ff command) you can relax the even-lighting requirement quite a bit. Flatfield correction evens out the light with high accuracy, so you need only make sure all parts of the target has sufficient light to avoid noisy patches. Some halogen lights may have an outer rim of light of a different light temperature. This is not well corrected with flatfield correction, so make sure the target is at least lit with the same light spectrum all over.

Using fewer lights (maybe only one) and compensate with flatfield correction can be a smart strategy when shooting glossy targets, as it's easier to keep the rest of the room dark. Room darkness is very important to reduce glare which is a real issue with (semi-)glossy targets.

Glossy and semi-glossy targets allow for higher saturation colors on the patches, but are also more difficult to shoot as they produce glare. Glare is minimized by being in pitch-dark room and having the light(s) outside the "family of angles". If the target is replaced with a mirror you should only barely see the dark room and camera in it, certainly not any lights. Having a long lens narrows down the family of angles, and a projecting light source (like a halogen spotlight) and dark/black cloth around the target makes sure as little stray light as possible bounces around in the room.

This may look as a perfect target shot, even diffuse outdoor light, no visible reflections. However as the target is semi-glossy the surrounding diffuse light coming from all directions add up the direct reflection component (glare) so the contrast of the target is lowered and the photograph will not match reflectance spectra measurements. Semi-glossy targets must be shoot in indoor lab setups with dark surroundings and projecting light(s) outside the family of angles.
(Semi-)glossy targets are virtually impossible to shot accurately outdoor as you cannot shoot from a dark position, that is if you put a mirror where the target is you will likely clearly see the camera and yourself, which means you will have glare. If you still shoot it in that light it will be affected by glare and produce a lower contrast result, the dynamic range easily drops from 7 stops (typical range in a semi-glossy target) to about half of that. This won't be visible until you make side-by-side comparison or note poor profiling results (typically an over-saturated profile with some bad non-linearities).

Veiling glare is a lens limitation of how large dynamic range it can capture. It's typically between 0.3% to 0.5% for high quality lenses, the fewer lens elements and better quality coating the lower veiling glare. I thought I'd mention it as you may have heard of it, but compared to other forms of glare this is negligible so you don't need to worry about it. Do avoid lens flare though, that is the lens must be in shadow, use a lens hood and make sure you have no light sources towards the camera. Also make sure the viewfinder is closed tight so no light comes in that way.

If you shoot a glossy target be prepared that you can have issues with dark patches, as those are affected most by glare. Removing those from the measurement (using an exclude list to the make-profile command for example) can be a better way to solve the problem than trying to correct the measurement error in other ways. Due to the many difficulties with semi-glossy targets I recommend to simultaneously make a profile from a matte target so you have a profile to sanity-check against.

In theory a gray scale step wedge in the target could be used to correct glare. With DCamProf you can enable "glare matching" in the testchart-ff and also directly in make-profile to compensate glare-induced non-linearity. However, glare distorts more than just linearity and in unpredictable ways meaning that any linearization or glare matching will only help to some extent, so don't rely on it. You can indeed improve results this way, but it often ends up worse than just excluding the darkest patches (those that are most affected by glare) from profiling. So the recommendation with DCamProf is to reduce glare to a minimum, and keep an extra eye on the performance of dark patches, and exclude them if they seem problematic.

White balance and camera profiles

The white balance setting in your raw converter and your camera profile interacts so before making profiles it's good to have some insight into that.

Both DCPs and ICCs make corrections on white-balanced data, that is you feed the profile with a white-balanced image. For DCPs it might seem that you don't as the "ColorMatrix" work on unbalanced image data (more on that later) but the actual color rendering is decided by the "ForwardMatrix" and the LUT which work on the white-balanced image.

Naturally this means that in order for the profile to make the "correct" adjustments it must be used with the exact same white balance as used during profile design. Which white balance is used during design? Per default DCamProf will re-balance the target such that the whitest patch in the target is considered 100% neutral (real targets usually differ 1-2 DE from perfect), which means that white balance picker on the whitest patch is the best balance. You can disable this (-B to make-profile) and then DCamProf calculates the optimal white balance automatically, which is when camera white matches the calibration illuminant reflected by a 100% perfect white patch, that is usually slightly different from the whitest patch in the target. In any case it's a picked white balance, not the "As Shot" camera preset balance (there is an ICC special case though where you can design a profile for a camera white balance preset).

A well-behaved profile, that is one with only small and wide area stretches in the LUT, will be robust against slightly different white balance so it won't matter if you set it a little bit off to get a warmer or cooler look for example. A profile which has strong and very localized stretches (not a good profile!) may make sudden strange color changes when you shift white balance. This is because if you change white balance you apply a cast on all colors, which means that the colors move to other start positions in the LUT, and will get corrections that was intended for other neighboring colors, and if there are strong localized corrections the result can become quite off.

Wouldn't it be better if the ideal profile white balance was applied first, then the profile, and then your own user-selected white-balance? Yes, if the illuminant would always be the same as the one used when shooting the target, but if you shoot outdoors that's not the case. And in any case that's not how raw converters work so you can't have it that way even if you'd like it.

The take-away message is that for ideal profile result you should always set the white balance to represent white as good as possible, and if you want to make a creative cast, for example a bluer colder look, you should ideally apply that look with other color tools rather than the white balance setting. However, many (most?) raw converters don't make it easy to apply a cool/warm look in a different way than using the white balance setting, so that's what we usually end up doing anyway. If you've made a well-behaved profile (which you should) that should not be any real problem. Yes, profile corrections will not be as exact as when used at its designed profile, but if you're creating a look anyway that won't matter.

The most robust profiles concerning white balance changes is a pure matrix-only profile (no LUT), as they are 100% linear.

DCP-specific white balance properties

DCPs are a bit special when it comes to white balance, they have a more immediate connection to it than ICC profiles. The embedded "ColorMatrix" is not used for any color corrections, but to figure out the connection between a camera raw RGB balance (internal white balance multipliers) and illuminant temperature and tint. When you use the camera's "As Shot" white balance, the raw converter will display the corresponding temperature and tint. That is if you change profile to one with a different ColorMatrix the "As Shot" temperature/tint will change even if the multipliers are exactly the same. Ideally the temperature/tint should of course show the "truth", the actual correlated color temperature of the illuminant for that white balance but it's an approximation that may differ quite much between profiles, for a temperature around 5000K a variation of several hundreds of degrees is normal. Naturally a profile is best at estimating temperatures close to the one that was used when the profile was made.

If you instead of using the "As Shot" white balance selects a different one with temperature and tint, the ColorMatrix is used to calculate the corresponding white balance multipliers, at least for Adobe Lightroom (other raw converters may use a hard-coded white-balance model rather than using the profile-provided ColorMatrix). This means that if you change profile to one with a different ColorMatrix the temp/tint will in this case stay the same but the actual multipliers will change and thus the actual visual appearance, that is you get a shift in white balance.

A DNG profile contains the calibration illuminant as an EXIF lightsource tag, that is there is a limited set of pre-defined light sources to choose from. For a single illuminant DNG profile this tag is not used though, so it can be set to anything. If you provide DCamProf with a custom illuminant spectrum during profiling the resulting DCP will contain "Other" as lightsource tag, that is no information of what temperature the profile was designed for, but as said it's no problem.

However if you don't provide the spectrum and instead provide the completely wrong illuminant, say you shoot the target under Tungsten but say to DCamProf that it's D50, the calculated color matrix will be made against incorrect XYZ reference values and the resulting profile will be bad at estimating light temperatures. For single illuminant profiles that still won't affect the color correction though.

Dual-illuminant profiles is an exception. In that case you have two matrices, usually one for StdA and one for D65. Both these are then used to calculate the temperature and tint, and the derived temperature is then used to mix the two ForwardMatrices, that is if it's exactly between the 6500K of D65 and 2850K of StdA the 50% of each is used. This means that the temperature derivation has some effect on the forward matrix and thus some effect on the color correction. So if you intend to make a dual-illuminant profile it's required to provide a proper EXIF lightsource for each, and for the profile to make accurate temperature estimations the actual lights used during profiling should match the EXIF lightsource temperatures as well as possible. It doesn't have to be exact though as any reasonable camera should have similar matrices over at least some temperature range.

Note that a DCP profile cannot be made to "correct" white balance, that is change your "As Shot" white balance multipliers to something else. In some reproduction setups you may want to do that, and for this you need to use an ICC profile instead.

If you design your own profile with DCamProf and use it in Adobe Lightroom for example it's as discussed highly likely that you will get a white balance shift compared to the bundled profile. This doesn't mean that there is something wrong with your profile, but simply that your calibration setup and matrix optimizations did not exactly match Adobe's. If you want to apply your profile that previously used the bundled one with a custom white balance settings this white balance shift can be problematic though. Avoiding it is fortunately simple: just copy the color matrix from the bundled profile, which is possible directly in the make-dcp command. As the actual color correction sits in the forward matrix and LUTs, this change of color matrix will not affect your color rendition (except for the slight effect caused by the dual-illuminant mixing described earlier), you just get rid of the white balance shift.

ICC-specific white balance properties

ICC profiles have unlike DCPs no connection to the raw converter's white balance setting. When DCamProf calculates its native profiles it makes both a ColorMatrix and ForwardMatrix, but if you convert to a ICC profile only the ForwardMatrix will be used as it has no element to identify illuminant color temperature as DCPs have.

Raw converters that use ICC profiles have some other method than using the profile to figure out a suitable temperature/tint to show in the user interface. It may be using some hard-coded ColorMatrix, some hard-coded preset values or other model.

Normally ICC profiles are designed to not affect the user white balance, so when you change profile to an entirely different one the white balance will still not change (except for tiny changes related to correction of neutrals). However ICC profiles can change the white balance if designed for that. One application could be to make an ICC profile that changes the camera's "As Shot" white balance to match a specific light source used in a reproduction setup. DCamProf can make such a profile if you instruct it to, as described in the make-icc reference documentation. This feature is unique to ICC, you can't make it with DCP as the DCP design prohibits white balance alterations by the profile.

Color for extreme white balances

Raw converters are designed such that "white" should be white, and for extreme color temperatures (candle light, Nordic winter dusk etc) this is not the case. In such a case you will have to adjust the look creatively to taste, as the available color models do not support profiling those situations in any accurate way.

Tone curves and camera profiles

An example image rendered with a linear tone curve using an accurate colorimetric profile. The exposure has been increased to make the image easier to compare with the others that have tone curves (which typically have a quite strong brightening component).

This should be used as reference when evaluating accuracy of colors. However, it does look flatter than the eye experienced in the brighter real scene, which is a normal appearance phenomenon. This means that we need to apply some sort of curve even when we want a neutral realistic look.

Same profile, but now with the DNG default curve, which is a modified RGB curve. Note the garish colors. Light colors are also desaturated, not so easily seen in this picture although the white shirt has lost much of its original slight blue cast. Desaturation issues can more clearly be seen in light blue skies for example.

DNG uses a hue-stabilized RGB curve (constant HSV hue) so it's better at retaining hue than a standard RGB curve (which most ICC-based raw converters use).

Same profile, with the tone curve applied on luminance only, while hue and saturation are kept constant. As luminance channel the J of CIECAM02 Jab is used here, similar to the more well-known Lab.

Intuitively one may expect this to be truest to the original, but as seen it looks desaturated. This is because in human vision color appearance is tightly connected to scene contrast, so if you increase contrast also saturation must be increased to keep the original appearance.

Same profile, here with DCamProf's built-in neutral tone reproduction operator. Color appearance is now very close to the original linear curve, but we have increased the global contrast so the photo displayed on a screen appears truer to the real scene.
Adobe Camera Raw's profile with the intended tone curve (same as in the others). Looks pretty natural, but some issues with saturated colors; too saturated reds and too little saturation on the purple and bright yellow-green. Additionally, skin-tones are slightly over-saturated and yellowish, and again the slight blue tint of the white shirt has been lost.

While some errors can be side effects of the curve, they're mainly deliberate subjective adjustments by Adobe's profile designers with the purpose to achieve a designed "look", like films had in analog photography. DCamProf's tone-curve operator intends to stay true to the color appearance of the original scene and leave subjective adjustments to the photographer.

Adobe Camera Raw's profile with linear tone curve. Here we can clearly see that it's not a "scene-referred" profile. The profile has been adapted for the S-shaped DNG tone curve and is therefore desaturated.

Note that comparing all these pictures may be hard directly on this web page as color shifts slightly with viewing angle. To critically compare then download the files, look straight at them while flipping through them in an image viewer. The images were made during development so the result from the current may differ a little, but you should from these images get an idea what the typical differences are and how large they are.

The tone curve is perhaps the most under-estimated factor in camera color reproduction. Here's why I think it deserves much attention: Per default a profile is designed to produce accurate colors with a linear tone curve. This makes sense as cameras match colors better if there is a linear conversion, especially when making a matrix-only profile which by nature is linear. In other words, a linear profile strives to make the camera into a colorimetric measurement device.

A linear tone curve is the right thing for reproduction work, for example when we shoot a painted artwork and print on corresponding media. In this case the input "scene" and output media have the same dynamic range and will be displayed in similar conditions. However in general-purpose photography the actual scene has typically considerably higher dynamic range than the output media, that is the distance between the darkest shadow and the brightest color is higher than we can reproduce on screen or paper.

The solution to this problem since the early days of photography is to apply an S-shaped tone curve. In film the curve compress highlights and shadows about equal (a sigmoid curve), while in digital photography there's been a shift to compress highlights more than shadows, which also brightens the image about a stop or so as a side effect. This suits digital cameras better as it retains more highlight detail. The principle is the same though, that is increased slope at the midtones with compressed shadows and highlights.

The need to compress highlights and shadows is obvious (otherwise we would not fit the scene's original range on the lower dynamic range available on screen), but do we really need to increase midtone contrast? The usual explanation is that the output media has lower contrast than the real scene and thus we need to compensate to restore original contrast. While this can be said to be true for matte paper, a calibrated screen will produce appropriate contrast for midtones. It surely cannot shine as bright as the sun and (probably) not make shadows as dark as in real life, but midtone contrast is accurate. In typical workflows we create the image first for the screen and then make further adaptations for prints (screen to print matching is a separate and well-documented subject), so when it comes to camera profiles comparing with screen output makes most sense which we will do here.

If we increase the midtone contrast with our tone curve, we will exaggerate. For a typical curve type this is mainly seen as increased saturation, as increased contrast separates the color channels more which leads to more saturation. Okay, so this is wrong then? Well, it's not that simple. Let's say we display a shot of a sunny outdoor scene. Although midtone contrast on the screen can be rendered correctly, the overall luminance is much lower. This makes the Stevens and Hunt color appearance phenomena come into play, that is the brighter a scene is the more colorful (=saturated) and contrasty it appears. That is to make the displayed photo appear closer to the real scene we need to increase both lightness contrast and colorfulness, which an S-shaped tone curve does for us.

So then all is good with the tone curves applied by typical raw converters? No. In fact if we're into a neutral and realistic starting point it's sort of a disaster. Most converters apply a pure RGB curve which has little to do with perceptual accuracy. Lightroom and many DNG raw converters apply a slightly different RGB curve that reduces hue shift problems (HSV hue is kept constant), but it's still in most situations almost identical in look to a pure RGB curve. It varies between converters in which RGB space this curve is applied, which also affects the result. In Lightroom/DNG it's always applied in the huge linear ProPhoto color space, while in many ICC raw converters it's applied in a smaller color space.

Let's start with the RGB tone curve problems. It will increase saturation more than is reasonable to compensate for Stevens and Hunt effects, so you get a saturated look. You might like that, but it's not realistic. Another problem is that for highly saturated colors one or more channels may reach into the compressed sections in highlights or shadows and that leads to a non-linear change of color, that is you get a hue shift. Typically the desired lightening and desaturation effect (transition into clipping) masks the hue shift so it's not a huge problem, but it's there.

Then there is the color space problem. If the RGB tone curve is applied in a large color space such as one with ProPhoto primaries (like in the DNG case) one or more channels can be pushed outside the output color space (typically sRGB or AdobeRGB) so we get clipping and thus a quite large hue shift. Some raw converters partially repair this through gamut mapping (Lightroom does), but still there may be residual hue shift.

To battle the various RGB tone curve issues bundled profiles typically have various subjective adjustments to counter curve issues. For example the profile may desaturate high saturation reds to avoid color space clipping. Naturally this means that the same profile used with a linear curve will produce too little saturation in the reds. That is a profile must be specifically designed for the intended curve.

I think this is bad design. In fact one could argue that staying with RGB curves (and similar) has inhibited the development of good profiling tools and makes it unnecessarily hard to get natural colors in our photos.

It doesn't have to be this way, the RGB tone curve is legacy from the 1990s when its low computational cost was one of the reasons to use it. It can also be seen as a nostalgic connection to film photography. In the film days the film had to produce the subjective look too, so exaggerated contrast and saturation were desirable properties. This thinking has been kept in most raw converters today despite that we have all possibilities to start off neutral and design our own look rather than relying on bundled looks. The RGB tone curve produces a saturated look that many like to have in their end result, but as said it still doesn't work well for profiles that aren't specifically adapted for it. Using a DCamProf neutral linear profile and applying and RGB tone curve will produce a garish look.

Tone reproduction operators

In the research world the problem of mapping colorimetric values from a real scene to the limited dynamic range on a screen or print is well-known and is the subject for many scientific papers. The scientific term for the "tone curve" that compresses the dynamic range to fit is "tone reproduction operator", and can instead of a simple global tone curve be scene-dependent and spatially varying, what we in the photography world call "tone mapping".

In science the goal is generally to make an as exact appearance match as possible, for example if we have shot a scene in very low luminance level (at night) also the eye's night vision with its desaturated color is modeled. Modeling all aspects of human vision at the scene and at reproduction becomes complex and is still a very active area of research.

Current raw converters are not designed for this type of advanced appearance modeling and it's generally not what a creative photographer is interested in. For example, in night photography we typically want to make use of the camera's ability to "see" more saturated colors than our eye can.

There is a middle way though. While we do want to increase contrast and don't really mind that it will be more than realistic for scenes not shot in bright sunlight, RGB tone curve color shifts are not beneficial. That is the tone reproduction operator we want for general-purpose photography is a basic S-shaped tone curve just like in traditional photography but without color shifts. This middle way has not got much attention in the research world though. Once computers got powerful enough researchers moved away from the "simple" tone curve models into tone mapping.

While tone mapping is useful in many cases, it's better handled separately in practical photography. It doesn't replace the need of a tone curve-based operator, it's just a complement. There is no widely used "standard" operator with this property though, so I had to come up with an own for DCamProf.

DCamProf's neutral tone reproduction operator

With DCamProf I've chosen the approach to render accurate neutral linear profiles (scene-referred), and then develop a new spatially uniform tone reproduction operator that doesn't have the hue shift and over-saturation problems of the commonly used RGB curve. This means that the profile can be developed just like a "reproduction profile" and no subjective tuning is required to adapt for the RGB curve's issues.

This operator can be applied when generating a DCP or ICC profile so you can achieve the intended look in your raw converter.

It has the following properties:

Most will be satisfied with the defaults weights, but if you like the weights can be tuned manually by providing a JSON configuration file. See the ntro_conf.json file in the data-examples directory for a documented example (shows the default weights).

The operator makes no local adjustments, and as it's just a part of a camera profile it couldn't do that anyway. This means that only the curve is analyzed for contrast, and as an image can vary in contrast locally (for example a large flat blue sky has low contrast even if the curve is a steep S-curve) also the eye's perception of color vary a little over the image surface, and thus some areas may receive a bit too much saturation or too little. This is not a large problem, but something to be aware of when evaluating results.

DNG profile implementation notes

When making a DNG profile the operator is implemented through the LookTable and curve. So if you strip away the LookTable and curve you have the standard linear colorimetric profile.

The DNG profile LUTs are not as flexible as ICC LUTs, most notable is that you cannot alter grays, not increase saturation or change lightness (value). As the LUT work with multipliers on saturation it's logical that you cannot increase saturation from zero. However, it's not logical that value cannot be scaled. Some DNG profile implementations support scaling grays (as the LUT itself does support), but the public DNG reference code as well as Adobe's products ignore the value multipliers for gray and instead set them to 1.0, that is no change.

This means that you cannot implement a curve directly in the LUT, as grays cannot be darkened or brightened (which a curve requires). The workaround is to embed a tone curve (which can scale grays), predict the result of that curve and reverse the undesired aspect to get the intended result. This is how DCamProf does it. There is one potential problem though: it's not specified in the DNG specification how the tone curve should work, so there may be raw converters out there that does not use Adobe's hue-stabilized RGB curve variant and if so you will not get the desired output.

If you come across such a raw converter and want to use this tone reproduction operator, please let me know.

The LookTable will per default be gamma-encoded for the value divisions, this will make perceptually better use of the range (that is higher density in the shadows) meaning that the default 15 value divisions should be enough for most curves. Some older or simpler raw converters may not support the gamma encoding tag though, and if so you can disable it.

Verifying tone reproduction accuracy

Once you have applied a curve you can no longer do normal automatic Delta E comparisons to check for accuracy. By definition a curve adds a lot of lightness "errors" as it applies contrast, and we also add saturation "errors" to perceptually compensate for the increased contrast. The one-to-one delta E comparisons only work for linear profiles.

There are no readily available color science models top help us out here, so the only method at hand is to verify by eye.To do this you make a linear profile first that can be measured for accuracy and use that as reference. Then you copy an image so you have two versions one with the linear profile applied and one with the curve, and do A/B swapping. It's important to do swapping and let the eye adapt for a couple of seconds, if you would compare side by side the eye will be confused by the two different contrast levels displayed simultaneously.

Check that individual hues seems to be the same and look globally and see if saturation seems to match (if you look closely on one isolated color without seeing the global contrast, saturation should be a little higher for the curve profile).

A photograph with faces in it is one good reference point, as our eyes are very good at detecting subtle differences in skin tones. I also recommend testing a sunny outdoor landscape scene, where you can check if the applied contrast is suitable (that is look globally and get a feel if the scene looks as contrasty as in real life but without exaggeration). Check if the color of the blue sky seems right, hue shift of light tones is typical for simpler curves. I also recommend testing a photo with various high saturation colors which you can find in flowers naturally or as artificial colors for example in toys or sports clothing. High saturation testing is a bit difficult as you can run into color space clipping. Using a wide gamut screen will certainly not hurt in this case.

As mentioned in the description of DCamProf's neutral tone reproduction operator there are limitations with operators that can only apply a global adjustment without adjusting specifically for the image content (which all profiles must do). Keep this in mind when evaluating the result.

The goal DCamProf strives for is a neutral starting point even when a curve has been applied, and then you can subjectively add saturation to your liking in the raw converter.

Scene-referred vs output-referred

You have probably heard/read that "DNG profiles are scene-referred and ICC profiles are output-referred", and in the next sentence it's said that scene-referred is better. What does this mean?

A scene-referred camera profile simply means that the purpose of the profile is to correct the colors so the output represents a true linear colorimetric measurement of the original scene. In other words we want the XYZ values for the standard observer, or any reversible conversion thereof. That is what we in daily speak would call an accurate linear profile, which DCamProf makes per default.

An output-referred camera profile should instead produce output that can be directly connected to a screen or printer ICC profile and produce a pleasing output for that media. As discussed, for cameras this means in practice that there should be some sort of tone-curve applied to get a pleasing midtone contrast and compressed highlights. In other words if the camera profile converts to XYZ space, those XYZ values should already have the curve applied and also any other subjective adjustments.

It's true that the ICC standard is written such that it expects camera profiles to work this way. However, raw converters that use ICC profiles don't necessarily follows this intention. Some let the ICC profile make a scene-referred conversion, while some makes some sort of mix between scene-referred and output-referred (let it do subjective color adjustments, but not apply a curve), and only a few do it the ICC standard way and make the ICC profile fully output-referred.

While DNG profiles can be 100% scene-referred, they can also have a "LookTable" LUT and/or a tone curve which are subjective adjustments for output, effectively making the profile output-referred. Adobe's own profiles have these type of adjustments, and are thus output-referred. I think scene-referred vs output-referred is a bit confusing concept as DNG profiles supports both things natively and ICC profiles do it in practice depending on raw converter design.

To support all-around use of scene-referred profiles the raw converter must have a type of tone reproduction operator that can change contrast without distorting color, otherwise scene-referred will only make sense with the linear curve. Of the big name raw converters few (none?) have such an operator but instead require profiles to be adapted for a curve. This is why DCamProf supports applying its own tone reproduction operator directly in the profile; raw converters in general are simply not up to using scene-referred profiles with their internal tone curves.

Tone curves and matrix-only profiles

To compensate to get good color after an RGB tone curve has been applied the profile needs to make non-linear adjustments. This is not possible with matrix-only profiles as they by nature are 100% linear.

However, a matrix profile made to match a matte target, such as the classic CC24, will most likely produce too low saturation of high saturation colors, and will thus produce a less garish look together with an RGB tone curve than a colorimetric LUT profile would (which can accurately reproduce high saturation colors as well).

It's generally not a good idea to try to get good match of high saturation colors for a matrix profile in any case, as that will reduce precision of the more important normal range of colors. That is a good matrix profile is generally a bit desaturated and therefor works okay (although not perceptually accurate) together with an RGB tone curve in most circumstances.

DCamProf does not provide any functionality to adapt matrix-only profiles for tone curves (unless you add a LUT on top which you can for DNG profiles), so if you intend to use your matrix profile with an RGB-like curve make sure you design it with not too high saturation colors.

The look of over-exposure

Digital cameras clip the raw channels straight off when over-exposed which may not result in a pleasing look, even with a roll-off in the profile's tone curve. So some raw converters apply some special rendering of over-exposed shots to simulate a more film-like behavior, mostly by desaturation.

This is not standardized and cannot be controlled by the camera profile. There should be no need to do so either, but it's good to be aware of this if you compare output of the same camera profile in two different raw converters. If the shot is over-exposed the raw converter itself may affect the look. Naturally if you lower exposure of a clipped image the raw converter's highlight reconstruction algorithm will affect the look, which also is outside the scope of a camera profile.

Chromatic adaptation transforms in camera profiling

If the light of a scene changes from say a blueish daylight (D65) to a reddish tungsten light (StdA) and we give some time for our eyes to adapt the colors will still look approximately the same. This is the eye's chromatic adaptation, and the phenomenon that colors appears the same despite a new light color is called "color constancy".

However, the eye is only approximately color constant, that is some colors will appear slightly different under the new light. In color science the chromatic adaptation behavior of the eye/brain has been tested with various psychophysical experiments where test persons match colors under different lights, in order to find "corresponding color sets". The corresponding color under a different light can be a different sample, which is an example of "color inconstancy".

These experiments have then served as basis when developing chromatic adaptation transforms, CATs, mathematical models of the human vision's chromatic adaptation behavior. A CAT thus models both the color constant and the inconstant parts of adaptation.

A CAT does the following: provided a CIE XYZ tristimulus value under a source illuminant, predict what the XYZ tristimulus value should be under a destination illuminant that provides the same color appearance. The illuminants are given as whitepoints, so the CAT does not need any spectral data.

In camera profiling a chromatic adaptation transform is needed when the calibration illuminant is different from D50. The reason for this is that the profile connection space is always D50 (for both ICC and DNG profiles), that is the color rendering pipeline in raw converters need the profile to output colors relative to D50, which then can be converted further to colors for your screen or printer.

If the profile is made for say tungsten light (StdA, 2850K) we then need to convert those XYZ coordinates to corresponding colors under D50. This can be made with a CAT, and the current best for these tasks is the CAT coming with the CIECAM02, CAT02. However, the CAT is still far from perfect. There are challenges concerning the accuracy of the experimental data they are based on, and the experiments also cover limited illuminant range (usually StdA to D65) and limited range of colors. In addition the CATs are designed with various trade-offs to make them easier to use mathematically. And finally, these transforms work on tristimulus values only, of both colors and illuminants. Any knowledge of spectral information won't contribute.

"Relighting" transform

There's another type of chromatic transform too which sometimes is needed in camera profiling. Let's say we have the XYZ value under D50 for a test target patch, and we want to predict which XYZ signal we will get from the same patch lit under StdA. That is we're relighting the patch. If we have the reflectance spectrum of the patch and the destination illuminant it's straight-forward, we just calculate the new XYZ values the normal way with spectral integration.

However some reference files provided with commercial test targets only have XYZ coordinates, and if we don't have a spectrometer to measure the target ourselves then we need to make a transform without having any spectra at hand.

This transform is not the same as a CAT. A CAT finds a corresponding color and models the color inconstancy aspects of human vision. However as human vision is approximately color constant many software applications use a CAT anyway when a relighting transform is called for, and there's not much else to do as the established color appearance models do not provide any other transform. There is no standardized name for the "relighting transform" which means that CAT is sometimes used in the literature also for this which causes some confusion. In this documentation "relighting transform" will be used.

With DCamProf there is a better alternative for relighting than using a CAT. If the reflectance spectrum is missing DCamProf can generate a virtual spectrum which matches the given XYZ coordinate, and that spectrum can then be lit by any illuminant. Of course the rendered spectrum will not exactly match the unknown real spectrum, but tests made on various sets show that for most colors this method outperforms both Bradford CAT and CAT02. Rendering virtual spectra often gets you within 1 DE from the correct answer, while the CAT is often in the range 2-4 DE.

Performance of CATs

The performance of a relighting transform is easy to verify as long as you have spectral data, and there are plenty of databases with various spectra.

With a CAT the only data to verify against is the correlated color experiments made, and CAT02 generally wins when it comes to the established models. However, as discussed all of these models are rather approximate, and the question arises that maybe they introduce more errors than they fix? A CAT02 conversion from StdA to D65 will have about 3-4 DE on average compared to the correlated color set experiments. Performance is probably not so good outside the StdA to D65 range as the reference experiments does not cover a wider range than that.

It would be most interesting to compare CAT with simple spectral relighting, as the latter is usually available when profiling. When using the relighting transform as a CAT we assume perfect color constancy, which indeed is wrong, but on the other hand the error will be no larger than the range of color inconstancy, which presumably is quite small. Unfortunately the correlated color experiments don't have spectral data so there is no way to make this comparison. What we can see though is that relighting is about 3 DE on average from CAT02, with up to 6-7 in saturated reds and yellow-greens.

From these results a fair guess is that a CAT is indeed better at predicting the color inconstancy aspects of human vision than just keeping perfect color constancy (that is do relighting from spectra), but also that relighting may be more robust and may have smaller appearance errors in some ranges.

When are CATs and relighting used?

If you make a D50 profile and have D50 XYZ target reference values no CAT or relighting is required. If you like you can make a D50 profile even if the actual light used when shooting the target is not D50. What then will happen is that the look will be as if lit by D50, but the profile will only work as intended in the light used at shooting time (if you make a DCP it's light temperature estimation will be off too, but that does not hurt performance in any way).

DCamProf needs target reference values as illuminated by the calibration illuminant (= the light the target was shot under). Why? There are two reasons, one is to calculate the color matrix which is used in DNG profiles to estimate light temperatures, and the other is to know the color appearance under that light so we can using a CAT get corresponding colors for D50 (the profile connection space) which the color correction is made for.

As the reference file is often calculated for D50 a relighting is often required. If spectra is available in the target file this is done by spectral calculation and thus very accurate results is had. If spectra is missing a relighting transform has to be applied.

DCamProf also needs D50 reference values, as this is the profile connection space where the color correction matrix (the "forward matrix") and LUT work. If the actual look of the calibration illuminant should be retained we need to model also the color inconsistency aspects of human color vision and then a CAT is used, so we take the reference values calculated for the calibration illuminant and transform those to D50 via CAT.

With DCamProf you can if you want force color constant behavior and then D50 values will be calculated via relighting rather than CAT, assuming target spectra is available. If you are making a reproduction profile this is likely what you want.

Note that if we don't make a DNG profile, or we don't care about its ability to estimate light temperatures, and we rather use color constant behavior than using CAT, the reference values for the calibration illuminant won't matter.

Summary:

Testing CAT-designed profiles

If CAT02 was employed when designing the profile, for example to keep the color appearance of colors under tungsten light, you should test the profile with the same criteria. Using DCamProf's test-profile command you can just mirror the parameters from make-profile. If you use some external software for testing it's likely that it will not apply a CAT and rather expect perfect color constancy. In that case you should either not use that software for testing, or redesign your profile with the -C flag, that is disable CAT.

Subjective looks in camera profiles

The camera profiles bundled with commercial raw converters generally don't try to reproduce a neutral accurate color appearance, but instead apply a designed look. The central aspect is of course that they apply a tone curve, as discussed separately in the tone curve section, but the appearance of colors are also adjusted with the intention to produce a more "pleasing" result than an accurate profile will. It can be about rendering smoother and less reddish caucasian skin-tones and more saturated colors to make images "pop".

This is very similar to how color films worked, few films had very accurate color but instead different types of subjective color that could suit more or less well depending on subject. Contrast (tone curve) differed between films too. That is one can say that the commercial camera profiles builds on the film tradition. Although we with digital technology could design the look separate from the profile (using the raw converter adjustments, or a photo editor), the traditional way with preset looks is still alive and well.

It differs between raw converters how these subjective profiles are grouped. The illuminant selection (typically tungsten, flash and daylight) is not about subjectivity but about adapting the camera response to a light source, but it's often a part of the profile choice unless it's automatically derived from white balance (dual-illuminant DNG profiles have it built-in). Then there's often a subjective choice depending on intended subject, "portrait", "product" and "landscape" are common genres. Sometimes the tone curve is integrated into the profile (lower contrast for portrait, higher contrast for product and landscape), or you can select it separately. As the tone curve affects color appearance I think it's better to have it integrated in the profile, so you know the designer have had full control over the end result.

In any modern raw converter you can as a user make many different color adjustments, as well as contrast adjustments. So why should the camera profile make these adjustments? Wouldn't it be better if the camera profiles just was as accurate as possible and then you as a user would choose color and curve adjustments using the readily available tools in the raw converter?

Well, first there is tradition as we already discussed which probably is the strongest reason why profile design has stayed this way. Choosing a profile is like choosing a film which renders the scene with colors and contrast in some way you prefer. But it's also non-trivial to make these subjective color adjustments, which is another key reason to provide the user with presets. Profiles don't have simple global adjustments like pulling the saturation slider, instead there are subtle adjustments here and there, to make skin color look flattering, slightly increase separation in foliage etc. They may contain lightness-dependent hue adjustments ("hue twists") for example make shadows more saturated and cooler (bluer) and highlights warmer (redder). We also know that adjusting contrast will change color appearance in ways which can be difficult to compensate. The average user may not have the skill or interest to do these type of fine-tunings. Of course the raw converter could still separate look from the profile by having look presets it would apply on top of an accurate colorimetric profile (which I personally think would be a better design), but few if any raw converters work that way today.

In fact, few raw converters actually have adjustment tools that allows for making the typical fine adjustments you find in profiles. Capture One has the "Color Editor" which is useful for some of these tunings, but Lightroom for example is quite limited in this regard.

When it comes to companies that produce both cameras and raw converters like Phase One and Hasselblad (and well, most other camera manufacturers too, but medium format makers color rendition stand out at least in terms of reputation), the profiles with their subtle subjective adjustments are part of their trade secret that sells cameras. While the camera hardware does play a very important role in how colors are rendered, the camera profile makes the largest difference and is thus very important in differentiating from the competition. The camera makers would probably not like to put this responsibility on the user.

So the reasons we have these profiles are because it's tradition, it's a way for camera and raw converter makers to differentiate, and as it's quite difficult to make the subtle adjustments yourself, so to most it's just easier if you get a preset look from the profile.

Should your custom profile apply a subjective look?

When you make an own profile using DCamProf you will per default get a profile designed for perceptual accuracy, and not get those fine-tuned subjective adjustments existing in typical commercial profiles. When applying a curve DCamProf will through it's neutral tone reproduction operator keep color appearance as true to the original as possible.

Is this a problem? Shouldn't we have some adjustments for skin tones and other subjects? Well, it's up to you to decide. First it should be noted that the neutral tone reproduction operator already does some of the adjustments you would expect, overall saturation is increased, saturation is increased in shadows, dampened for high saturation colors etc. This is not to make a look, but to compensate the appearance changes caused by the contrast curve, and I'd say that this is the most important aspect of the "subjective" adjustments you find in the bundled commercial profiles too.

If you want further adjustments that actually changes the appearance of colors depends on what type of subjects you shoot, what type of workflow you have and how much control you want during the workflow. If you shoot portraits with caucasian people you will probably want to adjust many of them to contain less red, and maybe even out the hues. You'd probably want to make a bit different adjustments from time to time, but still you may be helped by using a profile that has some skin tone adjustments built in to give you a better starting point. In that case you may want a specific "portrait" profile.

Don't forget though that any subjective adjustment in a profile will be global, so if it for example adjusts "skin tones" it will change any skin-like colors even if on entirely different objects. If you instead edit in Photoshop or similar application there are selection tools to isolate actual skin in the frame so you can modify only that, which of course makes more sense but does require time-consuming post-processing work for each image.

Also note that skin tones vary a lot person to person, and also varies depending on light, make up, tanning etc. Naturally this means that a profile that's good for one type of condition may be less good for others. Still some commercial raw converters have one subjective look that is supposed to suit any subject (Hasselblad's "Natural Color Solution" for example). If the profile makes quite small deviations from accuracy it can work quite well, but it should still be seen as a compromise.

If you do apply heavy manual post-processing to achieve a specific look it probably doesn't make much sense to have a subjectively fine-tuned profile from start, as no trace will be left of the original look anyway. Then I would prefer to get a neutral starting point so I would have an accurate baseline to start from, so I actually know what appearance changes that have been made.

A profile with a designed look is of course put to best use when you don't make much adjustments at all, or just smaller adjustments. If you have hundreds of images from a wedding a profile with some skin tone optimizations would probably not hurt. Also if your raw converter lacks tools to smoothen skin-tones you may want a profile that does that for you. You may also simply like the concept of selecting a preset look depending on subject, like having a portrait, landscape and a product profile.

So if you want a neutral profile or one with a designed look depends mainly on how you want to work, and to some extent also on the capabilities of your raw converter.

Designing your own subjective look

With DCamProf you can optionally design a subjective look and put into the profile. This is not an easy task, especially as DCamProf has no graphical user interface, but if you have a good bit of patience it can be done.

Here's a few examples of subjective adjustments you can find in profiles:

There are more things too, and there's no "right" set of adjustments. There are huge variations between manufacturers how they do it, just look at how differently the same camera look between different raw converters. If you are uncertain of what you like yourself you just need to experiment and don't be too nervous about it. As there haven't been much tools available to make profiles there's a lot of romanticizing of various raw converter's abilities to make great color. It's not that hard, and it's certainly not guaranteed that the manufacturer's taste concerning which adjustments that should be done or not is better than yours. The manufacturer often try to design a look that will impress the average user, and if you're into profiling your own camera you're probably not one of those.

When you develop your look it can be worthwhile to first produce a set of TIFF files of representative test images generated with other profiles you like (or don't like) so you have something to compare against.

In general, and especially when it comes to skin tones, I can recommend studying the subject of color correction. Not the least you will see things that a profile cannot and should not do, local adjustments, or adapting to conditions specific to the image. For example if a person wears bright colored clothing this can affect the tone of the skin, and naturally a profile that corrects for that will do bad in other conditions.

Handling extreme colors

When you make a camera profile for reproduction work you don't need to worry about clipping or colors that are outside the gamut, as you're using the camera as a scanner and you simply avoid both clipping and strange colors.

A general-purpose profile however needs to render gracefully into clipping and also handle "extreme colors" well.

What is an extreme color? I define this as a color that triggers a camera response that according to the profile corresponds to an impossibly high saturation.

When you profile the camera using a target, say a 24 patch matte color checker, a linear matrix will be created that matches those as well as possible and the match is then further refined with a non-linear lookup table (LUT).

Here's an example matrix for a real camera: 

  CIE X = R * 0.766 + G * 0.221 + B * -0.023
  CIE Y = R * 0.267 + G * 1.016 + B * -0.283
  CIE Z = R * 0.015 + G * 0.140 + B *  0.951
A representation of the "human eye's response" (CIE XYZ) is put together as a combination of the camera's raw RGB channels. The matrix is those nine constants. Within the range of a matte target like a CC24 the match will be quite good, a LUT will only do small refinements to an already good match. We can see something interesting in the matrix though: look at the blue channel especially for Y (luminance) output. As the camera has a broader/higher sensitivity than the eye in the blue range we actually need to subtract blue to get a good match. This is typical, although the value in the example (-0.283) is a stronger negative factor than for most cameras (the example comes from a Sony A7r-II).

Say that we get a raw color with zero on red and green and maximum on blue, then we actually get negative CIE Y output from the matrix, which would be clipped to black. In theory this would not be a problem as any normal colors would not trigger such raw channel combination. The matrix was optimized for a set of real colors and none of those comes close to outputting a negative CIE XYZ component. However, you can come across colors that trigger "strange" raw response, such as artificial narrow band lights that you can see in nightly cityscapes. Artificial emissive light sources in general are often problematic, and the deep blue range is typically the worst.

In this extreme range the difference between the CIE XYZ observer and the camera raw values will be exaggerated and it will be impossible to create a linear match (a matrix) which at the same time makes a good match for normal colors, or even matches a wide range of different extreme colors. A non-linear (LUT) correction would most likely be unfeasible with strong and contradicting stretches. Simply put, it's not a good idea to try to make an accurate colorimetric match in this range.

If you use a matrix-only profile you will get negative values in the extreme range, and unless the raw converter has some special handling for this range it will be clipped flat, in the worst case to black but more common to a plain strongly saturated color with no tonality information left. This is perhaps the largest drawback of matrix-only profiles when it comes to general-purpose photography.

If you make an ICC or DNG LUT profile DCamProf will handle those extreme colors through gamut compression on the colorimetric profile level.

DCamProf's color-correcting LUT will only work within the range where the matrix produces sane output. Outside the valid matrix range a generic gamut compression becomes active. It's purpose is to retain tonality (varying tones) where the camera captures tonality rather than being "correct", as the profile and camera can't be correct in any colorimetric sense in that range anyway. Some clipping will still take place, but it's controlled and it doesn't kill tonality.

The reason some clipping must take place is to be able to make a reasonable "increasing" gradient from neutral to full saturation clipping. Although this clipping doesn't kill tonality, the optimal tonality would be retained if no clipping would take place. Unfortunately the only way to achieve this on some cameras (with extreme blue sensitivity) is to desaturate the whole profile so you get a "longer range" to play with. This can indeed be observed in some commercial profiles. I don't recommend doing this as it sacrifices performance in the normal range, but DCamProf allows designing this type of profile too. An example can be found in the section describing custom deep blue handling.

The output in the extreme range may differ slightly between an ICC and DNG profile due to the different types of LUTs the formats use.

Note that this compression always takes place and is separate from the more configurable gamut compression you can apply on top. The user-controllable gamut compression is about reducing the gamut further, to say AdobeRGB or sRGB. The amount of compression can be controlled though, with the -k parameter in the make-profile command.

The maximum gamut DCamProf will work with is the intersection between the observer locus and ProphotoRGB. This means that the Prophoto triangle has it's deep blue corner cut (as it's outside human locus), and some of the cyan-green of the locus is cut. This gamut can be further limited if the profile's matrix has a smaller output.

Cutting away some of the locus may hurt the applicability of DCamProf profiles in some scientific applications, but DNG profiles are already limited to Prophoto, ICC Lab LUT has some range limitations as well, and cameras in general cannot perform well in the extreme range so this is a deliberate design choice. This gamut limitation makes the tone reproduction operator and other aspects of the software perform better.

Another aspect of "extreme colors" is colors that are so bright that they clip. Looking at the matrix you can see that there are such combinations. Clipping is quite small though so it's not too hard to handle and doesn't require much tricks. However in the tone reproduction operator handling clipping can be a complicated task, depending on how it's implemented. In the old days when tone reproduction was simply a plain RGB curve, no clipping issues was introduced. However if you work in other color spaces and want to stay free of color shifts you will end up with more clipping issues as you can't just compress one channel more because it's closer to clipping (that will shift hue, just like an RGB curve). DCamProf's neutral tone reproduction operator faces this challenge, and solves them with various methods. At the time of writing it's a moving target so you'll have to turn to the source code to see how it ended up.

File formats

JSON

DCamProf uses JSON as a base for its own file formats. Open the files that comes in the data-examples directory in the DCamProf archive for documentation. The JSON parser in DCamProf has been modified to parse floating point numbers with maximum possible precision.

If you get a JSON format error of your hand-edited files it can be hard to figure out where it is, then you can use one of the online JSON validators like JSON lint.

Argyll .ti3 (and similar)

DCamProf reads Argyll .ti3 files produced by the scanin tool. Note that the Argyll .ti3 format is rich in features and DCamProf only cares about a subset of it. It expects to get RGB measurement triplets matched with XYZ reference values, and possibly (hopefully) spectral data.

DCamProf can also generate .ti3 files and will then add some columns specific to DCamProf. Files remain compatible with Argyll though as unknown columns are ignored.

The .ti3 format (or rather an even more reduced subset of it) is also used when you want to import spectral data when you make a target to be processed by camera SSFs. An example of this exists in the data-examples directory.

DCamProf can also understand formats similar to .ti3, such as files coming from Babelcolor's patchtool.

Argyll .sp

With Argyll spotread you can read ambient light to a spectrum file, and this can be fed directly to DCamProf as an illuminant.

Argyll .ti1

DCamProf make-testchart and testchart-ff commands uses Argyll's .ti1 format to specify a test chart layout.

DCP

DCamProf can read and write DNG camera profiles (DCPs).

ICC

DCamProf can read and write ICC version 2 camera profiles.

Text

DCamProf can import spectral databases as raw text data formatted in various ways using the txt2ti3 command (not to be mixed up with Argyll's command with the same name).

Command reference

DCamProf is a collection of tools built into a single binary. The first parameter specifies the command (tool) you want to run, then followed by command-specific arguments: 
  dcamprof <command> [command-specific flags] <command args>
If you run the binary without parameters you get a list of all commands and their flags. Run dcamprof -v if you just want to check the version.

The basic workflow is:

  1. Make a target file containing test patches with camera RGB and reference XYZ values, and preferably also the reflectance spectra. This is either done with Argyll from test target raw photos, or by using the make-target command to render values based on provided camera SSFs.
  2. Make a camera profile using the target file, using the command make-profile. This will output a generic profile in DCamProf's own JSON format.
  3. Convert a DCamProf profile to a standardized format, using the command make-dcp or make-icc.
  4. Optionally manually edit the result (copyright strings etc) by using the dcp/icc2json and json2dcp/icc commands.
  5. Optionally evaluate target matching performance using the test-profile command.
Additionally you can use the make-target command to generate new RGB and XYZ values based on your chosen illuminant and observer. This requires the full spectrum of target patches, and to make RGB values you also need the camera's SSFs. For convenience value re-generation is supported also directly in the make-profile and test-profile commands.

Here follows a description of each command available.

make-target

  dcamprof make-target <flags, with inputs> <output.ti3>
Make a target file which contains raw camera RGB values paired with reference XYZ values, and (optionally) spectral reflectance. The file format is Argyll's .ti3, with some DCamProf extensions.

If you're using Argyll for measuring a target you don't need to use this command, but you can still use it to regenerate XYZ values with a different observer for example (this requires that the .ti3 file contains spectral data).

If you have your camera's SSFs you don't need to shoot any physical target, then you render the .ti3 file from scratch using this command.

Overview of flags:

Built-in spectral data

DCamProf has a few spectral databases built-in. These come from freely available sources, see the acknowledgments for information. This is a good start which you can do a lot with, but I'm looking for more spectral data to include in future releases of DCamProf, so if you know of some good source please let me know.

Generated spectral data

DCamProf has a spectral rendering algorithm that can make reflectance spectra to match any given XYZ coordinate for the chosen observer and illuminant. It's sort of an impossible task as there are an infinite amount of spectra to choose from. In this infinite set DCamProf finds a smooth spectra which has similar properties to real reflectance spectra.

Although not a full substitute to real measured data it can be used for experiments, test profile performance, establishing a baseline or filling out for areas where you don't have real spectral data. And indeed, a profile rendered completely from generated spectra will work, try if you like.

You can generate spectra along the chromaticity border of a gamut and optionally fill the inside with grid of patches. The samples are always made as light as possible (as high reflectance as possible) for the given chromaticity. Extremely saturated colors are by necessity narrow-band and will thus be darker than less saturated colors.

The gamuts available are locus, pointer, srgb, adobergb and prophoto. Add "-grid" suffix, eg "pointer-grid" to create a grid. The grid spacing can be adjusted with the -g parameter. Gamuts with extreme or even out of human gamut colors like locus and prophoto will cause the spectral renderer to fail producing spectra on some chromaticity coordinates, this is normal.

Be warned that spectral data generation is very processing intensive. DCamProf uses OpenMP to process several patches in parallel on all available cores, but it can still take minutes to produce a grid, or even hours if it's really dense.

A generated reflectance spectrum made by DCamProf (blue) together with a measured spectrum from a real Munsell color patch (red). Both lead to the same XYZ coordinate when integrated with the observer's CMFs. That is this shows one example of two different spectra that produces the identical vision color.

The DCamProf spectral generator strives for smooth spectra, and its result is thus a little bit more rounded than the Munsell patch in this example.

Importing raw text data

Spectral data is often delivered in text files with simply the floating point values listed on rows. Much of the data in the spectral databases linked here is in such a text format.

The separate command txt2ti3 (not to be confused with Argyll's command with the same name) can be used to convert those raw text files into .ti3 that make-target can read.

The flags should be self-explanatory so just run dcamprof without parameters to get the information.

Example: import text spectral data (here from Lippmann2000 found in the spectral databases section) and form a target where cc24 fills out where the imported data don't have patches: 

  dcamprof txt2ti3 -a "caucasian" -s 1 -f 400,700,2 \
    Reflect_AllCaucasian_400_700_2nm.txt caucasian.ti3
  dcamprof make-target -p caucasian.ti3 -p cc24 output.ti3

Reflectance vs emissive spectra

The default type of spectrum in a target is a reflectance spectrum, that is how much of the light that is reflected at each wavelength. Most spectral data is of this type. Reflectance spectra is first multiplied with the illuminant to form emissive spectra which is then integrated with the observer.

It's however also possible to specify emissive spectra, that is light sources or reflective objects with an illuminant reflected off them. If you want to define a transmissive object such as a backlit leaf, you specify it as an emissive spectra, like filtered light source.

In the .ti3 file the column SAMPLE_TYPE says "R" for reflective spectra and "E" for emissive. This is a DCamProf extension and is thus ignored by Argyll.

Observers

The observer is a mathematical model of the eye, defining its spectral sensitivity functions, or color matching functions (CMFs). It's not intended to exactly match the eye's cone response, but to provide "equal" results. The observer's CMFs have been mathematically transformed to work better in real applications.

When you integrate these CMFs with a spectrum you get the corresponding CIE XYZ tristimulus value. That is the observer is key element in modeling what colors we see.

As there's no method to actually measure the signals the eye sends to the brain the CMFs have been derived based on results from color matching experiments. The precision is thus dependent on the color matching skills of the people involved in the experiment.

The original observer was published as early as 1931, and it's still the number one standard observer. This is not because it's the most exact one, but because the CIE standard organization will not accept new standards unless significant improvement is made. Some minor improvements have been made over the years, but the original 1931 standard observer holds up well enough.

There are 2 and 10 degree variants of observers. This simply refers to how large area of the eye the tested color patch covers. With the more narrow 2 degree angle the eye is slightly better at color separation, but the 10 degree generally matches real situations better. The 1931 is a 2 degree observer, and the first standardized 10 degree observer was published in 1964.

DCamProf contains a number of observers, you can see a list when running the command without parameters. I'd like to use the 2006 observer as the default one as it's more accurate than the original 1931, and I'd also rather use the 10 degree observer as I think it matches real situations better than the 2 degree. However, as most color management software expects a 1931_2 observer and all the common color spaces sRGB, AdobeRGB, Prophoto are defined with a 1931_2 observer I've chosen that as the default. Only experiment with changing observer when you have full spectral information though, as changing observers will change XYZ values slightly so you can't have a reference file with XYZ values for a different observer for example.

If you change observer note that evaluation of profile-making results must be made with the same observer otherwise you will get larger Delta E than you should.

To get desired results with a different observer one needs at some point transform to colors for the 1931_2 observer as both DCP and ICC requires that the profile provides colors relative to that. Currently this transform model is very simplistic in DCamProf so the results will probably not be as good as they could be. Therefore it's currently best to stay with the default 1931_2 observer.

Examples

Re-generate XYZ reference values with a new illuminant (D65) and observer (using the default 1931_2) for an Argyll-generate .ti3 file: 
  dcamprof make-target -I D65 -p argyll.ti3 output.ti3
Generate targets files from scratch using camera SSF and built-in database: 
  dcamprof make-target -c 5dmk2-ssf.json -i StdA -I D50 -p cc24 output.ti3
  dcamprof make-target -c 5dmk2-ssf.json -i StdA -I D50 -p cc24 \
    -p munsell output.ti3
Use the spectral generator to make targets from scratch: 
  dcamprof make-target -c 5dmk2-ssf.json -i 7500K -I D50 -g 0.01 \
    -p pointer-grid output.ti3
  dcamprof make-target -c 5dmk2-ssf.json -i D65 -I D50 -p pointer \
    -p srgb-grid output.ti3
Generate a border around the Pointer's gamut and use the reserved word "illuminant" to get the spectrum of the illuminant (D65 here) into the patch set, which is necessary as with only the border there would be no white patch: 
  dcamprof make-target -c 5dmk2-ssf.json -i D65 -I D50 -p pointer \
    -p illuminant output.ti3
...and then we do the same thing by using the reserved word "white" to get a perfect white reflective spectrum, which really is smarter as the reflective white will still work if we later change the illuminant: 
  dcamprof make-target -c 5dmk2-ssf.json -i D65 -I D50 -p pointer \
    -p white output.ti3
Re-generate both RGB and XYZ values from a previously created file which contains spectral information, use D65 for the RGB values and D50 for the XYZ values: 
  dcamprof make-target -c 5dmk2-ssf.json -i D65 -I D50 -p input.ti3 output.ti3
Assemble a target from imported spectra and built-in database: 
  dcamprof make-target -p input1.txt -a "class1" -p input2.txt -a "class2" \
    -p cc24 output.ti3
Note that in this last case there is no SSF provided and while the input text files might have RGB values, no RGB values can be generated for the built-in cc24, and the output will thus contain dummy values (zeroes) for the RGB triplets. That is to be used when making a profile you need to run it through again to re-generate RGB values with provided camera SSFs. For convenience the make-profile and test-profile commands support re-generation directly so you usually don't need to re-generate reference values separately with the make-target command.

If you are using Argyll source files it's preferred that you include spectra throughout the workflow so XYZ reference will be re-generated with the observer chosen in DCamProf. If the XYZ reference values comes without spectra from a source you cannot control it's important to know which illuminant (and observer, nearly always 1931_2) that was used so you can later inform make-profile of that.

make-profile

  dcamprof make-profile [flags] <input-target.ti3> \
    <output-profile.json | .icc | .dcp >
Make a camera profile based on an Argyll .ti3 target file, either generated by Argyll from a raw test target photo, or by dcamprof make-target. The target file contains test patches with raw RGB values from the camera coupled with reference CIE XYZ coordinates of the patches, and possibly also the spectral reflectance of each patch.

The output is written in DCamProf's own native format, which can be converted later on, or if you satisfy with default conversion flags you can directly write a DNG or ICC profile.

Overview of flags:

Illuminants

It's important that you get illuminants right in order to generate a correct profile. The .ti3 file format does not contain information on which illuminant that was used for the camera raw RGB or XYZ values. This means that you must keep track of that yourself and provide the information to DCamProf via the -i and -I parameters.

There are a few possible scenarios:

For optimal results you want to avoid the first case. That is provide a target with spectral information, and a calibration illuminant with known spectrum. Then all XYZ values will be re-generated from spectra. If the target lacks spectra you can choose to simulate them by enabling the -S flag. It cannot exactly recreate the original unknown spectra of course, but if DCamProf has to perform a relighting transform the results will generally be more accurate than if not using simulated spectra.

In the most flexible case you have the camera's SSFs too. In this case also the RGB values are regenerated for the calibration illuminant you choose.

If you lack camera SSFs the RGB values from the file will be used directly. It typically means that the file comes from Argyll scanin of your converted raw shot of a physical test target and will by nature contain the RGB values for the light that illuminated the test target at the time of shooting. In this case it depends on use case if it's important that the calibration illuminant you specify matches the real one or not, as follows:

DCamProf will need XYZ values for both the calibration illuminant and the "profile connection space" which always is D50 (same for ICC and DCP). A target file only contains XYZ values for one illuminant, and thus the other or both must be calculated. If there is no spectral information Bradford CAT will be used, which does not provide as precise results as when calculating from spectra. With the -S flag you can enable rendering of virtual spectra which often gives a bit better result than using the Bradford CAT.

If you have spectra the XYZ values will be generated for the calibration illuminant first, and then converted via CAT02 to the profile connection space D50, and it that case it's of course important that the calibration illuminant is reasonably truthful. The purpose of using CAT in this case is to simulate the minor color appearance differences that occur due to the illuminant. You can disable this behavior with the -C flag.

In any case if you shoot the target in for example outdoor daylight you don't need to worry if you don't really know the exact temperature, guess one of D50 (midday sunny) or D65 (midday overcast). If you have a spectrometer you can bring a laptop and use Argyll's spotread to read the spectrum of the light and find out what the correlated color temperature is so you get help to choose the closest one. You can actually feed the actual measured spectrum to DCamProf as well, which makes a difference if CAT is enabled, and will make the color matrix as accurate as possible.

Here's the Argyll command to use to read the illuminant spectrum: spotread -H -T -a -s

(If you run spotread with -S (capital S) you get a spectral plot for each measurement which can be interesting. It's a bit user-unfriendly though, the program may seem to lock up. You need to activate the plot window and press space to get back to the program.)

If you lack reflectance spectra in the target file the specified XYZ illuminant must match the ones used in the target. The values could for example originate from a target manufacturer reference file, and is then often relative to D50 or D65. Make sure to look it up so you can provide the correct one. Unlike the calibration illuminant this is really important that it's exactly right.

If you have measured the XYZ reference values yourself using a spectrometer you should have spectra in the target file. If not they have probably disappeared along the way, look over the workflow and see if you can provide DCamProf with spectral information.

Weighting

Making "the perfect" camera profile is much about finding a suitable tradeoff between smoothness and accuracy. This is controlled using the weighting parameters.

If you don't provide any weighting parameters DCamProf will generate automatic weights to make some suitable tradeoff between smoothness and accuracy (the automatic mode tries to be on the safe side regarding smoothness and rather relax the profile too much than too little). For the casual user this is probably enough, but if you want the ultimate control you can tune them yourself.

You can control two aspects, one is the matrix optimizer and the other is the LUT optimizer.

To assign different weights to different groups of patches the target file must be split into "classes" (=groups of patches), specified through a "SAMPLE_CLASS" column in the file. The idea is that you can have a naming such as "skin", "forest_green", "textiles" etc and then for example assign greater importance to skin-tones.

Class names in the target file is a DCamProf concept and is not available in Argyll-generated files. By running an Argyll file through dcamprof make-target -p argyll.ti3 -a name out.ti3 you can add a class column, and then edit the text file manually and change names to split into more classes if you like. That way you can split even a 24 patch color checker into several classes. However, the main purpose of class-splitting is to be used when you have a number of distinct patch sets of different spectral types as you typically have when making a target directly with a camera's SSFs.

If you don't have any class names (or all patches are in the same class) there's no value in providing a matrix weight. However the delta E relaxation for the LUT is still useful.

Finding the right weights is a trial-and-error process. For analyzing matrix results applying the profile to a colorchecker image brings you far. For the LUT dumping reports and plots (-r report_dir) and visualize the results is a good idea, see the section on report directory files for more examples. You can also look at the patch matching image files.

Let us start with discussing the matrix. This is the linear base for which the LUT applies non-linear corrections to. If the matrix is close to the ideal, the LUT needs to stretch less which makes it easier to manage. Relaxing the LUT makes gradients smoother and the result closer to the matrix. In other words it's a good idea to have the matrix close to your desired end result.

A matrix is by nature perfectly linear and thus have no issues with gradients (smoothness). It can however be more or less precise, and be more or less robust when it comes to extreme saturation colors. Controlling the matrix optimizer through weights is pretty crude, which is a side-effect of the mathematical optimizing process itself which is difficult to steer in very specific directions. You can split your target into classes and assign different weights to them using -w parameter. It's generally not that effective for fine weighting, such as differing between several normal range colors like preferring skin-tone precision over forest greens, the matrix optimizer is likely to find some similar "best" anyway. It can be more effective if you for example group high saturation colors (from a glossy target) in one class and normal saturation colors in another. For example you may want the matrix to be precise on normal range colors and worry less about high saturation colors and then you could set the weight to 0 for your glossy class.

DCamProf makes a pre-weighting per default (the user weighting is added on top), this is to handle the situation when you combine several patch sets with different density. Some patch sets may have lots of patches concentrated around some specific color, and another may have few patches widely separated. This is common when using spectral databases. To not cause the dense sets to totally dominate, there's a pre-weighting based on Delta E distances that normalizes all patches. This is generally a good thing, but if you really want "1 patch = 1 unit" you can disable this normalization by adding -W. This normalization only affects the matrix optimizer, the LUT optimizer only looks at the max acceptable delta E deviations (specified with the -l paremeter). There is little reason to disable it.

If you have a simple target like the colorchecker 24, you will probably not do any matrix weight adjustments at all, as it doesn't really change much.

However, you can also control the smallest values in the matrix. As discussed in the extreme colors section a matrix which matches normal colors well may get strong negative components and cause for example deep blues clip to flat blue or even black. It can also be used as a crude way to lighten colors. Although the LUT will counter-act and correct to get the same result regardless of the matrix, when you relax the LUT you will get closer to the matrix result. This smallest value is controlled with the -y parameter. If you want to use it a typical start value could be -0.1. Note that the matrix optimizer uses this as a guide, the actual result can be slightly different (that is it may break the -0.1 limit anyway). Read the section on custom deep blue handling to get some further information on how this can be used.

The default value of -y is -0.2 and will thus limit the matrix of some cameras. This can have a quite strong effect, nearly always showing as a lighter blue (can be seen on the C01 deep blue patch on a CC24). If you want to start off with an un-limited matrix, which can be a good idea when you experiment with weighting, provide a large negative like -y -5 to make sure the matrix won't be limited. Note however that the default value is there for a reason, cameras that are limited by this value is likely to perform in unstable ways in the deep blue range if it's rendered "on the mark".

If you get the issue that the matrix optimizer perhaps make your red colorchecker patch a little bit too magenta (probably due to a tradeoff with other patches), you could in theory create a separate class for that red patch and assign it a much higher weight. This is however not likely to work well. If you want to achieve precise hue-changing results of the matrix optimizer you should use the target adjustment configuration and simply reduce the magenta component of the reference value, possibly exaggerate to get the desired effect. Adjusting the target is normally not needed, but it's there for those that need precise control of the matrix result. You may want to use target adjustment only for the matrix, but not for the LUT optimizer. In that case you need to design the matrix first, and the LUT later. You do this by ignoring the LUT result first (use -L parameter in make-profile and make-dcp/make-icc commands), and when satisfied with the matrix you store it in a separate file and provide that to a new make-profile run with the -f, -m and -e parameters.

Another powerful way to affect matrix optimization is simply to remove patches. While you could cut it from the target file itself, it's generally easier and more flexible to use the -x parameters and provide a list of patches to exclude. Again this affects also the LUT optimizer, so if you only do this for matrix control you need to do separate runs. Removing patches usually have quite strong effect on the result, but quite hard to predict. I only recommend to remove patches if you have those that seem problematic (bad measurement etc).

Now we're done with the matrix part. Let's go on with the LUT. For "casual" weighting you generally don't do anything with the matrix, but you do apply some generic relax on the LUT to improve overall smoothness.

A LUT can always stretch, compress and bend to match the target patches exactly, but that can result in sharp and even inverted bends causing ugly gradient transitions (typically most visible in photos with strong out-of-focus blur backgrounds when one color transitions into another). In this case it's better to relax the fitting, and the LUT optimizer will automatically relax in the best way based on the provided acceptable Delta E ranges (in CIEDE2000).

This is controlled with the -l parameter and in the simplest case you just specify one number, like -l 2. This instructs the LUT optimizer that an error of 2 Delta E is acceptable, and relax the stretching towards the linear matrix either until reaching the matrix or the error reaches 2 Delta E. If you set a very large number the LUT will be able to relax so much it becomes identical with the matrix result.

You can also specify this per patch class, in this case you specify the name first and then the Delta E range(s), for example like this: -l skintone 1 -l glossy 4, assuming we have the class names skintone and glossy. If you do such naming you should have names for all patches so you can specify range for them all. Those that are not named will be kept at 0 (stretch to maximum accuracy).

Instead of just providing one number you can specify the range exactly in all three dimensions in order lightness, chroma (saturation) and hue. For example this configuration: 

  -l -1,4,-3,2,1.5
specifies that in lightness patches may be no more than 1 DE darker, but up to 4 DE lighter is okay; in chroma we say up to 3 DE desaturation is fine, but only 2 DE over-saturation, and hue range is specified only with one number and this is set to 1.5 DE in the example. Lightness errors are generally the least disturbing, easy to detect when doing A/B swapping tests but it doesn't look "wrong" if you just look at one picture isolated. However it can often be a good idea to not let patches become too dark as it will hurt tonality visibility, thus specifying a tighter range towards dark is a common strategy.

Most modern cameras have widely overlapping filters and are therefore naturally desaturated on the raw level. Pushing for more saturation is thus likely pushing the profile into more stretching. Over-saturated patches is also arguably more disturbing than under-saturated. Thus a chroma range with larger negative DE than positive is also a common strategy.

Generally we want hues to be as exact as possible, but if we don't provide any relaxation at all it will become hard for the LUT to relax also in the chroma direction, so setting some non-zero value is recommended.

You can disable correction alltogether on an axis, simply by setting very large DE values (say 100). Disabling the lightness axis is a common strategy as it suffers more from measurement errors and is more likely to disturb gradients than chroma and hue corrections.

Note that all this weighting stuff is only about matching the specific provided patches, the profiler can't magically make the camera work better. For example, if a camera is bad at separating green colors, it will still be bad even if the particular green patches in the target have been mapped correctly.

Color matrix, forward matrix and LUT matrix

You may have noted that I have adopted the DNG profile names of matrices also for the native DCamProf format. This is simply because the names are familiar. It doesn't lock the native format to DNG profiles.

The forward matrix which operates in D50 XYZ space using D50 as the reference illuminant is not unique to DNG profiles, it's used for ICC profiles too. A matrix-only ICC profile can be said to contain a forward matrix. As the conversion from the calibration illuminant to D50 is needed by both profile standards DCamProf has adopted the forward matrix.

The color matrix is however DNG-specific, it's used for estimating the temperature and tint of the scene illuminant. It won't be used when generating an ICC profile.

Looking at DCRaw internals we find the color matrix again though ("cam_xyz" in DCRaw-speak), it's using a D65 color matrix per camera to render its default colors. So you can use DCamProf to contribute color matrices to DCRaw or other software that use DCRaw-style matrices.

There's also an additional matrix called "LUT Matrix". This is also DNG-specific and corresponds to the best (=least bad) ForwardMatrix that fits withing the ProPhotoRGB chromaticities. This typically means a matrix with very low saturation and overall light and dull colors, but with reasonably accurate hues. This matrix is used when generating a DNG profile with a LUT, and then it's used instead of the forward matrix, and the LUT is used to stretch colors into appropriate positions. The reason for this is purely format-technical: while DCamProf's native format implies gamut compression of negative values from the matrix output there is no such thing in the DNG format which just clips them. By using this special matrix premature clipping is thus avoided. This is not required by ICC (Lab) LUT profiles as there is no pre-matrixing in that case.

White balance

You can control white balance settings with the -b and -B parameters. Per default DCamProf will make a profile which expects the white balance to be set by color picking the most neutral light patch. In some cases the target "white" is actually considerably less neutral than a darker neutral gray patch and that will then be picked instead as reference. If you are going to use the target as white balance setter for a scene it's safest to specify a specific patch as reference, you do that with -b.

The most accurate correction is however had if you let DCamProf optimize towards a virtual 100% neutral patch, this will typically place the ideal white balance a little bit off the real target white. As it's only about 1-2 DE it's really only of mathematical interest it should not make a visible difference in any normal circumstance. If you want to do this you enable the -B flag.

Note that this only affects the forward matrix (which is used for the color corrections), the values used for color matrix calculation will not be re-balanced as it doesn't make sense; the color matrix is not used for color correction but only for figuring out the light's temperature and tint and thus re-balancing its data would only reduce its precision.

If you're working with SSFs and virtual targets you probably already have a perfect white in the target and then this setting will make no difference of course.

Target adjustment

While you can affect matrix optimization a little bit with weighting, if you really want to adjust how it matches a color (typically hue), changing the XYZ reference values are the easiest and most powerful way to do it. You could change directly in the target .ti3 file but if we work with spectra (like we hopefully do) it's not really feasible.

Therefore DCamProf provides the option to provide a target adjustment configuration file in JSON format, a documented example is provided in the distribution. You can make global adjustments there without pointing out specific patches, but those will only take effect if there is an actual patch matching the changed area. If you have relatively few patches (like a CC24) the easiest way is generally to target individual patches.

In earlier revisions of the software the intention was to control the matrix optimizer with DE range specifications, like the LUT relaxation is controlled. However, due to the specifics of matrix optimization that method becomes much too unreliable and unpredictable. Adjusting the reference values works a lot better for this task.

So while you can make subjective adjustments this way, you could also do this to compensate some error in the process (maybe bad reference values), or if you just want to shift which colors the matrix optimizer matches best. If you're making a matrix-only profile in the end you make strong and "wrong" adjustments as long as the matrix optimizer result suits you. If you are using the LUT, the LUT will strive for perfect color match so then you don't want to have exaggerations in the target adjustment.

If you're into the subtle parts of custom looks, you should be looking to use look operators instead which is applied when you generate the DCP or ICC profile.

Deep blue handling

As discussed in the section about extreme color handling deep high saturation blues can be a problem, especially with certain cameras. For normal colors this is not a problem, but if you often shoot nightscapes where artificial emissive light sources can trigger strong blue response you may experience a problem with a normally designed profile.

You can diagnose your camera's blue sensitivity in the resulting ForwardMatrix: if the middle row value of the third column (raw blue multiplier for CIE Y output) is less than say -0.15 your camera is likely to have some issues in this range. To diagnose you need to make sure the Y is not limited when making the profile though, by setting the -y parameter to a large negative value. It's set to -0.2 per default which means that problematic cameras will render blues too light, but will get more robust profiles (the default is there to help casual users).

There is another reason to handle blues in a custom way: the eye is not very sensitive in the deep blue range, so it's harder to see tonal variations in that range. Therefore many commercial general-purpose profiles render deep blues much lighter than they are experienced in real life, which makes tonal variations more visible, which may be a more pleasing result.

Normally these subjective "look" adjustments are made using look operators, which indeed from a design perspective is cleaner: that is you develop a a profile which is as accurate as possible in a colorimetric sense, and on top of that you make subjective adjustments. However from a practical processing perspective it's sometimes better to introduce some subjective adjustments already at the colorimetric stage. Limiting the blue range and render it lighter is one such case, and the reason is to minimize potential clipping and gamut compression in the base profile. If the colorimetric has strong compression in the blue range, it's hard to restore using a look operator. That is by lightening blue in the colorimetric profile we have a better chance to maintain optimal tonality in the range.

There are two ways to control the blues. 1) you can limit the range in the matrix, forcing the optimizer to subtract less blue than it does when optimizing freely (use the -y parameter, for example -y -0.1). And 2) you can provide a target adjustment configuration file and lighten blue patches there. For the classic CC24 the C01 patch gives good control of deep blue.

The eye is more sensitive to greens and reds and camera matching is less problematic there, so you generally don't need to make this type of handling of other colors.

Here's a real-world example for a Sony NEX6 which is problematic in the blue range: 

  dcamprof make-profile -y -0.15 -a adjust.json cc24.ti3 profile.json
Here we have a target adjustment file too (adjust.json), we choose to make the blues a bit less red as it in this case makes the matrix even more robust: 
  {
    "PatchAdjustments": [
      {
        "Name": "C01",
        "ScaleRGB": [ 0.8, 1.0, 1.0 ]
      }
    ]
  }
That is we've reduced red (0.8) of the deep blue patch (C01) in the CC24. If you're doing any hue adjustment of deep blue reducing red and/or increasing green (that is pull it away from magenta towards cyan) is often a good idea. Unstable deep blues that get a tiny bit too much red quickly becomes a strong magenta, which is very much a different color than blue. The transition to cyan is less consipicious and in terms of look it fits in there more natural looking like an "over-exposed blue".

Profile-making tips

Do experiment! Learn how to use a plotting tool and plot results. To get a general feel of how profiling works in practice you can play around with one of the example camera SSFs, and then use the acquired knowledge and feel when you tune settings for your targeted camera (where you may not have SSFs).

What you will see is that there is no such thing as perfect result, and the farther from the white-point you get tougher it will be to compensate errors. While it can be fun to try to get a profile that works all the way out to the gamut limit it will hurt performance of common colors. It's generally better to maximize performance for colors you're actually going to shoot. Pointer's gamut approximates the limit of how saturated real reflective colors can be, colors outside that need to be represented by emissive (or transmissive) light like lasers and diodes. It's generally not worth-wile trying to get a good match outside Pointer's gamut. If you have the camera's SSF you can plot and see how well the camera can actually separate colors, you will probably see that there are some issues when it comes to extremely saturated colors, and no camera profile can compensate for that.

Consider that a perfect match to a specific color checker does not mean that the color precision is perfect, not even for those colors. It's only perfect for the particular spectra the color checker has, somewhat compromised by various measurement errors throughout the profile making process. Therefore I suggest to always apply some LUT relaxing to smoothen profiles at least some. As true perfection cannot be had, it's better to make sure color transitions are smoothly rendered.

If you see very large errors after matrix-only correction, say 10 DE or more, the LUT may get a too tough job and be forced to make extreme stretches than can make bad gradients and an unpredictable profile. One way to test a profile for robustness is to load it in a raw converter, show a color checker with many colors, and change white balance. If some color suddenly changes must faster than the others the LUT is probably making a strong local stretch at some point. Of course you can see this by plotting as well, but the white balance test is a good sanity check.

Modern cameras should get a decent match with the matrix alone, if you see large errors, such as 10 DE or more, it's likely that there is some wrong with your input data, such as poor lighting of the test target, glare, bad references values or reflectance spectra.

Make sure to check what the dynamic range test shows (printed in the console output when running make-profile). Example output: 

Camera G on darkest patch(es) is 9.8% lighter compared to observer Y.
  Y dynamic range is 4.78 stops, G dynamic range is 4.64 stops, difference
  0.14 stops. A small difference is normal, while a large indicates that there
  is glare.
In the above example there's only 0.14 stop difference, and up to about 0.25 should be okay (that is very small effect on profiling result). By using the -g parameter and providing a target layout description you can let make-profile model the glare to compensate. This is usually a good idea, but don't expect perfect results for high amounts of glare.

Note that you can only trust the dynamic range test result if the target has pure black patches. If the darkest patch is colored it's a large risk that the result is misleading.

Examples

In all examples below I assume the target file has reflectance spectra. If not you need to specify the XYZ reference values illuminant using the -I parameter.

Example 1: basic profile making with default parameters, using calibration illuminant StdA (calibration illuminant = the light source the target was shot under): 

  dcamprof make-profile -i StdA target.ti3 profile.json
Example 2: assuming we have a target with cc24 and pointer border, we make sure the matrix is more focused on matching the cc24 (weight 1) than the pointer border (weight 0.5). This sets up the matrix for requiring less LUT stretch for normal colors. Then we specify the LUT max acceptable Delta E ranges, generally accepting less darkening than lightening, and less over-saturation than under-saturation, and requiring better precision of cc24 than pointer. We specifically allow the pointer border to be quite desaturation (-4). By providing camera's SSF (-c) the RGB values will be re-generated for the given illuminant (D65). Plotting data is saved to the "dump" directory (-r). 
  dcamprof make-profile -r dump -c ssf.json -i D65 \
    -w cc24 1 -w pointer 0.5 \
    -l cc24 -0.5,1.5,-1,0.5,0.7 -l pointer -2,4,-4,1,2 \
    target.ti3 profile.json
Example 3: make matrices using one target, and the LUT using another by running make-profile twice: 
  dcamprof make-profile -i D65 target1.ti3 m.json
  dcamprof make-profile -i D65 -m m.json -f m.json -e m.json \
    target2.ti3 profile.json

test-profile

  dcamprof test-profile [flags] [target.ti3 | test.tif]
  <profile.json|.dcp|.icc> [output.tif]
The test profile command is used to test how well a profile can match a specific target, or if you skip the target it will just run some diagnostics on the profile, or if you replace the target with a tif file it will pass the profile through that. The output.tif is optional, if passed a test gradient or the processed input tif will be stored there, otherwise in the report directory (if provided).

It will print a text summary on the console, for deeper information you should use the -r parameter to dump text files and plots. If you skip the target you should generally provide -r to get any useful information.

Overview of flags:

As always it's preferable with a target file which contains spectra so XYZ reference values can be re-generated rather than having to be converted using a chromatic adaptation transform.

White balance

Per default DCamProf will calculate the optimal white balance to match the target as well as possible. This is analogous to setting white balance in your raw converter with the white balance picker on the white patch on a color checker. You can adjust this white balance behavior in the make-profile command, and if you have done that you should mirror the same settings in the test-profile command.

Anyway, if you instead want to test how the profile will match colors when the camera is set to a different white balance (such as a camera preset) you can provide a custom white balance via the -w setting.

It's given as a balance between red, green and blue, or as channel multipliers. To find out what multipliers a camera is using you can use exiftool on a raw file. White balance can be stored in different ways depending on raw format, it's out of the scope of this documentation to cover it in full. Anyway, in most cases it's some sort of multipliers, and often green is repeated twice, like this: 

  WB RGGB Levels Daylight : 15673 8192 8192 10727
And then you simply provide "-w m15673,8192,10727" to DCamProf, note the "m" which say that we provide white balance as multipliers rather than actual resulting balance between the channels which is 1/m.

When DCamProf prints a white balance it will show the balance normalized to 1.0, meaning that the above example is translated to 0.52,1,0.76.

Analyzing camera color separation performance

There is a special feature embedded in the test-profile command, which is that if you provide the camera's SSF you can get an analysis of the camera's color separation performance. This is a pure "hardware" test and has thus no relation to the profile so if you are only interested in this result you can provide any dummy profile.

To get a sane result you need a highly populated grid of patches to test with. I recommend to generate a locus grid, like this: 

  dcamprof make-target -c cam-ssf.json -p locus-grid -g 0.01 locus-grid.ti3
This will take quite some time, but once generated you can reuse this grid with any camera since when you provide the SSF and illuminants the RGB and XYZ values will be regenerated from spectra: 
  dcamprof test-profile -r dump1 -c cam-ssf.json -i D50 locus-grid.ti3 \
    any-profile.json
To get the plot you need to provide the -r parameter, and then the file is named ssf-csep.dat. You can plot it for example with this gnuplot script: 
  unset key
  set palette rgbformula 30,31,32
  set cbrange [0:300]
  plot 'gmt-locus.dat' using 1:2:4 w l lw 4 lc rgb var, \
    'ssf-csep.dat' pt 5 ps 2 lt palette, \
    'gmt-adobergb.dat' w l lc "red", \
    'gmt-pointer.dat' using 1:2:4 w l lw 2 lc rgb var
What you see is a heat-map in a u'v' chromaticity diagram, here limited to 300 max. Each dot shows how much the camera signal will change in 16 bits (65536 steps) for 1 delta E change in chromaticity (= change in hue and saturation with constant lightness). No current camera is really 16 bit, this is just used as a fixed reference to get a number in a comfortable-to-read range. For this type of test you should not worry about a camera's dynamic range and read noise, shot noise will be the limiting factor.

A black dot means that the signal change is zero and thus the camera cannot separate color at that chromaticity location and no profile can ever change that.

The test is run against the target provided and it expects a dense grid-like layout of patches, if your target is coarse there can be misleading results. The locus grid generated in this example makes reflectance spectra, so the colors tested are all related to the illuminant, the colors are as light as the illuminant allows for that chromaticity. This means more saturated colors are naturally bit darker and thus harder to separate. However it becomes harder for the eye too. Often cameras will show good separation capability in the purple range, and that is partly because the eye is relatively poor at it. As the values are related to Delta E they will be related to the eye's capability (as modeled by the observer's color matching functions).

The diagram always shows values relative to a D50 white point. You can test with a different illuminant using the -i parameter. You will see the result changing, but note that the coordinates are always remapped to D50 in the diagram.

Note that the generated locus grid will not go all the way to the edge of the line of purples. This is because the line of purples is actually black (as it's at the border of the eye's sensitivity) so moving in a bit we get saner colors. The spectral generator can still have some issues to reach all the way to the locus and line of purples so you may get some gaps.

This diagram in u'v' chromaticity coordinates shows the color separation capability of a Canon EOS 5D Mark II. The locus, Pointer's gamut and AdobeRGB gamut is shown as reference. Only points that have a patch in the provided target will be plotted, so here you see some gaps at the borders where there are no test patches.

The unit of the heat map is how many 16 bit units (65536 steps) the camera raw signal changes if the color chromaticity changes with 1 CIEDE2000 unit. The test reflectance spectra is a generated grid related to a D50 illuminant, and is made as bright as possible for each chromaticity coordinate.

The darker heat (lower signal difference) the worse color separation, if it's zero the camera can't differ at all. For complete information of limits you need to relate to photon shot noise as well, which is out of the scope of this document. What we can see is that the camera gets problems towards the locus, mainly on the cyan side and towards the red corner. We also see it's good at purples, which is partly due to that the eye is not as good and thus it takes more distance to reach one delta E.

We can also see that the diagram is a bit "worried" and that we have a notable minima inside AdobeRGB towards the red corner on the purple side. Some odd minimas here and there and the messy look is typical, as the SSFs differs greatly from the observer's CMFs. We see a smoother behavior in the green area, this is because there all three SSFs are involved in producing the signal.

Gradient testing

Example crop from the gradient test file showing a few poor transitions, such as the yellow vertical band in the center
If you enable -r <dump> a generated gradient TIFF file will be dumped, first without any processing as gradient-ref.tif and then processed through the profile including the LUT(s) as gradient.tif. This means that the content in gradient-ref.tif corresponds to white-balanced "raw" camera data, and the output is what that becomes when processed through the profile.

The purpose of this is to diagnose the smoothness of the profile's LUT as a complement to plotting. Note that as the gradient goes through all combinations (with some spacing) there will be some "impossible" raw values too, for example maximum blue but no red and green output. It's quite common that a profile clips or make artifacts in those areas, but this is no problem as they will never appear in real images.

Dumping this artificial gradient image is also very useful for verifying the smoothness during design of a subjective look using look operators.

The RGB primaries in the output is ProPhoto, and an ICC is embedded in the files. Beware that poor gradients and clipping is likely to occur due to the screen's color management, so turning it off temporarily when analyzing the more saturated parts of the image may be worthwhile. Use the unprocessed gradient-ref.tif as sanity checking, if that has some banded gradient it's probably due to the color management of the display.

Custom image testing

If you provide an image instead of a target file the profile will be passed through it: 
  dcamprof test-profile test.tif profile.dcp output.tif
The source image must be an 8 or 16 bit ProPhoto RGB image. The linearized data will then be seen as white-balanced camera raw data and put through the profile, and then saved to a new ProPhoto RGB image (output.tif).

One use for this is to test a look design on some specific subject. Then you don't want to test the camera colorimetric profile at all, just the look. To do this use the nil-profile.json in the data-examples directory to render an ICC or DCP: 

  dcamprof make-dcp -t acr -o look.json nil-profile.json test.dcp
And then you run that through your test image: 
  dcamprof test-profile test.tif test.dcp output.tif
The test image should be a normal white-balance image without a curve and with ProPhotoRGB ICC profile. You can prepare it from a real raw file, just make sure you apply a basic profile without a curve and export to a 16 bit ProPhotoRGB TIFF.

The normal way to test a look is to apply a normal finished profile on a real raw file in the raw converter. An advantage of applying it to a TIFF file using test-profile instead is that you can merge several images into one to test several aspects of the look at once. You can also make artificial images, to test gradients or special hue ranges that are difficult to find in real images.

Examples

Example 1: test how well profile.dcp matches target.ti3 under illuminant StdA, and write text files and plot data to the directory "dump" (it's assumed target.ti3 has spectra, if not you need to provide the -I parameter too): 
  dcamprof test-profile -r dump -i StdA target.ti3 profile.dcp
Example 2: test how well the profile will match colors with a camera white balance preset (found out via exiftool for example): 
  dcamprof test-profile -r dump -w m15673,8192,10727 -i D65 target.ti3 profile.json
Example 3: disable the profile's LUT and see how well the matrix matches the target (note that some DCPs designed with other tools are made such that the matrix is very far from correct color and the LUT is required to get close): 
  dcamprof test-profile -r dump -L -i D65 target.ti3 profile.dcp
Example 4: don't run any patch matching test, but only dump plots and reports: 
  dcamprof test-profile -r dump profile.dcp

make-dcp

  dcamprof make-dcp [flags] <profile.json> [profile2.json] <output.dcp>
Converts a profile in DCamProf native format to Adobe's DNG Camera Profile (DCP) which can be used directly in various raw converters. There's really not much to this command, generally you only run it with the -c flag to specify unique camera name.

Overview of flags:

HueSatMap LUT generator

The DNG HueSatMap is generated from the native 2.5D LUT in the DCamProf profile. This is done by sampling it at the hue and saturation divisions provided. The default is 90,30 (controlled with the -h parameter) which is a quite dense table and there's little reason to change that. If you'd want to change it it's probably to reduce the table size to get a smaller profile.

The DCamProf native LUT is spline-interpolated while a HueSatMap is linearly interpolated. This means that you may get smoother gradient transitions if you have a bit denser HueSatMap than needed for actual target matching. Therefore I think the 90,30 density is quite good to have even if the profile is based on very few patches.

If you dump plotting data with the -r parameter you will get data for the HueSatMap so you can visualize it. This is useful if you experiment with the table density.

Example plot for comparing native LUT with HSM LUT (useful to see if you should adjust HSM table size):

The plot shows a zoomed in section of the HSM LUT (blue dots) and the native LUT (beige grid). 

  splot \
    'nve-lut.dat' w l lc "beige", \
    'hsm-lut.dat' pt 1 lc "blue", \
    'gmt-prophoto.dat' w l lc "red", \
    'gmt-locus.dat' w l lw 4 lc rgb var
The HSM LUT operates in linear Prophoto RGB space, converted to HSV. This means that in an u'v' coordinate system it looks very dense close to the white point, and then becomes gradually less dense.
While DCamProf's native format is 2.5D, the color space is different from the DNG profile and DCamProf also adds a subtle 3D gamut compression step for the extreme range.

This means that it's not possible to get an exact match with a DNG 2.5D table. However the difference is very small, and also the gamut compression from the extreme range will translate well (although look slightly different) so therefore the HueSatMap is wisely kept at 2.5D. You can force it to make a 3D table though, using the -D flag.

Manual edits

When running the make-dcp command you can specify many but not all tags. If you want to adjust some of the remaining tags you need to dot this manually by using the dcp2json and json2dcp commands:
  1. dcamprof dcp2json input.dcp dcp-profile.json
  2. edit dcp-profile.json using a text editor
  3. dcamprof json2dcp dcp-profile.json output.dcp

Dual-illuminant DNG profiles

If you want to make a dual illuminant profile you make two separate native profiles and provide them both to make-dcp, like this: 
  dcamprof make-dcp profile1.json profile2.json dual.dcp
The lower temperature illuminant should be listed first, and you must have illuminants with known temperature, ie you cannot have "Other" which the profile will have if you have used a custom calibration illuminant. If so, specify illuminants using the -i and -I parameters, set EXIF names that as close as possible matches the temperature of your custom illuminants.

Note that the light source temperature is the only thing that matters to DNG Profiles, it makes no difference if it's a fluorescent (peaky spectrum) or tungsten (halogen, smooth spectrum), so if your calibration illuminant was a 3500K halogen lamp, the EXIF light source "WhiteFluorescent" (3525K) is the best choice.

DCamProf makes no sanity check on your illuminant listing so if you use "Other" or place the highest temperature light source first the resulting profile may not work in your raw converter.

The most common dual-illuminant combination is StdA and D65. It generally makes little sense to combine say D50 and D65 as they're too close. The general idea of dual illuminant profiles is to make a generic profile that works in varied light conditions, and then you want to combine two light sources whose white points are relatively widely spaced. Look at the color temperatures plotted in a chromaticity diagram for example to get and idea of how much they differ.

Avoiding white balance shift

If you have used a previous profile and custom white balance in your raw converter, applying your new profile will likely cause a white balance shift. See the section on DCP-specific white balance properties for a description why this can occur.

If you want to avoid this you need to replace the color matrix/matrices in your new profile with those found in the old, by using the -m parameter. As color matrices are only used for whitepoint temperature calculations and no actual color corrections this will not affect color rendition. The ability to predict white point color temperatures is in full replaced by the old profile though.

Tone reproduction

If you're making a profile for reproduction work you should not apply any curve, likewise if the targeted raw converter is designed for linear colorimetric (scene-referred) profiles. This is the default. However, many raw converters expect general-purpose profiles to apply a contrast-increasing "film curve", and in the case of DNG profiles this curve is embedded in the profile itself.

Per default DNG raw converters use a type of RGB curve that has some color distortion issues as discussed in the tone curves section. DCamProf can instead apply an own curve type (via 3D LookTable corrections) which is more neutral. This is enabled per default (controlled by -o parameter), but will only be used if a curve is applied (-t parameter). The properties of this is discussed in the section about DCamProf's neutral tone reproduction operator. You may also want to read the DNG profile implementation notes regarding this before using.

The supplied curve is either one of the built-ins, "linear", "none", "acr" (Adobe Camera Raw's default curve which is a good choice in most circumstances), or a custom curve in a JSON file, or a RawTherapee curve (.rtc) file. The JSON file format can be the same as for the transfer function, but only the "GreenTRC" tag will be used, or "TRC" or "GrayTRC" if those are available. You can also provide a "ProfileToneCurve" from a DNG profile. As usual all other tags are ignored so you can provide a full JSON of a DNG profile (as produced by the dcp2json command).

The RawTherapee .rtc format is supported, but only for "Spline" curves. It's a simple text file format with XY handles for a spline curve in sRGB gamma (both X and Y axes are gamma-scaled), see the data-examples directory for an example. If you wish you can design the curve using RawTherapee and export it from there.

Current tone reproduction operators are the "neutral" operator, and "standard" which in the DNG profile case means just embedding the curve and make no change, and then the raw converter will likely apply an RGB type of curve. You can also provide the name of a JSON file that contains custom weights for the neutral tone reproduction operator. See the ntro_conf.json file in the data-examples directory for further details. Normally you should not need to provide custom weights, but should for example the auto curve analysis lead to a too large or small chroma scaling factor you can set it manually using the configuration file.

Some raw converters are meant to be used with colorimetric profiles without any curve, but may still not have any good tone reproduction operator built-in, that is it's very hard to achieve realistic colors as soon as you apply contrast. In that case it may still be worthwhile to apply the tone reproduction in the profile, if the raw converter supports both ways (which is the common case).

Gamut compression

As a part of the neutral tone reproduction operator you can optionally configure gamut compression. There's a commented configuration example in the data directory which serves as the main documentation.

The purpose of the feature is to compress the gamut so super-saturated colors fit into a smaller gamut. Typically used to make sure the profile doesn't output colors more saturated than sRGB or AdobeRGB. This is the "gamut mapping" feature, but as it's always used to compress from a larger to a smaller gamut and is much less complex than mapping to a printer's irregular gamut (the usual application for gamut mapping) it's just called "gamut compression".

Although I'm not a personal fan of gamut compression, most bundled profiles have it and few raw converters have good automatic gamut compression so if you shoot lots of saturated colors (such as flowers) and want good tonality without any manual adjustments applying, it can be a good idea to apply some gamut compression in the profile.

Designing a subjective look

An example image rendered with a neutral profile (that is colorimetric accurate, with a contrast S-curve and DCamProf's neutral tone reproduction operator).
Same image but here with a designed subjective look. Without layering the image on top it will be difficult to see any difference, and this is how it should be. A successful designed look is typically very close to neutral. The most visible change in this look is that yellows and greens have been warmed up.
You can extend the configuration file for the tone reproduction operator with "look operators". The purpose of these is to design a subjective look which is applied on top of the neutral tone reproduction.

Be warned that this is not an easy task, especially since DCamProf lacks a graphical user interface. The process of designing a look means rendering lots of profiles with minor adjustments and comparing until you are satisfied with the result. It requires that you have a good eye for color and know what you want to achieve.

The intention of DCamProf's "look operators" is to make very subtle adjustments, small deviations from the neutral look. That is it's not intended to make "filters" as seen on Instagram and other popular social network services.

Some key concepts:

As already touched upon it will be very difficult to drastically change the look and get good results, so if you not at all like how DCamProf renders colors in it's neutral mode you are in trouble, as the adjusted profile will typically still be quite close to neutral. In that case I suggest using some other software as DCamProf is foremost about neutral and realistic color rendering.

Available look operators:

As there is no GUI you need to work with trial-and-error. Using a raw converter that quickly and effectively can load new profiles (like RawTherapee) is necessary to keep sanity. To see what colors that will be affected in an image (that is what area the "Blend" section covers) a good alternative is to use "ScaleChroma" with "Value" 0 and then all colors covered by the blend will be monochrome (set "BlendInvert" to true if you want the inverse).

For example if you want to target skin tones you adjust the "Blend" section so only the faces become monochrome, and then you can use this section for various adjustments in the real profile.

Curves are used in blending, and in the "Curves" and "Stretch" operators. There are three type of curves: "Linear", "Spline" and "RoundedStep". The "RoundedStep" is just a step function with S-curve transition, the other two are self-explanatory. Be warned that it's difficult to design a spline in the blind as it easily suffers from overshoots. You can test curves in GNUPlot or design curves in RawTherapee. The RGB curves operator can be mirrored exactly in RawTherapee by selecting ProPhoto working space, and in the operator select "sRGB" gamma, so then you can design the curves operator look there and export the curves, open in a text editor, reformat and put it in your JSON file.

When blending in various look operators there is a risk that you disturb the overall smoothness of the profile, perhaps you're making too strong adjustments with a too narrow blending zone. An effective way to diagnose this is to use the test-profile command and dump and image with processed gradients.

To see how the syntax works and get further documentation, look in the data-examples directory to find a documented example.

Hue shift discontinuity

The DCP LUT works in RGB-HSV space, which means that the hue is defined as an angle 0 to 360 degrees, and modifications to the hue is defined as an offset to the input hue angle.

When the input hue angle falls in-between two LUT table entries the offset is interpolated. For example if entry A says "add +40 degrees" and entry B says "add -30 degrees" and the input angle falls exactly in-between the average is calculated as "(+40 + -30)/2 = +5 degrees".

If we have a large hue shift say going from +170 to -170, the actual difference between those to neighbors is only 20 degrees and the average would be +/-180, but most DNG pipelines (probably all) don't support hue shift discontinuity and simply calculates this as "(+170 + -170) / 2 = 0". I'd say this is a bug, hue angle discontinuity is a well-known caveat when working with these type of coordinate systems, something that well-design code handles. The discontinuity is just in the math (it must wrap around somewhere), not in the actual hue transition.

Unfortunately Adobe's DNG reference code doesn't handle the wrap, and thus probably all software supporting DNG profiles don't either. Therefore make-dcp will per default abort if it detects a hue shift discontinuity.

Fortunately it's very unlikely that a discontinuity would occur in a normal colorimetric profile. It can quite easily happen when you design a subjective look with look operators though, and the solution is then generally to fade out the operator on the "HSV-Saturation" axis.

The built-in DNG pipeline in DCamProf uses the DNG reference code and will thus cause discontinuity artifacts just like the others. This means that you can see discontinuity artifacts when dumping a test gradient.

Observer remapping

DNG profiles has linear Prophoto as a working space, which is defined with the 1931_2 observer. That is raw converters using DNG profiles expect the D50 whitepoint map to 1931_2's D50. If you have used a different observer you will get slightly different XYZ values, and the D50 whitepoint will thus have a slightly different coordinate. There may be a 1-2 Delta E difference.

Many raw converters sanity-check the profiles to see that the whitepoint in the forward matrix matches 1931_2 D50, and if not they consider the DCP invalid and refuse to load it.

Therefore DCamProf will also do this check and if it detects a different whitepoint it assumes a different observer has been used in profile making and adjusts the matrices and LUT-making with a linear Bradford transform to adapt.

This transform is certainly not perfect when it comes to transform from one observer to another, but as the coordinate shift between observers is so small the error of the transform is probably considerably less than the overall accuracy errors in the profiling process so I think one should not need to worry. Some brief testing I've made confirms this.

As the default observer is 1931_2 this remapping will only take place if you have changed the observer (-o parameter) when making the profile. If you want to compare errors you can run a test-profile on both the native profile and the resulting DCP. The native profile will not need observer remapping. Note that the mapping from the native LUT to the HSM LUT will also generate slight differences from the native profile. Make sure you provide the desired observer in test-profile too, otherwise you will see large errors.

The color matrix is not remapped, as it's not used for the LUT and the difference between observers is way smaller than the error you can expect in a plain matrix conversion it's kept as is.

Examples

Basic conversion, this is what you will do most of the time (replace name with your specific camera name): 
  dcamprof make-dcp -n "Canon EOS 5D mark II" profile.json profile.dcp
Dual-illuminant profile with the illuminants specified (overrides tags in source profiles): 
  dcamprof make-dcp -n "Canon EOS 5D mark II" -i StdA -I D65 profile1.json \
    profile2.json profile.dcp

dcp2json, json2dcp

  dcamprof dcp2json <camera.dcp> [<dcp.json>]
  dcamprof json2dcp <dcp.json> <camera.dcp>
Convert DCP profiles to and from JSON format, useful for making manual edits.

make-icc

  dcamprof make-icc [flags] <profile.json> <output.icc>
Converts a profile in DCamProf native format to an ICC profile which can be used directly in various raw converters. Note that ICC profiles that works for one raw converter may not work in the next.

Overview of flags:

Compatibility

While ICC profiles in general are rigidly standardized, it's not well standardized how camera ICC profiles are applied in raw converters. They are often not applied directly to a linear raw image like DNG profiles always are, but rather there is some extra pre-processing step before, and possibly a post-processing step after. This means that ICC profiles are not as easy to move between different software as DNG profiles, you may need to design your ICC profile specifically for one raw converter.

I intend to support the most popular raw converters, I think DCamProf already support most of them but I haven't tested all, so if you find any compatibility issue let me know. I cannot promise I will implement support for every ICC-using raw converter though, if it's too messy I won't support it.

DCamProf supports raw converters which either provide demosaiced linear raw data as input to the ICC (like for example DxO Optics can do), or the same with a curve (like for example Capture One does). If a curve is applied that must be taken into account during the workflow, see the ICC example workflow for further information.

White balance

Most (probably all) ICC-using raw converters will apply the camera's white balance before the ICC profile is applied. You can see this if you export a file for profiling, if the white balance seems applied, then it is.

Still, a camera's "as shot" white balance rarely perfectly matches the calibration illuminant, that is a perfectly white patch will not be perfectly white but have a slight tint. DCamProf which knows the the XYZ coordinates for each patch and thus what white should be can correct for this if you'd like.

However, this would mean that when the profile is loaded the white balance will change so a perfect white (rarely exists in the target so it's interpolated) becomes RGB 1,1,1. This might be what you want, but likely not. Probably you want to keep the camera's original white balance and therefore this is the default when DCamProf makes ICC profiles. DCamProf will simply make sure that the profile maps camera RGB 1,1,1 to D50, that is use the native "forward matrix" mapping as is.

Note that since DCamProf normalizes the white balancing when making it's native profile it doesn't matter which white balance the test image had, meaning that you can convert the same native profile to both a DCP and an ICC profile, even when it was made from non-white-balanced data (like DCP requires).

Are there cases when you do want the ICC profile to correct the white balance? Yes, for example in a fixed light reproduction setup when you want to use a white balance preset on the camera (easy to remember and recall) but still get as correct white balance as possible in the final image, then the ICC profile should correct it. To do so supply the -W flag when making the ICC. For this to work the native profile must have been made from a white-balanced test image though (using the camera's preset of interest).

ICC profile type and ICC LUTs

DCamProf can make a pure matrix profile (with shaper curves if transfer function is provided via the -t parameter), and a LUT profile with either camera RGB to XYZ conversion or camera RGB to Lab.

By specifying the type you can even make a LUT profile even if the input does not have a LUT, which may be useful for testing in some cases.

An ICC LUT is always 3D, a simple table with RGB triplets in and corresponding XYZ or Lab triplets out. Ideally you would have a table entry for all possible RGB combinations which would be 65535^3 for 16 bit data, but that would fill your hard-disk with just the ICC profile so it's not a good idea. Instead very small ranges are used (33 is default) and the inbetweeners are interpolated.

DCamProf generates the ICC 3D LUT by sampling native LUT (and possibly tone reproduction operator), and applies an input curve to get better perceptual spacing of the LUT cube divisions. ICC LUT resolution can at times be a problem, if you get problems matching some patches you can try increasing the cube divisions from the default 33, be warned though that the size of the ICC file will grow very fast. A reasonable test value is 128 which will give you about a 12 megabyte ICC profile, and then reduce from there towards the default 33.

DCamProf can make an RGB to XYZ or an RGB to Lab LUT, the latter is the default. Currently the XYZ LUT uses the forward matrix directly which means that extreme value handling is not as good as the Lab LUT, so I recommend staying at the default Lab LUT.

Tone reproduction

The tone reproduction functionality is largely the same as described for DNG profiles, I recommend reading that first. The difference is that ICC profiles don't embed a separate curve and look table, but the curve modifies the single LUT directly.

Many ICC raw converters apply a curve on the side though (like Capture One), and in that case you should employ a linear curve during profiling and use that when using the finished profile, as the LUT applies the curve.

If you want to apply a subjective look you can do so, as documented in the subjective look design section. A difference from DNG profiles is that ICC profiles will allow you to change the color of neutrals.

You can also enable gamut compression.

Plotting ICC LUTs

You can add the -r <report_dir> flag to get report files which include ICC plot files. As ICC LUTs are 3D they are a bit cumbersome to visualize. You can plot all points in the 3D LUT "cube" by plotting icc-lut.dat, but it may be better to plot a slice at a time using the icc-lutXX.dat files. The main thing to look for is if the LUT seems dense enough to replicate the stretching that is in the native 2.5D LUT. You don't want to have it overkill dense either as that will make the ICC file larger than needed, and LUT ICCs are always a bit large due to that they are always 3D.
Plotting the full 3D LUT with error vectors and target: 
  splot \
    'icc-lut.dat' w d lc "beige", \
    'gmt-locus.dat' w l lw 4 lc rgb var, \
    'gmt-adobergb.dat' w l lc "red", \
    'gmt-pointer.dat' w l lw 2 lc rgb var, \
    'target-icc-lutve.dat' w vec lw 2 lc "black",\
    'targetd50-xyz.dat' pt 5 ps 2 lc rgb var
The example shows the default LUT with 33x33x33 points, it still becomes very dense in a 3D plot. The sides of the "gamut" are quite sharp and the shape is boxy, this is because the LUT reaches the full range of the LUT and clips (this is outside the real color range though, so don't worry).
The same plot as above, but now with just a slice: 
  splot \
    'icc-lut10.dat' w d lc "beige", \
    'gmt-locus.dat' w l lw 4 lc rgb var, \
    'gmt-adobergb.dat' w l lc "red", \
    'gmt-pointer.dat' w l lw 2 lc rgb var, \
    'target-icc-lutve.dat' w vec lw 2 lc "black",\
    'targetd50-xyz.dat' pt 5 ps 2 lc rgb var
There are 20 slices indexed 00 to 19, here we plot index 10 which means 0.5 to 0.55 in the native LUT lightness range (which is Lab lightness scaled to 0.0 - 1.0 range).
The same plot as above, that is a LUT slice with target and error vectors, now viewed straight from above and zoomed in on a detail around skin-like colors.

We see here that the profile is less accurate on darker colors (longer error vectors), while spot on on the brighter. The beige crosses show the LUT points in the slice. They are here in close-by pairs as the slice fits two levels (look from the side to see), so for the actual "2D" density think of the nearby pairs as one point.

icc2json, json2icc

  dcamprof icc2json <camera.icc> [<icc.json>]
  dcamprof json2icc <icc.json> <camera.icc>
Convert ICC profiles to and from JSON format.

ICC is a large standard and supports many types of devices in addition to cameras, such as printers, scanners and monitors. DCamProf's ICC parsing is only focused on ICC version 2 camera profiles, and will ignore any irrelevant tags and refuse to parse ICC profiles that are not camera profiles. That is it's intended to look at and edit camera profiles, nothing else. This means that icc2json does not work well as a general ICC dis-assembler. If you really need to see all tags in an ICC Profile you can for example use Argyll's iccdump tool.

tiff-tf

  dcamprof tiff-tf [flags] <target.tif> [<transfer-function.json>]
Extract transfer function (TIFFTAG_TRANSFERFUNCTION) from a TIFF file and write it to a JSON file. The transfer function is a linearization curve, that is if the data has been made non-linear of by a tone curve the transfer function will be the inverse of the tone curve.

The extracted transfer function can then be used in other relevant commands such as make-icc to linearize data. However make-icc/make-dcp etc can take the TIFF file directly so extracting it first is generally only for informational purpose.

You can however also calculate a tone curve using this command (as the difference between two transfer functions), which cannot be done with any of the other commands.

Some raw converters, like Capture One, applies the tone curve before the ICC profile. If you want to extract that tone curve to use in a DCamProf workflow you need to remove the transfer function for the linear component. You then do the following: export one TIFF with linear response "linear.tif", and one with the desired curve "curve.tif", and then you run the command: 
  dcamprof tiff-tf -f linear.tif curve.tif tone-curve.json
The output will then contain a tone curve in linear space calculated by applying the transfer function from "linear.tif" to the inverse of "curve.tif". This tone curve can then be provided to make-icc or make-dcp with the -t parameter.

txt2ti3

  dcamprof txt2ti3 <input.txt> <output.ti3>
Import spectral data from a text file, further described in the make-target section.

make-testchart

  dcamprof [flags] make-testchart <output.ti1>
Generate an Argyll .ti1 file (like Argyll's own targen) that can then be used with Argyll's printtarg command to make a test chart that can be printed. Overview of flags: Currently this command is very basic, it only supports RGB output (as most inkjet printers today are controlled as pseudo RGB devices), and you can only control the patch count, not which patches that are generated. The patches are generated such that it starts with one white patch, and then patches are spread out with as long perceptual distance as possible, with the constraint that only the lightest possible color of a certain hue and chroma are used. That is there will for example be no brown patch, as brown is actually dark orange. The rationale behind this is that as the LUT is 2.5D it's only necessary to profile the lightest colors, any darker colors will be grouped together in a chroma-group anyway.

The patch placement in terms of perceptual distance will not be perfect as the command is unaware of the printer's profile, but as the coverage is intended to be dense it doesn't matter that much. If it becomes popular to make own targets I may further develop this command to support printer profiles and more.

Today's inkjet printers typically have more colorants than older models, which means that the spectra can be a bit more varied. However the spectral variation will still suffer compared to commercial test targets made with special printing techniques. Your mileage may vary.

The test chart generator intention is to fill the gamut so it will need quite many patches to not miss out any corner. 50 patches is probably more than enough, but if you're printing an A4 sheet you could just fill it even if it will be a bit overkill. You can increase the white patch percentage to save ink.

With the -b and -g parameters you can add step wedges for linearization. This might be an advantage for targets that will be shot in situations where glare can be an issue.

testchart-ff

  dcamprof testchart-ff [flags] <input.ti1 | layout.json> \
    <input.ti3> [<input2.ti3>] <output.ti3>
...or 
  dcamprof testchart-ff <input.tif> <flatfield.tif> <output.tif>
Either flatfield correct .ti3 data or a linear .tif file. If you're correcting a .ti3 file it must be a target speckled with white patches and the layout need to be specified via a .ti1 file and the layout flags, or a layout JSON format (documented example in the data-examples directory). If you are correcting a .tif file the input files must be 16 bit linear gamma TIFF files.

It's also possible to model glare, this requires a neutral step wedge in the file, or even better neutral patches (black, white and middle gray) spread out over the whole surface. Overview of flags:

If you shoot indoor and have only one light it's difficult to get even light. In this example difference between the lightest and darkest white patch is as much as 1 stop. The flatfield correction algorithm use all the white patches as anchors and makes one thin plate spline surface per channel to correct. While I recommend to have more even light than shown here, this will work.
Animated image showing the result of a profile made with glare matching and one without, designed from the actual shot of the ColorChecker SG semi-glossy target shown in the picture.

At first glance the target may look well-lit and without issues, and indeed the white patches along the border are all equally bright, indicating perfectly even light (no need for flat-field correction here!). However look at the black patches, the ones along the right border are considerably lighter than the ones on the left. This is not uneven light, the neighboring white patches are the same brightness, the problem is instead glare. The target thus get lower contrast on the right side where there is more glare than on the left.

The animated images shows what happens if you make a profile ignoring the glare. Look at the dark red-purple patches in the top right corner. They become much darker with the uncorrected profile. The reason is that the make-profile gets much brighter camera raw samples than it should (affected by glare) and thus makes a profile that darkens them heavily to compensate.

As glare lowers contrast affecting dark colors the most, the resulting profile will thus compensate and the result will be opposite, too much contrast and too high saturation.

The left side has much less glare and thus the colors change less between the two profiles. However if hue is shared with a patch on the heavily affected side there is still a strong effect which can be noted in the pink patch in the CC24 section of the target. Note that the animated GIF image is limited to 256 colors so the more subtle differences in saturation etc cannot be seen.

The ColorChecker SG target is interesting as it's one of the few commercial targets that has white/black/gray calibration patches along the border of the target. DCamProf makes use of this and thus both flat-field corrects and makes a locally varying glare matching. This way the right side patches have been strongly corrected while limiting the effect on the left side.

While a good result is had with this shot, I strongly recommend against relying on glare matching. Instead make a proper setup which has less glare than shown in this example. It's much more important to minimize glare than to have even light, as flat-field correction can even out light precisely without adverse effects, while glare matching is by nature an imprecise model.

If you have been really careful when shooting your target to have uniform light the difference of applying flat-field correction will be negligible so it's certainly not mandatory. If you shoot a large target indoors and have only one light it's however most likely that you need to flatfield correct.

If your target is speckled with white patches you don't need to shoot an extra flatfield shot, correction can be made directly on the .ti3 data. When the target is photographed we know that if the lighting is perfect all white patches should give the same RGB values. Light is never 100% uniform though so the white patches will vary. Based on the positions of those white patches and the variations thin plate spline correction maps are created to scale all patch values to match uniform light.

The indexes of white, black and gray patches are found out from the provided .ti1 file. If you have used the make-testchart command you have already such a file, if not you're better off making a target layout JSON file, look in the data examples directory to find a documented example. For flat-field correction you only need to point out the white patches.

Most commercial targets are not speckled with white patches though unfortunately, and then you need to pre-process the TIFF file before you feed it to Argyll's scanin. First shoot the target, then with the exact same lighting place a equally large or larger white card in the exact same position as the target, and shoot it from the exact same camera position. Then make the exact same crop/rotation of both files and export to linear 16 bit TIFF. The image must be cropped enough so that only the white section of the white card is visible, if any surroundings or edges of the card is visible the result will not be good.

Feed those TIFF files to the testchart-ff command and you will get a new flat-field corrected output file which you then can feed to Argyll's scanin.

Another alternative is to print a chart with only white patches (ie only a grid) that exactly matches the target you have, and swap in that in a second shot (light and camera setup must be stable of course). You then run testchart-ff with this extra .ti3 file, so you have first the layout .ti1 file (showing only whites in this case), the white target .ti3, and then your real target .ti3 and finally the output .ti3 file. This is a bit cumbersome way to make flatfield, in this case it would probably be simpler to shoot a gray or white card instead and pre-process the TIFF file.

There are specific white card products to buy, but these are quite expensive. Instead you can for example use an unprinted high quality photo paper (without see through), I recommend a smooth matte OBA-free paper, make sure it lays perfectly flat just like the target. It does not matter if the card is slightly off-white, flat-field correction just corrects differences from the global average so the color of the card does not matter (if the card is colored, the global average changes too so it cancels out).

Glare matching

Many targets contain a neutral step wedge that can be used to linearize the raw samples to compensate for glare. However this process is hard to get stable and robust so DCamProf has chosen a different approach: add glare to the reference values (or spectra) so it matches the camera's raw samples.

That is instead of removing glare from the raw samples, we add glare to the reference data. As DCamProf knows the response of the observer (unlike the camera) it can apply certain robustness features to the glare model. The problem with glare modeling is that unlike flatfield correction it cannot be very precise. There are too many unknown factors, so DCamProf must rely on very coarse models. With more advanced models there's a large risk that you actually make things worse than better.

In DCamProf's glare matching robustness is main priority, that is don't make it worse than it was from the beginning. For example the glare matching will not alter hue, only lightness and chroma.

You can enable glare matching with the -L flag. It only works on .ti3 files, so if you have a TIFF you can flatfield it first, then scan it and then glare match the .ti3 file by running this command again. A flatfield pass is always run first (if possible), then linearization is applied.

I do recommend to do everything you can to minimize glare though so glare matching will not be required. While it's perfectly ok to rely on flatfield correction, as it can accurately even out light, it's not a good idea to rely on glare matching.

If you run with a report directory (-r parameter) you will find glare-match.tif there which shows how the reference patches were adjusted (that is lightened) to match the camera glare.

average-targets

  dcamprof average-targets <input1.ti1> [<input2.ti3> ...] <output.ti3>
If you have problems with too much noise in the darkest patches in your test target photos, you can make multiple shots, convert all to .ti3 files and then average them using this command. Averaging shots is an alternative to classic HDR merging and has the advantage that all shots are fully usable and thus scannable by Argyll's scanin command.

Of course you can do averaging/merging of images in other software too and make a new image which you then feed to Argyll's scanin, however you must then be absolutely sure that the software produce 100% linear results.

match-spectra

  dcamprof match-spectra [flags] <reference.ti3> <match.ti3> <output-match.ti3> \
    [<output-ref.ti3>]
Find the spectra in match.ti3 that best matches the spectra in reference.ti3, either as seen by an observer or by camera SSFs.

Overview of flags:

This command is typically not used in any profiling workflow, but is instead used for informational purposes. You can for example test how well the "skin-tone patches" of your commercial target matches real skin-tones from a spectral database.

As DCamProf's camera profiles are 2.5D it often makes sense to scale lightness to match both in comparison (-N) and in output (-S). If you specify one output it will contain spectra from the match.ti3 that matches, and if you specify two the second output will contain the patches from reference.ti3 for which an acceptable match was found. Per default there's no error limit and non-unique matches are allowed and then the second output will be a copy of reference.ti3

If a report directory is given (-r), spectra and XYZ coordinate plots for inputs and outputs are written there.

si-render

  dcamprof si-render [flags] <spectral image> <output.tif>
Render a normal RGB TIFF from a spectral image, specifying illuminant and observer or camera SSF.

Overview of flags:

The spectral image format must either be the SPB format or be a directory with normal monochrome TIFF files with spectral band (in nanometers) in the filenames. An example image with this format is the METACOW test image from Munsell Color Science Laboratory.

For the SPB format description and links to example files you can go to the www.multispectral.org web site.

Spectral images can become very large and DCamProf will read it all into memory, so make sure you have enough RAM.

While you can use this to just test differences between observers or CATs, the typical use case in camera profiling is when you have camera SSFs and want to test how camera profiles react under different light. You then render "virtual raws" with your desired illuminant and camera SSF using this command, and then you process that through the profile by using the test-profile command: 

  dcamprof si-render -c 5dmk2-ssf.json -W -g 1.8 -i D65 input.spb test.tif
  dcamprof test-profile -c 5dmk2-ssf.json -i D65 test.tif 5dmk2.dcp output.tif

Report directory files

When DCamProf is run with the -r <report_dir> flag enabled it will write data files for plotting and report text files. The files are suitable to plot with gnuplot, but you can use any other plotting software if you like as it's just text files with the numbers in columns.

Text files

The report text files contain patch matching reports: A patch matching row looks like this: 
A1 RGB 0.076 0.095 0.040 XYZref 0.130 0.113 0.057 XYZcam 0.129 0.112 0.054 \
  sRGB #7C5547 #7C5445 DE 0.60 DE LCh -0.23 +0.46 0.31 (dark brown)
First there's the patch name (A1 in this example) then camera raw RGB values (0.0 - 1.0 range), then CIE XYZ reference values (0.0 - 1.0 range), and then what XYZ values the profile transform came up with, and then sRGB values of reference and profile (note that these will only be accurate if the color is within the sRGB gamut), and then CIEDE2000 values for color difference between reference and converted value, related to the test illuminant.

The first delta E value is the total with 1,1,1 k weights, the following three is considering lightness (L) chroma (=saturation, C) and hue (h) separately. The lightness and chroma have a sign so you can see if the color is lighter (+) or darker (-) than it should be, and if it's more saturated (+) or less saturated (-) than it should be. In the above example we see that most part of the color difference sits in chroma (0.46 delta E), and it's a tiny bit too dark and too saturated.

Finally there's a text name of the color. This text name is highly approximate and may not really be that correct, but it roughly points out the type of color in lightness (light, dark), chroma (grayish, strong, vivid etc) and hue. Look at the corresponding image files if you want the reports with actual colored squares to represent the patches.

Image files

Crop from a patch matching report image. To make it easier to see the difference the patch square has been split diagonally. The reference value is in the top left half, and the profile result in the other.
A few TIFF image files can be dumped:

Plot files

It varies a bit between commands and parameters used which files that are produced, but many will be same, for example if the command processes a target it will produce files related to the target.

Most files have u'v' chromaticity coordinates, and if there's lightness there's CIE Luv / CIE Lab lightness divided by 100. The division by 100 is there to make it about the same scale as u'v'. This is the same 3D space as the DCamProf LUT operates in and is roughly "perceptually uniform", that is moving a certain distance in the diagram makes up a certain color difference. However as the space is linear and lightness is normalized it's not as uniform it could be, especially towards the line of purples which in reality goes towards black and thus hard to differ for the eye.

Here's a list of data files you can find in the report directory after a run:

Example gnuplot scripts

As patch colors are often involved I recommend using gnuplot with a gray background rather than the default white. If you use the X11 terminal you do this by starting gnuplot with the following command gnuplot -background gray. All examples here are adapted for a gray background.

In gnuplot you do 2D plots with the plot command, and 3D plots with splot. It's often useful to view a 3D plot in 2D though, and thanks to gnuplot's isometric perspective viewing a 3D plot straight from above makes it perfectly 2D.

You can rotate a 3D plot using the mouse, and you can zoom in by right-clicking and drawing a zoom-in-box. Type reset and replot to return to the original view. It's not a quick thing to master gnuplot, but with the help of the example scripts here you should be able to get around and do the tasks necessary for visualizing DCamProf data.

You can label the axes etc, but I usually make it simple and just remove all labels with unset key.

Plotting SSF and observer CMF: 
  plot \
    'cmf-x.dat' w l lc "pink", \
    'cmf-y.dat' w l lc "greenyellow", \
    'cmf-z.dat' w l lc "cyan", \
    'ssf-r.dat' w l lc "red", \
    'ssf-g.dat' w l lc "green", \
    'ssf-b.dat' w l lc "blue"
Basic plot for a test target, first the target spectra in 2D: 
  plot 'target-spectra.dat' w l lc rgb var
The example shows cc24
...and then the target patches in 3D: 
  set grid
  splot \
   'gmt-locus.dat' w l lw 4 lc rgb var, \
   'gmt-adobergb.dat' w l lc "red", \
   'gmt-pointer.dat' w l lw 2 lc rgb var,\
   'target-xyz.dat' pt 5 lc rgb var
Not shown in the picture, but you can also get text labels beside each patch by adding: 'target-xyz.dat' using 1:2:3:5 with labels offset 2
A suitable plot after a make-profile or test-profile run with a target with relative few patches (such as a cc24): 
  splot \
   'nve-lut.dat' w l lc "beige", \
   'gmt-locus.dat' w l lw 4 lc rgb var, \
   'gmt-adobergb.dat' w l lc "red", \
   'gmt-pointer.dat' w l lw 2 lc rgb var,\
   'target-nve-lutvm.dat' w vec lw 2 \
      lc "black", \
   'targetd50-xyz.dat' pt 5 ps 2 \
      lc rgb var
The image shows a zoomed in section, viewed directly from above, so we see a 2D chromaticity diagram with the LUT stretching in the chromaticity dimension. The black LUT vectors are only a little visible as the matrix alone makes a fair match.
A plot after a test-profile run with a dense target, such as a locus grid: 
  splot \
   'nve-lut.dat' w l lc "beige", \
   'gmt-locus.dat' w l lw 4 lc rgb var, \
   'gmt-adobergb.dat' w l lc "red", \
   'gmt-prophoto.dat' w l lc "blue", \
   'gmt-pointer.dat' w l lw 2 lc rgb var,\
   'target-nve-lutve.dat' w vec lc "black"
Here we only plot the error vectors, the actual color (reference XYZ) is at the start of the arrow and where it ends up after profiling is at the end of the arrow. For a perfect profile on a perfect camera the vector length should thus be zero over the whole field. As we can see in the example to the right errors typically grow large towards the locus, the matrix even moves points outside the human gamut.
A plot after a test-profile run with a DCP profile: 
  splot \
   'hsm-lutv.dat' w vec lc "beige", \
   'gmt-locus.dat' w l lw 4 lc rgb var, \
   'gmt-adobergb.dat' w l lc "red", \
   'gmt-prophoto.dat' w l lc "blue", \
   'gmt-pointer.dat' w l lw 2 lc rgb var,\
   'targetd50-xyz.dat' pt 5 ps 1.2 \
     lc rgb var
Here we plot the DCP HSM LUT as vectors, it can't be plotted like a grid like the native LUT. The vectors show each table position at vector start and their shift in chromaticity and lightness at vector end. Note that a DCP HSM LUT actually changes values through multiplication in linear Prophoto RGB HSV space, that's why the LUT looks like a star fitted in the ProPhoto triangle with high density at the white-point. The lightness axis has been transformed to match the same scale as the native LUT so the LUTs can be compared directly.

Be careful to watch gnuplot's auto-scaling of axes. The lightness axis in a LUT often gets greatly exaggerated or compressed due to it's not plot at the same scale as chromaticity. Use the set view equal command to turn on/off equal scaling (xyz = equal scaling on all axes, xy = default meaning chromaticity equal and lightness scaled to fit). 

  set view equal xyz
  set view equal xy
With equal scale on the L axis a LUT typically looks very flat as L adjustments are generally minor.

More example scripts are found throughout the documentation.

Call for spectral databases and camera SSFs

DCamProf contains some built-in spectral data that has been retrieved from public sources. I'd like to have more. A database with spectral reflectance of human skin is currently the most desired, useful for rendering portrait profiles.

There are reference standard sets such as the ISO TR 16066, but those are not free and cannot be freely redistributed so I can't include that in DCamProf.

If you know of any database you think is useful for inclusion please let me know.

The other aspect is camera SSFs. It's quite complicated and/or costly to measure camera SSFs so most users will not be able to do that and thus have to rely on public sources. If you can provide camera SSFs or have links to sources I have missed please let me know.

Links to camera SSFs

Links to spectral databases

Acknowledgments

I'd like to thank those that have made camera SSFs and spectral databases available, without those DCamProf would not have been possible in its current form. Currently DCamProf has spectral databases from University of Eastern Finland and BabelColor, see the section with links to spectral databases for references.

I also would like to thank all early adopters for testing the software.

Thanks to Mike Hutt for the Nelder-Mead simplex implementation which is used in DCamProf for solving various multi-variable complex optimization problems. I also want to thank Jarno Elonen for publishing a thin plate spline implementation which served as base for the DCamProf TPS used for getting a smooth LUT.

The copyright for the TPS source is required to be repeated in the documentation, so here it is: 

  Copyright (C) 2003, 2004 by Jarno Elonen

  Permission to use, copy, modify, distribute and sell this
  software and its documentation for any purpose is hereby
  granted without fee, provided that the above copyright
  notice appear in all copies and that both that copyright
  notice and this permission notice appear in supporting
  documentation. The authors make no representations about
  the suitability of this software for any purpose. It is
  provided "as is" without express or implied warranty.


Copyright © 2015 – 2016 — Anders Torger.