User:Clipka/Gamma

What is this Gamma Thing All About?

A reflective sphere on a checkered plane... or is it? Something appears to be wrong with the reflection. This POV-Ray 3.6 render showcases how a raytracing engine ignorant about gamma issues will produce inconsistent output.

In short, the whole "gamma issue" is about nonlinearity in image processing, regarding how intermediate brightness levels between 0% and 100% are interpreted.

To let you take a first brief glimpse at the can of worms we're about to open: A standard 8-bit image file has pixel values ranging from 0 to 255, equivalent to 0% and 100% brightness respectively. So what brightness would you expect for a value of 128? Well, 50%, right? Wrong. A meagre 22% is usually closer to the mark. Or 20%, or 25%, or whatever exactly your system happens to make of it. Unless we're talking about perceived brightness: A physical light intensity of 22% actually looks surprisingly bright, and an observer may perceive it as something around 45% in between black and white. Depending on ambient viewing conditions, just to make matters worse.

For a raytracing engine, linear physical light intensity is the only useful internal representation of intermediate brightness values, as it is in this domain that computations can be performed most efficiently; and any attempt to apply the same math in some other domain will produce inconsistencies in the output, which will usually be subtle enough to be unable to pinpoint them, yet obvious enough for an observer to somehow sense that something is wrong.

On the other hand, wherever the engine interfaces to the "outside world"—whether generating the output image file, submitting preview window content to the operating system for display, loading some input image for texturing, or just having the user enter color values picked from some other application—a raytracing software package has to account for the "gamma issue", or the output will still be just as wrong: Input colors will not be interpreted as intended, and the output image will not appear as computed.

Perceptual Brightness and Gamma Encoding

This image has suffered 6-bit linear encoding. Notice the particularly strong color banding in the shadow.

With 6-bit gamma encoding instead (using a gamma of 2.2), all brightness levels suffer a roughly equal share of color banding.

You may wonder why pixel values represent brightness levels (and thus colors) in such a strongly non-linear fashion—and why you probably never noticed before. However, if you have read the introduction carefully you may already guess the answer to the second question, which in fact also leads to the answer to the first one: The "pixel value way" of encoding brightness levels—known as gamma encoding—is much closer to how a human observer perceives brightness levels. For instance, while a human can easily distinguish a 1%-intensity grey from a 0%-intensity black, he is perfectly unable to tell a 99%-intensity grey from a 100%-intensity white. As a result, gamma encoding can get away with a lower bit depth than linear encoding.

In its simplest form, gamma encoding is based on a straightforward power-law function (aka gamma function, hence the name), i.e.:

$f(x)\!\,=x^{\gamma }$

where $\gamma$ is either:

• the encoding gamma when encoding from linear brightness values, or
• the decoding gamma—being the inverse of the encoding gamma—when decoding to linear brightness values.

A typical encoding gamma would be somewhere around 0.45, with the decoding gamma consequently being somewhere around 2.2.

Another popular function used in this role is the sRGB transfer function. It can roughly be approximated by a classic power-law function with a decoding gamma of 2.2, but has slightly superior properties at very low brightness levels. As the name suggests, it is an integral part of the sRGB color space, which is the officially recommended standard color space for the World Wide Web.

Display Gamma and Pre-Correction

To get the history straight, storage efficiency is probably not the main reason why gamma encoding originally evolved in the first place. Instead, it seems to have been around ever since the first use of multiple brightness levels in computer displays, and is more likely due to the inherent power-law response curve of cathode ray tubes, which just happens to fit quite well with the peculiarities of human vision. To minimize hardware complexity, the graphics adaptor would use linear DACs to proportionally convert color data to signal voltages for the display, which in turn would more or less directly drive its CRT from these signals; without any electronics to compensate for the nonlinearity of the CRT, the color data would have to be pre-corrected for the display subsystem's inherent display gamma.

A typical display gamma is somewhere around 2.2 on PC systems (remember that value?), and 1.8 on Mac systems prior to Mac OS X 10.6 (the proper pre-correction would consist of applying a power-law function with a gamma of 0.45 or 0.56 respectively, but in this context it is more common to specify the display gamma to correct for). There is a wide bandwidth though, especially in the heterogenous PC world, as an arbitrary system's exact gamma depends on the particular combination of graphics card, display, and individual display settings like brightness and contrast. In professional image-processing environments, it is therefore customary to calibrate a computer's display subsystem to match a certain well-defined overall gamma.

While the non-linear response curve is a CRT-typical phenomenon that is not inherent in LCD panels, LCD-based displays are typically equipped with circuitry to deliberately achieve a CRT-like gamma—possibly not only for the sake of compatibility, but also for efficient use of digital interfaces.

File Gamma

As mentioned before, the nonlinearity in CRTs happens to roughly coincide with the nonlinearity in human vision, so that the same mathematical transformation traditionally required to pre-correct color data for the graphics hardware also happens to provide a more efficient encoding. It should therefore come as no surprise that image files have a long tradition of not storing linear light intensity data, but rather gamma pre-corrected—and thereby also gamma-encoded—color values.

The downside to this is that gamma pre-correction is inherently system-specific in nature, and therefore an image pre-corrected to fit one computer's display gamma may not fit another one's; and while in a professional environment there would usually be a standard to which all computers would have been calibrated, this standard might still differ between individual organizations. As a consequence, most older file formats do use gamma encoding in practice, but allow for broad variations in the encoding gamma.

Fortunately, the situation has improved a lot recently, as image viewing software starts taking over the responsibility for properly pre-correcting the image data to be fed into the graphics card, reducing the importance of gamma in image files to that of mere gamma encoding. At the same time, the normative power of the World Wide Web has been pushing the sRGB color space—and with it the sRGB transfer function—as the de-facto encoding standard for virtually all major file formats that traditionally carried pre-corrected content.

Various file formats have also been extended to optionally carry meta-information about the encoding gamma or color space used, although in practice such features are only used for a small subset of the major file formats. One notable example is the PNG file format.

A recent development in another direction are high dynamic range image file formats, which employ floating-point or floating-point-alike number formats to represent light intensity values in a vast range far exceeding that of traditional file formats. As a side effect they can get away without gamma encoding, and are therefore explicitly defined to store linear light intensity levels, avoiding both unnecessary processing steps and the potential confusion associated with gamma.

Gamma in POV-Ray

With raytracing being essentially an attempt to simulate the physical interaction of light rays with object surfaces and media, it should come as no surprise that the math behind it is easiest (and fastest) when brightness levels are expressed as physical light intensity, rather than perceptual brightness (which varies with viewing conditions anyway) or the more or less arbitrary concept of pixel values. To get realistic results at reasonable speed, a raytracing system must do virtually all its internal computations with physical light intensity.

Still, in theory this would not stop us from having the user specify all colors by pixel values, and automatically convert to physical light intensity just as the macro above does. So why does POV-Ray not do it?

The answer lies in the power of the scene description language: You can do too much with colors in POV-Ray's SDL that it is practically imposible to come up with a watertight automatism of identifying which non-color value specified in a scene file will wind up being interpreted as a color nonetheless (and therefore would have to be subject to conversion), which color value will end up being interpreted as a non-color (and therefore would have to be exempt from conversion), and whether some mathematical operation on colors should be performed in the pixel value domain or in the light intensity domain instead.

POV-Ray's philosophy therefore is to always presume individual colors to be expressed by linear light intensity values, and leave it up to the user to explicitly specify when and where conversions are to be applied.

Gamma Settings

POV-Ray's gamma handling is governed by the following parameters:

• The File_Gamma INI file setting, which controls the encoding gamma to be used for image file output. (Note that for some file formats, this traditionally double-acts as gamma pre-correction; also note that while this file format controls the encoding gamma, what you specify is actually the corresponding decoding gamma.)
• The Display_Gamma INI file setting, which specifies the display gamma for which to pre-correct render preview output.
• The gamma scene file keyword applied to input images, which overrides the default decoding gamma rules.
• The gamma scene file keyword applied to color values, which forces POV-Ray to apply gamma decoding to individual colors.

In addition to these, there is also an Antialias_Gamma INI file setting. however, this is a somewhat different animal, as it does not affect the colors in the scene, nor the output file encoding, but merely serves to optimize the antialiasing mechanism (see the section on antialiasing for details).

Practical Implications

Even as a POV-Ray beginner, you'll possibly care about the general phenomenon: The 50% vs. 22% vs. 45% thing—pixel values and perceptual brightness vs. physical light intensity. These differences have two major implications for you:

• Colors you pick in Photoshop (or Gimp or whatever your favorite image processing software may be) will not seem to work in POV-Ray as expected; greyish colors will simply appear too bright, while more colorful ones will appear washed out, or even off hue (this is because the nonlinearity affects each color component individually). The same goes for HTML color values you try to convert.
• When trying to achieve a certain brightness level, you may have difficulties getting the level right at first attempt.

As an advanced POV-Ray user, you may also worry about different systems exhibiting more or less subtle differences in the nonlinearity:

• The same image file may display differently on a friend's computer. Or the new LCD you're about to buy. Or the machine you're running your overnight renders on. Or the computers of the jurors of that internet ray tracing competition you're taking part in. You probably want to make sure they can see the barely-lit creature lurking in your nighttime scene, or that your daytime scene doesn't look too washed-out on their displays. Presuming they have done their own gamma homework of course.
• You want to be sure that the scene you share across the internet renders on the recipient's computer just as it does on your own.
• You may want to use input image files that were created with an unconventional encoding gamma, or (mis)used as a data container with linear encoding.

Getting Individual Colors Right

When picking pixel values from Photoshop (or whatever you have), you can use the following formula to approximate the color parameters to use in your POV-Ray scene file:

$colorvalue=\left({\frac {pixelvalue}{255}}\right)^{2.2}$

Note: Transparency information is not normally gamma-encoded, so you'll want to apply this formula to R, G and B components only.

If you frequently pick colors, or feel more at home with defining colors by pixel values rather than physical brightness levels, you can use the gamma keyword to have POV-Ray do the computations automatically for you, e.g.:

#local MyColor = color rgb <255,128,0>/255 gamma 2.2;

This syntax also allows for using the sRGB transfer function; as the sRGB color space has become quite widespread through the influence of the World Wide Web, where it is the official standard, it is probably an even better bet than a power-law function with a gamma of 2.2:

#local MyColor = color rgb <255,128,0>/255 gamma srgb;

Getting The Render Preview Right

Under Construction

In a nutshell, the answer is: Calibrate your system, and encourage others to do the same.

See Documentation:Reference_Section_1.1#Setting_your_Display_Gamma.

Getting The Output Image File Right

In order to get gamma right in output image files, you may or may not need to set the File_Gamma INI file variable, depending on the file format you use (note that the numeric value specifies the decoding gamma, which is the inverse of the encoding gamma; e.g. the default value of 2.2 corresponds to an encoding gamma of about 0.45):

• When using OpenEXR (+FE) or Radiance HDR (+FH) high dynamic range output, you do not need to worry about File_Gamma, as it has no effect with these formats; both are explicitly defined to store linear light intensity values, using floating-point or similar numeric representations instead of gamma encoding, and POV-Ray will always comply with this convention.

• The PNG file format (+FN) provides a reliable means to inform reading applications (e.g. file viewers) about the encoding gamma used, so most software will display the output image virtually identical regardless of the encoding gamma you choose; however, it is recommended to leave File_Gamma at the default value of 2.2 for PNG files, as this will make the image display reasonably well even with applications that ignore the encoding gamma information, and is also of benefit to prevent excessive color banding in dark areas.

• All other output formats available in POV-Ray do not provide a reliable means to inform about the encoding gamma used; for these, File_Gamma needs to be set according to the encoding gamma expected by the reading application, or the image will not display correctly. If you don't know which gamma to use, or intend to distribute the image to a wide audience, it is recommended to leave File_Gamma at the default value of 2.2.

Note: POV-Ray now also allows to use the sRGB transfer function for display and file output, by setting File_Gamma and/or Display_Gamma to sRGB. While this is currently not the default, it is probably an even better choice.

Occasionally you may want to render a scene to generate a data set instead of an actual picture, and have the resulting data be encoded linearly (e.g. when creating height field data). You can achieve this by setting File_Gamma to 1.0. (Note that this is not strictly necessary when outputting to OpenEXR or Radiance HDR, but it doesn't hurt either; also note that when outputting to PNG you will not see a difference in most image viewers.) For such use cases, it is recommended to set Antialias_Gamma to 1.0 as well.

Getting Input Image Files Right

Typically input image gamma is handled automatically by POV-Ray, according to file format specifications, encoding gamma information embedded in the image, official recommendations or common practice. Some cases may need special attention though:

• Some OpenEXR or Radiance HDR files may erroneously have been gamma-adjusted.
• Some PNG files may carry wrong encoding gamma information.
• Some files using other formats may deviate from official recommendations or common practice.
• For some file formats (e.g. PPM) there is no clear recommendation or common practice.
• Some files may be mere data containers using linear encoding (e.g. height field data).

In these cases, you can override POV-Ray's automatic handling by specifying gamma GAMMA directly after the filename, e.g.:

#declare MyPigment = pigment { image_map { jpg "foobar.jpg" gamma 1.8 } }

This would cause POV-Ray to use a decoding gamma of 1.8 (corresponding to an encoding gamma of roughly 0.56) when reading this particular file.

Besides numerical values, you can also specify gamma srgb, indicating that POV-Ray should assume the file to be encoded using the sRGB transfer function, which is actually POV-Ray's default assumption for most file formats.

Note that when used directly in a height field, bump map or image pattern, POV-Ray will automatically assume linear encoding (by default; again you can override this on a per-file basis); however, images used indirectly in a height field (e.g. via a pigment function and function image) will still be assumed to be gamma-encoded by default.

Legacy Scenes

Backward Compatibility

Under Construction

In a nutshell, the answer is: Legacy scenes will generally continue to work, as long as they have a proper #version statement, and don't use assumed_gamma for artistic effect; a notable exception are PNG in- and output files.

Artistic Use Of assumed_gamma

The backward compatibility mechanisms designed into POV-Ray 3.7 presume that POV-Ray 3.6 legacy scenes use assumed_gamma either to enforce proper math (assumed_gamma 1.0), or easy color use (assumed_gamma matching the system's gamma, typically 2.2). For legacy scenes using assumed_gamma for artistic purposes instead, the following procedure can be used to reproduce the old effect:

• Note the assumed_gamma scene file setting used to render the scene with POV-Ray 3.6 (old_assumed_gamma henceforth).
• Note the Display_Gamma INI file setting used to render the scene with POV-Ray 3.6 (Old_Display_Gamma henceforth); if not set, presume a value of 2.2.
• Compute the quotient of the two values, Old_Display_Gamma/old_assumed_gamma (New_Display_Gamma henceforth).
• Change the assumed_gamma scene file setting to 1.0.
• Render the scene with the Display_Gamma and File_Gamma INI file settings set to New_Display_Gamma.

Note: This is more of a hack than anything else, and will not work when using PNG or any of the HDR file formats (Radiance HDR, OpenEXR) for output.

PNG files

PNG output file handling has been changed significantly in POV-Ray 3.7, and is not fully backward compatible. If the PNG output does not suit your needs, it is recommended to use a different output file format instead, and use an external image processing tool to convert to PNG if required.

Planned changes

Note: The following information is the intended behaviour of a change currently under development; it is not true for current beta of POV-Ray 3.7 (beta 39 at the time of writing).

For backward compatibility with legacy scenes, POV-Ray 3.7 will mimick the gamma handling of POV-Ray 3.6 if the scene has a #version statement specifying a version of 3.6x or earlier, and/or specifies an assumed_gamma, with the following exceptions:

• If the scene overrides the default input file gamma, POV-Ray will fall back to the new gamma handling model.
• If the scene is rendered using the File_Gamma INI file setting, or using an INI file setting or command-line option to specify a version of 3.7 or later, input file gamma handling will still mimick 3.6 behaviour, but output file gamma handling will use the new model.
• Input file gamma handling of palette-based PNG files (an uncommon flavour of PNG) contained a bug in POV-Ray 3.6, which is not replicated. If your legacy scene relies on such an input file, you may need to override the encoding gamma for this particular file using the gamma keyword.

Migrating a Scene

Under Construction

Scenes with assumed_gamma 1.0

These are typically the easiest to migrate, as the scenes themselves are already designed for proper gamma handling to some degree. The following steps should make the scene render exactly as before:

• Change #version to 3.7.
• Remove the assumed_gamma statement.
• Add gamma 1.0 to any input image files.

Note however that this will not necessarily make your images render right, most notably as far as input image files are concerned, so especially if you always have been struggling with those images, it may be worth giving the scene's illumination a more thorough overhaul:

• Only add gamma statements to input image files used for non-color purposes, e.g. transparency maps, bump maps, height fields, or mere data containers.

Your image should render properly now, in the sense that color computations will be realistic; however, you may need to tweak your light sources and materials again to get the scene to look the way you want it to.

Scenes with assumed_gamma 2.2

As these scenes were designed for wrong maths, they will typically be next to impossible to migrate in a way to render exactly the same, so they are likely to need a major overhaul. The following steps should get you started:

• Change #version to 3.7.
• Remove the assumed_gamma statement.
• Add gamma 2.2 to any color literal, except where ultimately used to compute non-color values.
• Add gamma 2.2 to any place where a non-color value is used as a color.
• Add gamma 1.0 to any input image file used for non-color purposes, e.g. transparency maps, bump maps, height fields, or mere data containers.

(See also this HowTo.)

Your image will now render properly, in the sense that color computations will be realistic; however, this is unlikely to be exactly what you want, so you'll most likely need to give your light sources and materials some tweaking again.

Scenes without assumed_gamma

These scenes, too, will typically be next to impossible to migrate in a way to render exactly the same, and are likely to need a major overhaul as well. In this case, the following steps are recommended to get you started:

• Change #version to 3.7.
• Add gamma 1.0 to any input image file used for non-color purposes, e.g. transparency maps, bump maps, height fields, or mere data containers.

Your image will now render properly, in the sense that color computations will be realistic; however, this is unlikely to be exactly what you want, so you'll most likely need to give your light sources and materials some tweaking again.

PNG files

Under Construction

PNG input file handling has been changed significantly in POV-Ray 3.7, and is not fully backward compatible.

In a nutshell, for PNG input files you may - or may not - need to add a gamma statement, and toy around with its value, to get the same look as with POV-Ray 3.6. Good bets would be 1.0, srgb or occasionally 1/2.2. For output files, it is recommended to use a different file format if PNG output doesn't look as desired and a complete migration of the scene is not an option.