User:Wfpokorny/DensityFile

Note! While the following density_file (.df3) pattern documentation is often valid with respect to the current 3.7.0 forms of df3 value interpolation, it is written as if the additional interpolation modes proposed in https://github.com/wfpokorny/povray/tree/feature/newDensityPatternInterpolations have been adopted.

Note! After some discussion it's been decided the 'assist' interpolation modes 6-12 will be dropped for methods already supported. Wiki edits are mostly complete to match and the source code has been updated.

Density File Pattern

The density_file pattern is a 3-D bitmap pattern that occupies a unit cube from location <0,0,0> to <1,1,1>. The data file is a raw binary file format created for POV-Ray called the df3 format. This pattern was originally created for use with halo or media, but it may be used anywhere a pattern may be used. The core syntax whether used in a normal, pattern or pigment block is:

density_file df3 [file name] [interpolate Type]

where the file name would be placed in double quotes and the interpolation type would be an integer value 0 through 5. For example:

density_file df3 "star.df3" 1

The general forms for each of the normal, pattern and pigment blocks are:

normal {
  density_file df3 [filename] [,Bump_Size]
  [interpolate Type]
  [NORMAL_MODIFIERS...]
}

where Bump_Size is an optional float value,

pattern {
  density_file df3 [filename] [interpolate Type]
  [PATTERN_MODIFIERS...]
}

and

pigment {
  density_file df3 [filename] [interpolate Type]
  [PIGMENT_MODIFIERS...]
}

The df3 file format

The df3 format consists of a 6 byte header of three 16-bit, unsigned integers with high order bytes first (big-endian order). These three values give the x,y,z size of the data in voxels.

The header is followed by x*y*z unsigned, big-endian encoded, integers with resolutions of 8, 16 or 32 bits. The resolution of the data is determined by the size of the df3 file. That is, if the file - minus the header - is twice as long as an 8 bit file, it is taken to contain 16 bit data. If it is four times as long, it is taken to contain 32 bit data.

How the density_file pattern works

The density pattern occupies the unit cube <0,0,0> to <1,1,1> regardless of the x, y, z dimensions in voxels. It remains at 0.0 for all areas beyond the unit cube. Data values in the ranges of 0 to 255 (8 bit/1 byte), 0 to 65535 (16 bit/2 bytes) and 0 to 4294967295 (32 bit/4 bytes), are scaled by the various interpolation methods to a float value in the range 0.0 to 1.0. The larger 2 byte and 4 byte depths providing better resolution.

Interpolation options.

There are currently 6 data interpolation types numbered 0 through 5, with the types larger than 2 being available only in releases 3.7.1 and later. If nothing is specified via the interpolate keyword, the default is 0 which does no interpolation - meaning for a given location in the unit cube, the 0.0 to 1.0 value of the containing voxel is returned. The other interpolation methods available are:

1. Tri-linear. Similar to two dimensional image method, but for three dimensions.
2. Tri-cubic. Similar to two dimensional image method.
3. Weak exponential blobbing of defined voxels in 4x4x4 local sub regions.
4. Moderate exponential blobbing of defined voxels in 6x6x6 local sub regions.
5. Strong exponential blobbing of defined voxels in 8x8x8 local sub regions.

If an interpolate value not supported is specified, the interpolation defaults to tri-cubic interpolation, 2, in POV-Ray versions 3.6 and later.

Exponential blobbing interpolations.

Exponential blobbing techniques in POV-Ray date to the introduction of isosurface objects. The method here borrows from one common technique with the form:

1/exp((function's value)*(blobbing strength))

As implemented, for each evaluated point in the unit cube, the code sums the result of the equation:

(1/exp(pow(point_to_voxel_distance,2)*(internalTuningBias/voxel_value))

for all adjacent df3 values >0 within the NxNxN sub-cube around the evaluated point. Resulting values >1 are clamped to 1.0. The larger the sub cube, the stronger the blobbing which can be accommodated.

The algorithm has been tuned so 1.0 values for each of the exponential blobbing interpolations just fit within the corresponding sub cube. The blobbing can be adjusted without the need to regenerate a df3 by using the standard pattern wave modifier poly_wave <float>. Poly_wave values >=1.0 should always work and reduce the blobbing. Values < 1.0 increase the blobbing and can cause sub cube overruns. Overruns will appear as discontinuities in the final pattern.

Only interpolation 0 is voxel centered in 3.7.0 and earlier releases.

For version 3.7.0, the interpolations 0 to 2 are shown below in order, left to right, as a pigment on a plane for an 11x11x11 df3 file where a single center voxel has a value of 0.5 while all others are 0.0. The red dots mark the vertices around our center voxel.

Density_file interpolations for single center voxel as a green pigment on a plane with release 3.7.0.

The tri-linear and tri-cubic interpolations are vertex centered prior to 3.7.1 resulting in a half voxel negative shift in x,y,z . This shift is perhaps of no consequence if rendering media. It does however matter if, for example, building a df3 based isosurface or when trying to control textures for media for any formed objects using interpolation 0.

Offset tri-linear and tri-cubic results can be centered with a positive translation of one half a voxel size in x,y and z. For the 11x11x11 df3 example use:

translate <-0.5+(1/11/2),-0.5+(1/11/2),-0.5+(1/11/2)>

to get the density_file pattern exactly centered upon the origin. In a function for an isosurface this adjustment to center on the origin would be something like:

FnctAdjustToCenter(x+0.5-(1/11/2),y+0.5-(1/11/2),z+0.5-(1/11/2))

Release 3.7.1 makes all interpolations voxel, and so too unit cube, centered.

In addition to adding density_file pattern interpolation modes, 3.7.1 centers all the interpolated results. The 3.7.1 interpolations 0 to 5 are shown in order, left to right, in the following image.

Density_file interpolations for single center voxel as a green pigment on a plane.

Note. Backward, shifted compatibility for the older 1 & 2 methods is available by using interpolations 11 & 12, respectively.

Tri-linear interpolation has creases, tri-cubic interpolation wobbles/rings.

The following figure shows isosurfaces for interpolations 1 through 5, left to right, setting the threshold very near zero to better see the non-zero values returned. The df3 used is a 15x15x15 arrangement where an x,z box has been centered in y and through which there is a y column of >0.0 values. The values for the non-zero voxels in the top row are 0.1 and the bottom row 0.9.

The former better showing the crease effect of tri-linear interpolation, the latter, the ring of tri-cubic interpolation. The tri-cubic mode always wobbles, but the most severe artifacts can be avoided by using values not near 0.0 and 1.0 where pattern clipping occurs.

Density_file interpolations 1-5 for defined voxels arranged as box on spindle.

Lastly, with respect the the ringing of the tri-cubic interpolation, there are at times, additional harmonics as shown at the bottom of the following image which has been gamma adjusted to make this clear. The df3 sets only 3 voxels, here seen as the brighter green rectangles.

Density_file second ring example with tri-cubic interpolation.

DF3 value and density_file interpolation effects on an isosurface max gradient.

Interpolation methods and df3 values affect the max gradient in isosurface use. For tri-linear and tri-cubic Interpolations 1 and 2, lower values result in lower gradients due the typically longer ramp from the >0 edge to the voxel value used as a threshold. With the exponential blobbing of modes 3, 4 and 5, the lower gradients happen instead at larger voxel values - lower voxel values result in small spheres with much sharper local gradients. Gradients for interpolations 0-5 tend to increase as the df3 file's voxel count increase.

Interpolations 3, 4 and 5 max values always go to 1.0 at any >0.0 voxel center, where interpolations 1 and 2 peak at the voxel's specified >0.0 value.

Reported maximum gradients for df3 based isosurfaces will often be much higher than what can be used. These higher gradients occur where a ray catches the far corner or edge of the df3-function in a direction more perpendicular to the best overall local gradient. Note. This far corner, near threshold, gradient effect happens with other patterns too and can often be ignored in practice.

Use a buffer of five 0.0 voxels on all sides to avoid density_file edge effects.

While not a hard and fast rule, all the interpolations other than 0, and 6-12, will wrap or repeat values on the sides. The general recommendation of five comes from interpolation five's need for an 8 grid range (4 per side) about any evaluated point - plus a half voxel grid for the internal vertex to voxel center correction of interpolations 1-5. Often smaller buffer amounts can be used.

Below are two images. In the first, the lower-left-forward corner of the df3 has been filled with 0.2 values with all others 0.0. In the second image the top-right-back corner of df3 file has been set to values of 0.2 with all others set to 0.0. Interpolations 0 through 5 are shown left to right in each of these two images as a pigment on an x,y plane at the df3 unit cube's z center.

(1) Density_file interpolations 0-5 with defined left, bottom, front voxels.

(2) Density_file interpolations 0-5 with defined right, top, back voxels.

All truncate sharply to 0.0 where values have been run to the unit cube sides. This abrupt change is especially inconvenient for isosurface use, but the hard edge can often be seen in other pattern uses as some artifact.

The tri-linear and tri-cubic interpolations, 1 and 2, wrap side to opposite side in the lower-left-forward side-abutted case shown in the first row. These same interpolations truncate in the rop-right-back side-abutted case shown in the second row.

The exponential blobbing of 3 through 5 all repeat - or add values - at the last voxel position at the sides. This can sometimes be compensated for by lowering the on side and in-corner values.

df3 file size.

Df3 files can get very large, very quickly given the cubed dimensions. A df3 1000x1000x1000 at 1 byte resolution will take a gigabyte of storage. It practice, aim to reduce the z dimension as much as possible letting the density_file pattern scale up to the unit cube. Later scale the unit cube dimension to the desired final size.

3D isosurface extrusion of image via shallow in z, density_file pattern.

An example compressing the df3 in one dimension comes naturally when the density_file pattern is used for extrusion of a 2D image into 3D shape. The outline for such an extrusion is shown in the following code snippets with a resulting image. A 500x500x13 deep df3 of the red in a 2D image, at left below, is created with the following code:

#include "arrays.inc"           // For ARRAYS_WriteDF3()
#include "arraycoupleddf3s.inc" // Array coupled df3 macros & functions.

#declare ImageFileNamesAry=array[3] {
   "BlackRedLettersIn.png",
   "BlackRedLettersIn.png",
   "BlackRedLettersIn.png"
}
//                                  Red     
ConvertImagesToDF3(500,500,0.0,1.0,<1,0,0>,0.2,1,ImageFileNamesAry,"BlackRedLetters.df3",8,5)

#error "Stopping after parse stage for df3 creation from image." 

The include file arrays.inc is already part of POV-Ray and the arraycoupleddf3s.inc file is given in the examples section. Once the df3 representation of the 2D image exists, the following code is used to create the function for the isosurface text using blobbing interpolation 5.

#declare Fnct00 = function {
    pattern { density_file df3 "BlackRedLetters.df3" interpolate 5
        translate -0.5 // Center about origin for isosurface
        scale <1,1,1/((500+10)/(3+10))>
        warp { turbulence <0.015,0.015,0> octaves 6 omega 0.43 lambda 6 }
   }
}
#declare Fnct01 = function (x,y,z) {
    0.025-Fnct00(x,y,z)
}

Density_file as isosurface extrusion of 2D image while selecting color for extrusion.

In the function declaration, Fnct00, the pattern is centered about the origin with a translate. The z dimension, which would otherwise render stretched, is scaled back to a desired size. The turbulence warp in x and y roughens up the 3D text sides leaving the front and back smooth. The +10 accounts for the the buffer of 0.0 values about the 3 deep, in z, df3 data.

Function Fnct01 gets used in the isosurface and the general form here works more or less universally with the density_file pattern when the isosurface threshold is 0.0. Note. While a couple of functions are required for the isosurface set up, the standard pattern modifiers do most of the work.

Performance of various interpolations.

Generally the internal pattern performance in terms of the interpolation numbers runs 0 --> 1 --> 3 --> 2 --> 4 --> 5 from fastest to slowest with a strong dependence on df3 data density, values and ultimate pattern use. As pigments all perform well. Media and isosurface use will amplify the performance differences. With respect to isosurfaces and remembering how the gradients change, it happens for larger df3 values, interpolation 5 can be the fastest because the isosurface gradients are in that case the most gradual.

Tip. Using frequency 0 for better performance.

The density_file pattern like many other patterns supports the frequency, phase and waveform modifiers. If these are not in use, some of the code which does the wave related adjustments can be bypassed by setting the frequency to 0. For example:

#declare Fnct00 = function {
    pattern { density_file df3 "test.df3" interpolate 1 frequency 0 }
}

The performance gains are typically less than 2 percent per density_file and there is one side effect important to maps. The wave code bypassed reduces all values downward by a very small amount to keep values from actually reaching 1.0. Maps wrap back to zero at 1.0 and because users often code maps as [0-1] this wrap might lead to confusion.

The 1.0 value isn't typically an issue for isosurfaces and, if you know the df3 or other pattern as interpolated never reaches 1.0, the frequency 0 trick can be used.

Additional examples and arraycoupleddf3s.inc

POV-Ray ships with one example df3 file in the include directory called spiral.df3 which is used by two sample scenes in: scenes -> textures -> patterns -> densfile.pov and scenes -> interior -> media -> galaxy.pov.

arraycoupleddf3s.inc

Creating a df3 file with arrays.inc's ARRAYS_WriteDF3 macro

Simple bash script to query x, y, z ranges of an existing df3 file.

Orthogonal depth map with trace. Storing +-64 bit values in a df3 file.

Lathe like greebling using density_file pattern with black_hole warps.

Flower formed using multiple black_hole warps on df3, density_file pattern.

Various turbulence warps applied to simple axis crossing df3.

Converting functions and objects to df3s with FunctionToDF3 macro.

Filling a volume with many individually textured shapes.

Isosurace lathe where base df3 created from array of strings.

Place holder for twist example. Perhaps not worth keeping without interpolations 9,11-12?