User:Wfpokorny/DensityFile
Note! While the following density_file (.df3) pattern documentation is often valid with respect to the current 3.7.0 forms of df3 value interpolation, it is written as if the additional interpolation modes proposed in https://github.com/wfpokorny/povray/tree/feature/newDensityPatternInterpolations have been adopted for the 3.7.1 release.
Note! After some discussion it's been decided the 'assist' interpolation modes 6-12 will be dropped for methods already supported. A quick edit has been done to this end. I'll update the example code later as I have time. The code update itself will come last.
Density File Pattern
The density_file pattern is a 3-D bitmap pattern that occupies a unit cube from location <0,0,0> to <1,1,1>. The data file is a raw binary file format created for POV-Ray called the df3 format. This pattern was originally created for use with halo or media, but it may be used anywhere a pattern may be used. The core syntax whether used in a normal, pattern or pigment block is:
density_file df3 [file name] [interpolate Type]
where the file name would be placed in double quotes and the interpolation type would be an integer value 0 through 5. For example:
density_file df3 "star.df3" 1
The general forms for each of the normal, pattern and pigment blocks are:
normal {
density_file df3 [filename] [,Bump_Size]
[interpolate Type]
[NORMAL_MODIFIERS...]
}
where Bump_Size is an optional float value,
pattern {
density_file df3 [filename] [interpolate Type]
[PATTERN_MODIFIERS...]
}
and
pigment {
density_file df3 [filename] [interpolate Type]
[PIGMENT_MODIFIERS...]
}
The df3 file format
The df3 format consists of a 6 byte header of three 16-bit, unsigned integers with high order bytes first (big-endian order). These three values give the x,y,z size of the data in voxels.
The header is followed by x*y*z unsigned, big-endian encoded, integers with resolutions of 8, 16 or 32 bits. The resolution of the data is determined by the size of the df3 file. That is, if the file - minus the header - is twice as long as an 8 bit file, it is taken to contain 16 bit data. If it is four times as long, it is taken to contain 32 bit data.
How the density_file pattern works
The density pattern occupies the unit cube <0,0,0> to <1,1,1> regardless of the x, y, z dimensions in voxels. It remains at 0.0 for all areas beyond the unit cube. Data values in the ranges of 0 to 255 (8 bit/1 byte), 0 to 65535 (16 bit/2 bytes) and 0 to 4294967295 (32 bit/4 bytes), are scaled by the various interpolation methods to a float value in the range 0.0 to 1.0. The larger 2 byte and 4 byte depths providing better resolution.
Interpolation options.
There are currently 6 data interpolation types numbered 0 through 5, with the types larger than 2 being available only in releases 3.7.1 and later. If nothing is specified via the interpolate keyword, the default is 0 which does no interpolation - meaning for a given location in the unit cube, the 0.0 to 1.0 value of the containing voxel is returned. The other interpolation methods available are:
- Tri-linear. Similar to two dimensional image method, but for three dimensions.
- Tri-cubic. Similar to two dimensional image method.
- Weak exponential blobbing of defined voxels in 4x4x4 local sub regions.
- Moderate exponential blobbing of defined voxels in 6x6x6 local sub regions.
- Strong exponential blobbing of defined voxels in 8x8x8 local sub regions.
If an interpolate value not supported is specified, the interpolation defaults to tri-cubic interpolation, 2, in POV-Ray versions 3.6 and later.
Exponential blobbing interpolations.
Exponential blobbing techniques in POV-Ray date to the introduction of isosurface objects. The method here borrows from one common technique with the form:
1/exp((function's value)*(blobbing strength))
As implemented, for each evaluated point in the unit cube, the code sums the result of the equation:
(1/exp(pow(point_to_voxel_distance,2)*(internalTuningBias/voxel_value))
for all adjacent df3 values >0 within the NxNxN sub-cube around the evaluated point. Resulting values >1 are clamped to 1.0. The larger the sub cube, the stronger the blobbing which can be accommodated.
The algorithm has been tuned so 1.0 values for each of the exponential blobbing interpolations just fit within the corresponding sub cube. The blobbing can be adjusted without the need to regenerate a df3 by using the standard pattern wave modifier poly_wave <float>. Poly_wave values >=1.0 should always work and reduce the blobbing. Values < 1.0 increase the blobbing and can cause sub cube overruns. Overruns will appear as discontinuities in the final pattern.
A note on a possible extension to the density_file exponential blobbing methods. These today are based upon the square of the distance to nearby voxels with a value >0.0 and a strength blobbing factor based upon the value of those >0.0 voxels. When blobbing, the deeper 2 byte and 4 byte df3 formats are not often used. It is possible, with the 2 byte and 4 byte depths, to use 1 byte for the blobbing strength factor, and the remaining byte(s) to offset the evaluated voxel center for the distance squared component. At the 32 bit depth there would be 8x8x8 or 512 possible sub-voxel positions. Unsure of the ultimate performance for such code, the modeling/creating of df3s is certainly less straight forward, and such a method would be restricted to sparse data representations. Up to a 3x3x3, same weight (1/32 accuracy), bit representation at the 32 bit depth would be available too, if density of voxels were the aim.
Only interpolation 0 is voxel centered in 3.7.0 and earlier releases.
For version 3.7.0, the interpolations 0 to 2 are shown below in order, left to right, as a pigment on a plane for an 11x11x11 df3 file where a single center voxel has a value of 0.5 while all others are 0.0. The red dots mark the vertices around our center voxel.
The tri-linear and tri-cubic interpolations are vertex centered prior to 3.7.1 resulting in a half voxel negative shift in x,y,z . This shift is perhaps of no consequence if rendering media. It does however matter if, for example, building a df3 based isosurface or when trying to control textures for media for any formed objects using interpolation 0.
Offset tri-linear and tri-cubic results can be centered with a positive translation of one half a voxel size in x,y and z. For the 11x11x11 df3 example use:
translate <-0.5+(1/11/2),-0.5+(1/11/2),-0.5+(1/11/2)>
to get the density_file pattern exactly centered upon the origin. In a function for an isosurface this adjustment to center on the origin would be something like:
FnctAdjustToCenter(x+0.5-(1/11/2),y+0.5-(1/11/2),z+0.5-(1/11/2))
Release 3.7.1 makes all interpolations voxel, and so too unit cube, centered.
In addition to adding density_file pattern interpolation modes, 3.7.1 centers all the interpolated results. The 3.7.1 interpolations 0 to 5 are shown in order, left to right, in the following image.
Tri-linear interpolation has creases, tri-cubic interpolation wobbles/rings.
The following figure shows isosurfaces for interpolations 1 through 5, left to right, setting the threshold very near zero to better see the non-zero values returned. The df3 used is a 15x15x15 arrangement where an x,z box has been centered in y and through which there is a y column of >0.0 values. The values for the non-zero voxels in the top row are 0.1 and the bottom row 0.9.
The former better showing the crease effect of tri-linear interpolation, the latter, the ring of tri-cubic interpolation. The tri-cubic mode always wobbles, but the most severe artifacts can be avoided by using values not near 0.0 and 1.0 where pattern clipping occurs.
Lastly, with respect the the ringing of the tri-cubic interpolation, there are at times, additional harmonics as shown at the bottom of the following image which has been gamma adjusted to make this clear. The df3 sets only 3 voxels, here seen as the brighter green rectangles.
DF3 value and density_file interpolation effects on an isosurface max gradient.
Interpolation methods and df3 values affect the max gradient in isosurface use. For tri-linear and tri-cubic Interpolations 1 and 2, lower values result in lower gradients due the typically longer ramp from the >0 edge to the voxel value used as a threshold. With the exponential blobbing of modes 3, 4 and 5, the lower gradients happen instead at larger voxel values - lower voxel values result in small spheres with much sharper local gradients. Gradients for interpolations 0-5 tend to increase as the df3 file's voxel count increase.
Interpolations 3, 4 and 5 max values always go to 1.0 at any >0.0 voxel center, where interpolations 1 and 2 peak at the voxel's specified >0.0 value.
Reported maximum gradients for df3 based isosurfaces will often be much higher than what can be used. These higher gradients occur where a ray catches the far corner or edge of the df3-function in a direction more perpendicular to the best overall local gradient. Note. This far corner, near threshold, gradient effect happens with other patterns too and can often be ignored in practice.
Use a buffer of five 0.0 voxels on all sides to avoid density_file edge effects.
While not a hard and fast rule, all the interpolations other than 0, and 6-12, will wrap or repeat values on the sides. The general recommendation of five comes from interpolation five's need for an 8 grid range (4 per side) about any evaluated point - plus a half voxel grid for the internal vertex to voxel center correction of interpolations 1-5. Often smaller buffer amounts can be used.
Below are two images. In the first, the lower-left-forward corner of the df3 has been filled with 0.2 values with all others 0.0. In the second image the top-right-back corner of df3 file has been set to values of 0.2 with all others set to 0.0. Interpolations 0 through 5 are shown left to right in each of these two images as a pigment on an x,y plane at the df3 unit cube's z center.
All truncate sharply to 0.0 where values have been run to the unit cube sides. This abrupt change is especially inconvenient for isosurface use, but the hard edge can often be seen in other pattern uses as some artifact.
The tri-linear and tri-cubic interpolations, 1 and 2, wrap side to opposite side in the lower-left-forward side-abutted case shown in the first row. These same interpolations truncate in the rop-right-back side-abutted case shown in the second row.
The exponential blobbing of 3 through 5 all repeat - or add values - at the last voxel position at the sides. This can sometimes be compensated for by lowering the on side and in-corner values.
df3 file size, 2D extrusion, twist and lathe methods.
Df3 files can get very large, very quickly given the cubed dimensions. A df3 1000x1000x1000 at 1 byte resolution will take a gigabyte of storage. It practice, aim to reduce the z dimension as much as possible letting the density_file pattern scale up to the unit cube. Later scale the unit cube dimension to the desired final size.
Twist shallow in z, density_file pattern.
A 90x90x15 df3 is created with 12 non-zero entries. Using the gradient z interpolation 9 and the two twist about z interpolations in 11 and 12 a four part twist about the z axis running through the center of the unit cube is created. The core code:
#declare Fnct00 = function {
pattern { density_file df3 "I3_I9an11an12_Twist.df3" interpolate 3 }
}
#declare Fnct00x = function {
pattern { density_file df3 "I3_I9an11an12_Twist.df3" interpolate 11
turbulence 0.1 octaves 3 lambda 3 omega 0.7 }
}
#declare Fnct00y = function {
pattern { density_file df3 "I3_I9an11an12_Twist.df3" interpolate 12
turbulence 0.1 octaves 3 lambda 3 omega 0.7 }
}
#declare Fnct00z = function {
pattern { density_file df3 "I3_I9an11an12_Twist.df3" interpolate 9
turbulence 0.1 octaves 3 lambda 3 omega 0.7 }
}
#declare Fnct01 = function (x,y,z) {
0.025-Fnct00(Fnct00x(x,y,z),Fnct00y(x,y,z),Fnct00z(x,y,z))
}
#declare Magenta = srgbft <1,0,1,0,0>;
#declare Iso00 = isosurface {
function { Fnct01(x,y,z) }
contained_by { box { 0.0,1.0 } }
threshold 0
accuracy 0.0005
max_gradient 6
all_intersections
pigment { color Magenta }
}
with Fnct01 representing the final twisting result where a turbulence was also applied. The depth of 15 has been stretched in z by the density_file pattern to fill the 0 to 1 range. The df3 here is 6 times smaller than it would be if 90x90x90.
Note. The twist direction about the unit cube's z axis is left by default, but can be flipped to a right twist by changing all z call parameters to 1-z in the body of Fnct01.
The resulting image:
3D isosurface extrusion of image via shallow in z, density_file pattern.
A second example compressing the df3 in one dimension comes naturally when the density_file pattern is used for extrusion of a 2D image into 3D shape. The outline for such an extrusion is shown in the following code snippets with a resulting image. A 500x500x13 deep df3 of the red in a 2D image, at left below, is created with the following code:
#include "arrays.inc" // For ARRAYS_WriteDF3()
#include "arraycoupleddf3s.inc" // Array coupled df3 macros & functions.
#declare ImageFileNamesAry=array[3] {
"BlackRedLettersIn.png",
"BlackRedLettersIn.png",
"BlackRedLettersIn.png"
}
// Red
ConvertImagesToDF3(500,500,0.0,1.0,<1,0,0>,0.2,1,ImageFileNamesAry,"BlackRedLetters.df3",8,5)
#error "Stopping after parse stage for df3 creation from image."
The include file arrays.inc is already part of POV-Ray and the arraycoupleddf3s.inc file is given in the examples section. Once the df3 representation of the 2D image exists, the following code is used to create the function for the isosurface text using blobbing interpolation 5.
#declare Fnct00 = function {
pattern { density_file df3 "BlackRedLetters.df3" interpolate 5
translate -0.5 // Center about origin for isosurface
scale <1,1,1/((500+10)/(3+10))>
warp { turbulence <0.015,0.015,0> octaves 6 omega 0.43 lambda 6 }
}
}
#declare Fnct01 = function (x,y,z) {
0.025-Fnct00(x,y,z)
}
In the function declaration, Fnct00, the pattern is centered about the origin with a translate. The z dimension, which would otherwise render stretched, is scaled back to a desired size. The turbulence warp in x and y roughens up the 3D text sides leaving the front and back smooth. The +10 accounts for the the buffer of 0.0 values about the 3 deep, in z, df3 data.
Function Fnct01 gets used in the isosurface and the general form here works more or less universally with the density_file pattern when the isosurface threshold is 0.0. Note. While a couple of functions are required for the isosurface set up, the standard pattern modifiers do most of the work.
Array of strings to isosurface lathe via shallow in z, density_file pattern.
Interpolation 10 is a 2 D gradient in x and z. However, its first purpose is to implement lathe effects based upon substantially compressed in z, df3 files.
With existing lathe and sor methods, the 2D values to be rotated would be to the right of the y axis running through the origin. Similarly, the density_file interpolation 10 rotates values to the right of the y axis at x,z=0.5 within the density_pattern's unit cube.
Aiming for a bowl, start by using another macro part of the arraycoupleddf3s.inc set called, ConvertArrayOfStringsToDF3, which creates df3 files from and array of text strings. The code to create the bowl df3 file looks like this:
#include "arrays.inc" // For ARRAYS_WriteDF3() #include "arraycoupleddf3s.inc" // Array coupled df3 macros & functions. #declare AryStrs = array[25][1] #declare AryStrs[0][0] = "-------------------------------------2------------"; #declare AryStrs[1][0] = "------------------------------------3-------------"; #declare AryStrs[2][0] = "-----------------------------------2--------------"; #declare AryStrs[3][0] = "----------------------------------2---------------"; #declare AryStrs[4][0] = "---------------------------------2----------------"; #declare AryStrs[5][0] = "--------------------------------3-----------------"; #declare AryStrs[6][0] = "-------------------------------4------------------"; #declare AryStrs[7][0] = "-------------------------------5------------------"; #declare AryStrs[8][0] = "------------------------------4-------------------"; #declare AryStrs[9][0] = "-----------------------------5--------------------"; #declare AryStrs[10][0] = "----------------------------6---------------------"; #declare AryStrs[11][0] = "----------------------------6---------------------"; #declare AryStrs[12][0] = "----------------------------6---------------------"; #declare AryStrs[13][0] = "----------------------------6---------------------"; #declare AryStrs[14][0] = "---------------------------7----------------------"; #declare AryStrs[15][0] = "--------------------------7-----------------------"; #declare AryStrs[16][0] = "-------------------------7------------------------"; #declare AryStrs[17][0] = "------------------------7-------------------------"; #declare AryStrs[18][0] = "-----------------------7--------------------------"; #declare AryStrs[19][0] = "-------------------6666---------------------------"; #declare AryStrs[20][0] = "------------------7-------------------------------"; #declare AryStrs[21][0] = "-----------------7--------------------------------"; #declare AryStrs[22][0] = "----------------7---------------------------------"; #declare AryStrs[23][0] = "---------------7----------------------------------"; #declare AryStrs[24][0] = "666666666666666-----------------------------------"; // // <---------------------- 50 ----------------------> ConvertArrayOfStringsToDF3(AryStrs,"StringsToDF3.df3",8,1) #error "Stop before render"
The essential code using the resulting StringsToDF3.df3 looks like:
#declare Fnct00 = function {
pattern { density_file df3 "StringsToDF3.df3" interpolate 5 }
}
#declare Fnct00x = function {
pattern { density_file df3 "StringsToDF3.df3" interpolate 10
warp { turbulence 0.02 octaves 6 omega 0.23 lambda 6 }
}
}
#declare Fnct02a = function (x,y,z) {
0.025-Fnct00(x,y,z)
}
#declare Fnct02b = function (x,y,z) {
0.025-Fnct00(Fnct00x(x,y,z),y,0.5)
}
#declare BrownOchre = srgb <0.7216,0.3059,0.1569>;
#declare Iso00 = isosurface {
function { Fnct02b(x,y,z) }
contained_by { box { 0.0+1e-6,1.0-1e-6 } }
threshold 0
accuracy 0.0005
max_gradient 6
all_intersections
pigment { color BrownOchre }
finish { phong 0.9 }
}
#declare Object00 = object {
Iso00
translate -0.5
scale <3.3,0.7,3.3>
}
The images using Fnct02a and Fnct02b for the df3 based isosurface are left and right:
With Fnct02a the isosurface is the stretched-in-z profile created by ConvertArrayOfStringsToDF3. In Fnct02b the x coordinate is replaced with the rotate around y function called Fnct00x. The z is also fixed at 0.5 on the call to Fnct00. For the rough glazed look, a slight turbulence warp was added to Fnct01. Lastly, the isosurface is scaled so as to flatten the bowl's original tall df3 profile.
Performance of various interpolations.
Generally the internal pattern performance in terms of the interpolation numbers runs 0 --> 1 --> 3 --> 2 --> 4 --> 5 from fastest to slowest with a strong dependence on df3 data density, values and ultimate pattern use. As pigments all perform well. Media and isosurface use will amplify the performance differences. With respect to isosurfaces and remembering how the gradients change, it happens for larger df3 values, interpolation 5 can be the fastest because the isosurface gradients are in that case the most gradual.
Tip. Using frequency 0 for better performance.
The density_file pattern like many other patterns supports the frequency, phase and waveform modifiers. If these are not in use, some of the code which does the wave related adjustments can be bypassed by setting the frequency to 0. For example:
#declare Fnct00 = function {
pattern { density_file df3 "test.df3" interpolate 1 frequency 0 }
}
The performance gains are typically less than 2 percent per density_file and there is one side effect important to maps. The wave code bypassed reduces all values downward by a very small amount to keep values from actually reaching 1.0. Maps wrap back to zero at 1.0 and because users often code maps as [0-1] this wrap might lead to confusion.
The 1.0 value isn't typically an issue for isosurfaces and, if you know the df3 or other pattern as interpolated never reaches 1.0, the frequency 0 trick can be used.
Additional examples and arraycoupleddf3s.inc
POV-Ray ships with one example df3 file in the include directory called spiral.df3 which is used by two sample scenes in: scenes -> textures -> patterns -> densfile.pov and scenes -> interior -> media -> galaxy.pov.
Creating a df3 file with arrays.inc's ARRAYS_WriteDF3 macro
Simple bash script to query x, y, z ranges of an existing df3 file.
Orthogonal depth map with trace. Storing +-64 bit values in a df3 file.
Lathe like greebling using density_file pattern with black_hole warps.
Flower formed using multiple black_hole warps on df3, density_file pattern.
Various turbulence warps applied to simple axis crossing df3.
Converting functions and objects to df3s with FunctionToDF3 macro.