SciVis 2017 Project 2: "mapr"

Assigned Sun Jan 15; Due Thu Jan 26 at 9:00pm

The basic goal of this project is to generalize to two dimensions the abstraction implemented in the first project: a continuous field (reconstructed by convolution with discretely sampled data) that may be evaluated and differentiated at arbitrary points in world-space. The only kind of image that you'll have to handle a 1-channel (gray-scale) image of floats. We also include colormapping, shading (based on the gradient), and 2-D isocontouring via "Marching Squares". In the end you will have a utility that can create nice shaded topographic maps.

• Getting Started
• What to do
• For 33710 students
• Example mapr/rmapr commands to try
• Grading

Getting Started

You can work individually or in pairs for this project. All individual CNETID-scivis-2017 directories should now be populated with a p2mapr directory containing the files for this project. Use https://work-groups.cs.uchicago.edu to work with another student (our class will be identified as 23710 even for 33710 students). Once the joint repository is created for your work together on a project, that project directory in your individual repository will be ignored by graders. If you partnered with the same person for previous project, you will re-use the same pair repository created then. If you are able to create a pair repo, but it is not seeded with the p2mapr files, ask a question in the p2mapr folder on Piazza.

• (these steps copied from the previous project) See the SVN for scivis-2017 page for instructions on getting the necessary repositories within a terminal on a CSIL Mac, and setting environment variable $SCIVIS.
• "cd CNETID-scivis-2017", where CNETID is your CNetID (or the pair of CNetIDs identifying your joint repository), to access your repository.
• "cd p2mapr" to get to where you will modify and commit files.
• Running "make" should build the mapr executable, which you run with "./mapr". If you can't compile or run mapr from the distributed files, tell us in the p2mapr folder on Piazza.
• Use the provided reference executable rmapr for comparison and debugging.

What to do

Structurally, this is set up like the previous project. The specification of what to do lies mainly within the given source files, and executable mapr. Run "./rmapr" to review the commands available, and run "./rmapr about" to see the structs to finish defining, and the functions to implement. You may add code within indicated student code blocks. Read the comment blocks above the functions you finish implementing to understand the technical definition of what a correct implementation should do. The reference implementation rmapr should conform to this. For each of the mapr X commands corresponding to work you have to do, carefully read the usage information (generated by "./mapr X") to understand the purpose of the command and the meaning of its command-line options. You may not create new source files; grading will be done with the distributed Makefile, .integrity.sh, and .strip.sh files.

Unlike in the first project, you may also edit the header file (mpr.h) to augment the definition of the mprContext struct, to include resources (buffers) or state that you want to set on a per-image or per-kernel basis, independent of where the convolution is being evaluated. Resources allocated in mprContextNew should be cleaned up in mprContextNix. Thus, your implementation of mprConvoEval will involve making multiple coordinate changes to multiple files, unlike the more strictly sequential nature of work in the first project. In all the .c files that you may edit, you may add new #define macros or new functions, but they should be static functions (only for use within that source file).

Here is a prioritized listing of the source files involved:

• Must read and understand: mpr.h, mprMath.h
• Read and understand because you're adding code: ctx.c, convo.c, mapr_sdg.c, picture.c, march.c
• Skim to understand the functions if their purpose isn't clear from above: misc.c (implements the dynamically resized mprArray needed for mprIsocontour), cmap.c, image.c, mapr_info.c.
• Can skim if interested (the rest of the files): kernel.c, kparse.c, mapr.c, mapr_about.c, mapr_ceval.c, mapr_cma.c, mapr_cmrs.c, mapr_eps.c, mapr_foo.c, mapr_iso.c, mapr_klist.c, mapr_psamp.c

All together, about 380 lines were stripped from the reference solution to generate the student code. As before, do not take too seriously the comments about how many lines were in the reference version of a student code block; they are only for rough guidance.

The code for mprPictureSample() in picture.c may be especially informative to read through. It shows how:

• different "modes" (different values of the mprMode enum) generate different kinds of images (in terms of img->type and img->channel)
• a view of world-space within a rectangular window specified by arguments size0,size1 (image size), centX, centY (rectangle center), fov (vertical field of view), and angle (rectangle orientation).
• how the output image index space is sampled, and what happens at each sample: first mprConvoEval(), and then mprPictureEval(), which draws on the information learned by convolution and stored in the mprContext.

mprPictureEval() in picture.c is also good to understand, since it shows the context of calling mprPictureRGB(), which you have to implement.

For 33710 students

For this project there is some additional work for 33710 students to do relative to 23710 students:

• The mprIsocontour implementation should avoid computing isocontour segments for cells (i.e. the squares between four adjacent data samples) that lie outside the rectangular view window defined by the -sz, -c, -fov, and -a options to mapr iso (which set the size0, size1, centX, centY, fov, and angle arguments to mprIsocontour()).
• The mprIsocontour should gracefully handle the case of having one cell corner's value be exactly equal to the given isovalue. That means that a zero-length segment should not be generated there, nor should adjacent segments disappear (relative to what would be seen were the isovalue a tiny bit higher or lower).
• The interpolation between colormap control points in HSV space correctly handles that hue=0 is the same as hue=1 (these are two ways of identifying the same point on a circle). When linearly interpolating between two points on the hue circle, you should choose the shorter of the two paths to move along. So, interpolating from hue 0.86 to hue 0.03 should be like interpolating from 0.86 to 1.03, except that interpolated hues above 1.0, or below 0.0, are wrapped back into [0,1], by subtracting or by adding one, respectively, before conversion to RGB.

Example mapr/rmapr commands to try

The commands here use "./rmapr"; you should make sure that you get the exactly same results by running "./mapr". NOTE: There are new files for this project in the scivis-2017 repository, which below will be referred to below as "$SCIVIS". There are sample datasets in data/2d (see the 00-info.txt there for details) and sample colormaps in cmap.

• Use mapr info to learn about the geometry of the images your code processes. You can use this command as is, or you can augment the information computed and printed if you want (modifying mapr_info.c). Try "./mapr info $SCIVIS/data/2d/tiny.nrrd". The output of mapr info includes (at the end) parameters you can pass to mapr psamp that define a view rectangle for for viewing the image.
• You will debug your convolution code with different samplings of known synthetic functions, which you create with mapr sdg (synthetic data generation). mapr sdg -w 0, for example, samples f(x,y)=x. Regardless of the grid orientation or sample spacing, reconstructing by convolution at (x,y) should give back x, and the gradient should always be (1,0), within floating-point rounding errors. Here are some different samplings of f(x,y)=x; for all them points within a circle of radius 0.5 around the origin can be reconstructed with even a 6-support kernel. After you read through the usage info of "./mapr sdg" you should try others that you believe fully exercise your reconstruction code.

  ./mapr sdg -w 0 -sz 10 10 -fov 10 -scl 1 -1 -a 0 -sh 0 0 -o x0.nrrd
  ./mapr sdg -w 0 -sz 14 14 -scl 1 -1 -a 0 -sh 0 0 -o x1.nrrd
  ./mapr sdg -w 0 -sz 22 23 -scl 0.66 1.66 -a 0 -sh 0 0 -o x2.nrrd
  ./mapr sdg -w 0 -sz 22 23 -scl 0.66 1.66 -a 30 -sh 0 0 -o x3.nrrd
  ./mapr sdg -w 0 -sz 32 33 -scl 0.66 1.66 -a 30 -sh 0.4 0.4 -o x4.nrrd
  ./mapr info x?.nrrd
  

  Note how the ItoW matrix gets increasingly messy for these datasets, and how the axes of x4.nrrd are not perpendicular (the sampling grid is sheared). Yet because they all sample the same underlying function, the results of convolution should be the same (within floating-point roundoff).

  for I in 0 1 2 3 4; do
    ./mapr ceval -i x${I}.nrrd -k tent -w 0.22 0.33 -g true
  done
  

  Try all these with the different kernels (as listed by "mapr klist") in place of "-k tent". Note that "-k box" will not correctly reconstruct f(x,y)=x or its derivative.
• You can use the psamp command in its default mprModeValue mode ("-m v") as a thin wrapper around mprConvoEval, to test mprConvoEval. For example, to see the results of nearest-neighbor interpolation from a low-res sampling of a parabola function
  

  ./mapr sdg -w 2 -sz 12 23 -scl 1 0.5 -a 20 -sh 0.2 0 -o parab.nrrd
  ./rmapr psamp -i parab.nrrd -sz 300 300 -c 0 0 -fov 1.4 -a 0 -k box -o parab-v.nrrd
  unu 2op exists parab-v.nrrd 0 | unu quantize -b 8 -o parab-v.png
  open parab-v.png
  

  The "unu 2op exists" usage says "where the value in parab-v.nrrd is finite (not NaN), use it, otherwise use 0". This is then quantized to 8-bits for visual inspection.
• Debugging your mprConvoEval may be easier with some way of seeing other pre-requisites and results, like the world-space position probe (-m wpos), the index-space position (-m ipos), and the value of ctx->outside (-m out). Using the above parab.nrrd:

  for M in wpos ipos out; do
    ./rmapr psamp -i parab.nrrd -sz 300 300 -c 0 0 -fov 1.4 -a 0 -k ctmr -m $M -o parab-$M.nrrd
  done
  unu pad -i parab-wpos.nrrd -min 0 0 0 -max 2 M M -b wrap | unu quantize -b 8 -o parab-wpos.png
  unu pad -i parab-ipos.nrrd -min 0 0 0 -max 2 M M -b wrap | unu quantize -b 8 -o parab-ipos.png
  unu quantize -b 8 -i parab-out.nrrd -o parab-out.png
  

  You should have a sense of why parab-out.png looks the way it does: the values increase as one goes further and further outside where convolution can be done (the black interior). The images of world (parab-wpos.png) and index (parab-ipos.png) map the first and second component of the 2-vector to the magenta (RGB=(255,0,255)) and green (RGB=(0,255,0)) components of the output image, which may offer some qualitative feedback. It might be more informative to probe specific values in parab-wpos.nrrd and parab-ipos.nrrd:

  unu slice -i parab-wpos.nrrd -a 1 2 -p 45 80 | unu save -f text
  unu slice -i parab-ipos.nrrd -a 1 2 -p 45 80 | unu save -f text
  

  This prints out the 2-vectors at [fast,slow] index [45,80] in parab-wpos.nrrd and parab-ipos.nrrd. This is a pixel that is black in parab-out.png. The results are consistent with a call to mapr ceval:

  ./rmapr ceval -i parab.nrrd -w -0.48766667 0.32433331 -k tent
  

  We can confirm that at this location ctx->outside was zero:

  unu slice -i parab-out.nrrd -a 0 1 -p 45 80 | unu save -f text
  

  Note -a 0 1 instead of -a 1 2 because this is an array of scalars, not 2-vectors (on the fastest axis).
• One test of correctly handling image orientation is enabled by the different ways of showing the "2-by-3" (or "3-by-2") matrix of integers from 0 to 5 in the same locations in world space. This is done by the eight six?.nrrd datasets in $SCIVIS/data/2d/, all of which arrange values as so:

  0 1 2
  3 4 5
  

  We can see this via:

  rm -f six?.png
  for X in A B C D E F G H; do
     echo six$X.nrrd ====
    ./rmapr psamp -i $SCIVIS/data/2d/six$X.nrrd -k box -c 1 0.5 -fov 2 -sz 300 200 -o tmp.nrrd
     unu minmax tmp.nrrd
     unu 2op exists tmp.nrrd 0 | unu quantize -b 8 -min 0 -max 5 -o six$X.png
  done
  open six?.png
  

  The unu minmax commands should all print exactly:
  

  min: 0
  max: 5
  

  and not anything about "has non-existent values", which would signify that your mprConvoEval code set ctx->outside to non-zero. All the six?.png images should be identical. We can check this visually with "open six?.png" or numerically with "unu cksum six?.png".
• To test your handling of colormaps in mprPictureRGB, you can use the mapr cma command to apply a colormap per-sample, without any convolution (you can see in mapr_cma.c how this is done).
  

  rm -f italy-*.png
  for M in gray bbody helix bow; do
    ./rmapr cma -i $SCIVIS/data/2d/italy.nrrd -cmap $SCIVIS/cmap/$M-cmap.txt -o italy-$M.png
  done
  open italy-*.png
  

  italy-bow.png will reveal whether you are correctly handling interpolation of hue near hue=0, which is the same red as hue=1 (hue values from 0 to 1 trace out a circle). Doing this right is expected of the 33710 students.
• To test your combined handling of colormaps and convolution, start with the datasets x?.nrrd generated above by sampling f(x,y)=x. We use the -imm option to rmaps psamp to fix the range of values that are lerped into the colormap domain, rather than have this be discovered as the range of values in the input image.

  rm -f x?-{gray,helix}.png
  for I in 0 1 2 3 4; do
    ./rmapr psamp -m vc -i x${I}.nrrd -k tent -c 0 0 -fov 1.5 -cmap $SCIVIS/cmap/gray-cmap.txt -imm -0.8 0.8 -o x${I}-gray.png
    ./rmapr psamp -m vc -i x${I}.nrrd -k tent -c 0 0 -fov 1.5 -cmap $SCIVIS/cmap/helix-cmap.txt -imm -0.8 0.8 -o x${I}-helix.png
  done
  open x?-{gray,helix}.png
  

  Try all the other available kernels in place of "-k box"
• To test your handling of gradients (c.f. FSV 4.6), you can use the ceval command to test how rmapr vs your mapr compute gradients at specific locations (of your choosing, in images of your choosing). That is done above with the x?.nrrd images. You can also use the psamp -m g, which selects mode mprModeGradient, to make a whole image of gradients, which are qualitatively visualized by converting the components to an RGB image. We can start by doing various samplings of the parabola function f(x,y)=x^2+y^2 via sdg -w 2 (mimicking what was done to make the x?.nrrd images)

  ./mapr sdg -w 2 -sz 10 10 -fov 10 -scl 1 -1 -a 0 -sh 0 0 -o p0.nrrd
  ./mapr sdg -w 2 -sz 14 14 -scl 1 -1 -a 0 -sh 0 0 -o p1.nrrd
  ./mapr sdg -w 2 -sz 22 23 -scl 0.66 1.66 -a 0 -sh 0 0 -o p2.nrrd
  ./mapr sdg -w 2 -sz 22 23 -scl 0.66 1.66 -a 30 -sh 0 0 -o p3.nrrd
  ./mapr sdg -w 2 -sz 32 33 -scl 0.66 1.66 -a 30 -sh 0.4 0.4 -o p4.nrrd
  rm -f p?.png
  for I in 0 1 2 3 4; do
    ./rmapr psamp -m g -i p${I}.nrrd -k ctmr -c 0 0 -fov 1.5 -o - |
      unu pad -min 0 0 0 -max 2 M M -b wrap |
      unu 2op exists - -0.5 |
      unu quantize -b 8 -min -3 -max 3 -o p${I}.png
  done
  open p?.png
  

  Note that the output of the psamp command is being directly piped into unu pad, which turns the 2-channel image of gradient 2-vectors into a 3-channel image by wrapping the x-component of the gradient into the 3rd channel, which is then quantized to an 8-bit color image, in which gray means zero, magenta means positive X, and green means positive Y. Think through why these images look the way that do: the gradient of f(x,y)=x^2+y^2 is 2*[x,y], so the RGB values of the output are based on [x,y,x], so when x is large and y is small, the color is magenta ([1,0,1]) and when y is large and x is small, the color is green ([0,1,0]).
• To show how gradients can be used for shading a topographic map:

  ./rmapr psamp -i $SCIVIS/data/2d/italy.nrrd -cmap $SCIVIS/cmap/diverg-cmap.txt \
     -imm -2000 2000 -m vcs -l -1 1 1 -zs 7 -sh 0.5 -c 0 0 -fov 1040000 \
     -k bspln3 -sz 660 880 -o italy.png
  open italy.png
  

• The three hemispherical domes producecd by mapr sdg -w 3 are good test cases for the shading effects of mapr psamp -m vcs. domes0.nrrd is a simple isotropic sampling, domes1.nrrd is an anisotropic, sheared, and rotated sampling. The difference in the underlying values is visible in domes?.png The different renderings vary the light direction and the amount of scaling along Z.

  ./mapr sdg -w 3 -c 0.8 0.8 -sz 90 90 -fov 5 -o domes0.nrrd
  ./mapr sdg -w 3 -c 0.8 0.8 -scl -1.4 0.7 -sz 130 170 -a 115 -sh 0 -0.5 -fov 8 -o domes1.nrrd
  rm -f domes?.png rend?-domes?.png
  LDIR=("1 0 0" "0 1 0" "-1 1 1" "-1 1 1" "-1 1 1")
  ZSCL=(1 1 1 5 0.5)
  for D in 0 1; do
     unu quantize -b 8 -i domes${D}.nrrd -o domes${D}.png
     for I in 0 1 2 3 4; do
        echo $I: -l ${LDIR[$I]} -zs ${ZSCL[$I]}
        ./rmapr psamp -i domes${D}.nrrd -cmap $SCIVIS/cmap/white-cmap.txt \
           -m vcs -l ${LDIR[$I]} -zs ${ZSCL[$I]} -sh 1 -c 0.8 0.8 -fov 4 \
           -k c4hexic -sz 500 500 -o rend$I-domes$D.png
     done
  done
  open domes?.png; open rend?-domes?.png
  

• The output of mapr psamp and mapr iso can be combined by mapr eps into an EPS file (Encapsulated PostScript), a format that can combine vector and raster graphics in a simple self-contained way (unlike SVG). mapr psamp saves orientation meta-data in the its output PNG file for this purpose. You can use all this to test your isocontouring code. Start with a tiny dataset for which you can compute the isocontour segments by hand, and then make sure they agree with what your code does: $SCIVIS/data/2d/tiny.nrrd. First, look at its properties:

  ./rmapr info $SCIVIS/data/2d/tiny.nrrd
  

  You can also look at the file itself, to see how those properties are stored on disk:
  

  cat $SCIVIS/data/2d/tiny.nrrd
  

  Then to make a picture of those values, with box and tent kernels:

  rm -f tiny-*.png
  for K in box tent; do
    ./rmapr psamp -i $SCIVIS/data/2d/tiny.nrrd -m vc -cmap $SCIVIS/cmap/bbody-cmap.txt \
    -k $K -c 0 0 -fov 1 -sz 400 400 -o tiny-$K.png
  done
  open tiny-*.png
  

  Reconstructing with tent is the same linear interpolation along grid edges that is used by Marching Squares, and within cells the resulting bilinear interpolation determines the correct topology of Marching Squares results. You compute isocontours with mapr iso and draw them with mapr eps:

  ./rmapr iso -i $SCIVIS/data/2d/tiny.nrrd -v 1.5 -o seg-1.txt
  ./rmapr eps -i tiny-tent.png -c seg-1.txt -th 0.01 -o tiny-iso-1.eps
  open tiny-iso-1.eps
  

  NOTE: On Macs, when you open an EPS file, it converts it to PDF for display, but unlike when you open a PNG image, changes to the EPS file after opening will not trigger a redisplay of the updated file: you have to close the window and re-open the EPS file. You can compute multiple isocontours with one mapr iso, and then pass these to mapr eps:

  ./rmapr iso -i $SCIVIS/data/2d/tiny.nrrd -v {0,1,2,3,4,5}.5 -o seg-{0,1,2,3,4,5}.txt
  ./rmapr eps -i tiny-tent.png -c seg-{0,1,2,3,4,5}.txt -th 0.005 -o tiny-isos.eps
  open tiny-isos.eps
  

• To make sure that the isocontouring code correctly handles all the cases correctly (the 16 cases of the cell corners being above or below the isocontour), its nice to have a small dataset that exercises all those possibilities. $SCIVIS/data/2d/allcases.nrrd was created for this purpose, at isovalue 0.5. We can first visualize what the isocontour should look like, with a colormap that selects a very narrow range of colors, and otherwise uses a clear threshold:

  ./rmapr psamp -i $SCIVIS/data/2d/allcases.nrrd -k tent -m vc \
    -cmap $SCIVIS/cmap/thresh-cmap.txt -sz 600 520 -c 9.5 9 -fov 19 -o allcases-cmap.png
  open allcases-cmap.png
  

  And then computing and drawing an isocontour on top of allcases-cmap.png:

  ./rmapr iso -i $SCIVIS/data/2d/allcases.nrrd -v 0.5 -o seg.txt
  ./rmapr eps -i allcases-cmap.png -c seg.txt -scl 1.3 -th 0.04 -o allcases.eps
  open allcases.eps
  

  Note how the black computed isocontour segments should agree with the white approximate isocontour generated via bilinear interpolation and then colormapping. The agreement should be exact at the cell boundaries. In the "ambiguous" cases the colormapped isocontour will have a more curved shape rather than a straight line, but the topology should be correct nonetheless. The test dataset is designed to present the ambiguous cases at all orientations and with both signs (left and right side of image).
• Another way to approximate the appearance of isocontouring is afforded by mapr psamp -m fiso, for "fuzzy isocontours". This is like a colormap, in that it is computed per-pixel, but it uses both the value and the gradient magnitude to draw an approximately constant thickness band of bright pixels.

  ./rmapr psamp -i $SCIVIS/data/2d/allcases.nrrd -k tent -m fiso -fiso 0.5 0.08 \
    -sz 600 520 -c 9.5 9 -fov 19 -o - | unu quantize -b 8 -min 0 -max 1 -o allcases-fiso.png
  open allcases-fiso.png
  

  The thickness of the outline drawn by the bright pixels here is much more consistent than produced by the colormap (of value alone) in allcases-cmap.png.
• Having fun with the Italy dataset again:

  ./rmapr psamp -i $SCIVIS/data/2d/italy.nrrd -cmap $SCIVIS/cmap/diverg-cmap.txt \
     -imm -2000 2000 -m vcs -l -1 1 1 -zs 7 -sh 0.5 \
     -k bspln3 -sz 990 1320 -c 0 0 -fov 1000000 -o italy2.png
  rm -f italy-iso*.txt
  iso=( -1800 -1200 -0600 00000 +0600 +1200 +1800 )
  isoc=$(printf ",%s" "${iso[@]}")
  isoc=${isoc:1} # all isovalues joined by comma
  eval ./rmapr iso -i $SCIVIS/data/2d/italy.nrrd -v ${iso[@]} -o italy-iso{$isoc}.txt
  eval ./rmapr eps -i italy2.png -c italy-iso{$isoc}.txt -th {300,300,300,1000,300,300,300} -o italy.eps
  open italy.eps
  

  This makes a decent quality shaded isocontour map. The isocontour at zero elevation (aka the coastline) is thicker than the others. The large number of isocontour segments means that the file is a little slow to open. Once open, though, you can zoom in and pan around to see the kinds of details that the isocontouring as picked up.
• The same Italy map provides an opportunity to test (for 33710 students) the ability to isocontour only a subset of the dataset. In italy.eps above you can see some isocontour segments extending beyond the bounds of what was sampled in by italy2.png. These commands repeat the above, but only work on what is visible inside a smaller rotated view window.

  VIEW="-sz 760 480 -c 130000 -210000 -fov 330000 -a 50"
  ./rmapr psamp -i $SCIVIS/data/2d/italy.nrrd -cmap $SCIVIS/cmap/diverg-cmap.txt \
     -imm -2000 2000 -m vcs -l -1 1 1 -zs 7 -sh 0.5 \
     -k bspln3 $VIEW -o italy-zoom.png
  rm -f italy-zoom-iso*.txt
  iso=( -1800 -1200 -0600 00000 +0600 +1200 +1800 )
  isoc=$(printf ",%s" "${iso[@]}"); isoc=${isoc:1}
  eval ./rmapr iso -i $SCIVIS/data/2d/italy.nrrd -v ${iso[@]} $VIEW -o italy-zoom-iso{$isoc}.txt
  eval ./rmapr eps -i italy-zoom.png -c italy-zoom-iso{$isoc}.txt -th {300,300,300,1000,300,300,300} -o italy-zoom.eps
  open italy-zoom.eps
  

Grading

The grade will be based on correctness (80%) and style (20%). If your code does not compile with the provided Makefile, .integrity.sh, .strip.sh, you will get a ZERO for correctness. Correctness will be evaluated with examples such as those above, and maybe some additional tests for corner cases. For the style points:

• Make sure your final code compiles without warnings
• Avoid needlessly recomputing things with every call to mprConvoEval. This is why you're allowed to add variables and state to the mprContext. Resources allocated in mprContextNew should be cleaned up in mprContextNix.
• As noted in FSV 4.5, for a kernel with support s, the number of kernel evaluations should be 2s, not s^2.
• If your implementation mptConvoEval and mprPictureSample runs markedly slower than the reference implementation, you may lose some points. The rate (in kilohertz or kHz) of the computation is printed by mapr psamp.