Programming Project #2: Fun with Filters and Frequencies!

By Sai Kolasani

Introduction

In this project, I produce a "morph" animation of my face into someone else's face, compute the mean of a population of faces, and extrapolate from a population mean to create a caricature of myself and more.

Part 1. Defining Correspondences

First, I cropped and rescaled an image of myself and an image of Tom Cruise to have the same dimensions so that our faces have a similar size and aspect ratio. I then used the spec-provided online tool to select 24 correspondence points for each of the two images (including the four corners). Finally, I generated a Delaunay triangulation at the midway points of the two sets of correspondence points.

Results

Part 2. Computing the "Mid-way Face"

In this section, I compute the "mid-way face" by averaging the geometry and appearance of two input images. The steps to achieve this are outlined below:

Methodology Overview

  1. Compute Average Shape:

    The average shape is computed by taking the midpoint of each corresponding keypoint between the two faces. This creates a set of midway key points , representing the geometric average of both faces.

  2. Delaunay Triangulation:

    Next, I apply a Delaunay triangulation to the midway keypoints. This divides the face into triangles, used for affine transformations of both input faces.

  3. Affine Transformation Calculation:

    For each triangle from both original faces, an affine transformation matrix is computed using the computeAffine() function. This matrix maps the triangles from their original positions to the corresponding triangle in the average face.

  4. Warping Faces into the Average Shape:

    Each face is warped into the average shape by applying the affine transformation for each triangle. A polygon mask is generated using polygon() to isolate the pixels in each triangle, and the transformation is applied to warp the original image pixels to the average shape.

  5. Combine Warped Faces:

    Once both faces have been warped into the average geometry, I compute the final "mid-way face" by averaging the pixel values of the two warped images. This gives us a face with the average geometry and a blend of the colors from both input images.

Part 3. The Morph Sequence

In this part, I create a morph sequence between two images (A and B) using the key points and the Delaunay triangulation structure from Part 2. The morph sequence smoothly transitions between the two images by gradually warping and cross-dissolving them over multiple frames.

Methodology Overview

  1. Warp and Dissolve:

    I use the key points from both images to interpolate between them. The warping is controlled by warp_frac, which transitions the shape between the two images, while dissolve_frac controls the blending of the colors from both images.

  2. Morph Function:

    The morph() function takes two images, their key points, and the Delaunay triangulation to warp each face into an intermediate shape. For each frame, warp_frac and dissolve_frac are adjusted to gradually transition between the original and final images.

  3. Create Sequence

    The morph sequence is generated by creating multiple frames, each corresponding to a different value of warp_fraction and dissolve_fraction in the range [0,1].

Part 4. The "Mean face" of a population

In this part, I compute the average face shape from a population dataset and morph each face into this average shape. I then display the final mean face and provide comparisons between your face and the average geometry.

Methodology Overview

  1. Compute the Average Shape:

    I first compute the average shape by taking the mean of the key points for each face in the dataset. This results in a set of average key points representing the mean geometry of the population.

  2. Morph Each Face into the Average Shape:

    For each face in the dataset, I use Delaunay triangulation on the average key points and morph the face to match the average shape. This is done by warping the facial features using an affine transformation for each triangle.

  3. Compute the Mean Face:

    I compute the mean face by averaging all the morphed faces. The resulting mean face image shows the population's average facial geometry and texture.

Below are 3 example faces of people in the population and the population mean face.

Face Morphs

Part 5. Caricatures: Extrapolating from the mean

In this part, I produce a caricature of my face by exaggerating the differences between my face and the population mean. I compute an extrapolated version of my key points and morph the original face into this exaggerated geometry.

Methodology Overview

  1. Extrapolate from the Mean:

    I compute the exaggerated key points by extrapolating the differences between my face's and the mean face's key points. This is done using a scaling factor, alpha, which determines how much to exaggerate the features. A value of alpha = 1.4 is used in this example.

  2. Perform Delaunay Triangulation:

    I apply a Delaunay triangulation on the new, exaggerated key points to divide the face into triangles. This allows us to warp the face to match the new caricature geometry.

  3. Morph the Face into the Caricature Geometry:

    Using the affine transformation for each triangle, I morph the original face into the exaggerated key points. The result is an exaggerated version of my face or a caricature.

Notes

Notice how when my face was warped to match the average geometry, it was widened, but the caricature had the opposite effect where my face became much more narrow. Particularly shrinking my face to a smaller size and expanding the corner triangles.

Bells and Whistles: Attempting to convert myself to a woman

In this part, I attempt to change the gender of the headshot of myself to appear feminine and with blue eyes. As can be seen by the results below, I look much more feminine in the blended image.