Mutation 22

An example of using temporal rank median filtering on the spatially aligned stack of portrait virus images. I'm using limited temporal windows for processing for all of these recent posts (as opposed to processing the entire stack of images).

Mutation 21

Another example of using temporal rank max filter on the spatially aligned stack of portrait virus images.

Mutation 20

One of the advantages of working with the spatially normalized set of portrait images is that you can start to pull out features associated with the portraits in a more intelligent way since all of the facial images are spatially aligned across the stack. Here's an example using temporal rank max filtering on the spatially normalized portrait stack..

Mutation 19

The temporal pixel jumble effect applied to the complete stack of original portraits. Not particularly effective in my opinion, but i thought i'd post one example to show off what it does. I guess there is some rough similarity to the temporal mean rank filter effect in a more pixelated way.

Mutation 48

This portrait was generated by processing the original set of self portraits with the temporal motion rank max filter.

Mutation 18

There's a ton of stuff that can be done by treating the stack of source imagery as a movie brush for building photo mosaics. We'll get more heavily into that later but i thought i'd post a really simple example. I modified the 3.5 Collection Graffiti category to use the original portrait virus images. I used a different brush load image processing algorithm for the colorization, the meanshift1 algorithm seems to have some issues with the dark areas of the siggraph 2010 portraits. So i'll look into that, not sure if it's a bug or an artifact associated with the compression i used for the movie brush.

I think movie background textures may end up being more effective for this set of source imagery since you can use the source portraits at full resolution in the generated texturing to get at all of the detail of the original paint stroke texturing. I had to size the portraits down so small to make a source movie brush from the original portrait stack that you lose most of the paint effects in the individual portraits that make up the mosiac.

Mutation 17

Temporal motion rank filter with the min setting. So we're looking at minimum motion in local neighborhoods across the sequence of original portraits.

Mutation 16

A different set of temporal difference matte parameters applied to the original stack of portrait images. Looking for similarity as opposed to variation in the sequence of portrait imagery.

Mutation 15

The temporal difference matte is a pretty versatile effect, depending on which settings you choose. I'm using the original stack of portraits as input for this effect.

Mutation 14

A good example of a failed experiment. I loved the earlier results i was getting painting of of source edge maps, so i thought i'd use the Color Edge ip op to min composite source edge maps across the entire portrait stack and then interpolate in the white spaces. A big mess in my opinion. Compare it to the original approach below where i paint the generated paths while masking out areas already painted. The masking is the key to avoiding too much build up.

Mutation 13

A real simple variation on building PASeq that processes the entire stack of portraits. I'm using the original portrait stack and just painting with partially transparent circles. I ran the PASeq twice on the stack to build up the image density.

Mutation 12

Another variation on the theme of building a PASeq that processes the entire stack of portraits to generate a final output image. This one works by building source edge maps and then sketching them in. Surprisingly different than some of the other stack filter effects.

Mutation 11

Variation on the previous eye detection experiment, but this time i'm using automatic face detection and then matting the results across the stack with some partial transparency. So i use a PASeq to process the entire stack of portraits to generate the final single image. I used the original portrait stack so that the variation in face positioning would help build up the composite face image.

Mutation 10

Another temporal scan tracker image using a limited range of the overall portrait stack. I'm using the spatially normalized stack in this case. I'm also using a diagonal scan.

Mutation 9

Another example generated with the temporal scan tracker. This time i used the spatially aligned portraits and a smaller number of frames. I think that works better for this set of stack images, it gets rid of the venetian blind effect and generates some interesting composite heads. Depending on where you start the scan in the image sequence you can generate a variety of different effects. Again, i'm blending with maximum energy as opposed to a linear blend.

Mutation 8

Slit scan effect generated by using the temporal scan tracker on the entire stack of original portraits. I'm using maximum energy blending. You get a tighter head when you use the spatially aligned portrait stack, i'll post some of those later.

Mutation 7

This image was generated by using the temporal displacement operation. I started with a set of randomly colored rectangles in a grid. The grayscale values in the squares map a displacement into the stack of portrait frame images. There's an infinite number of different effects like this you can generate. This image was generated off of the original portrait stack.

Mutation 6

This image uses another temporal motion rank filter, but it's choosing the pixel with the minimum motion for each spatial location in the stack of portraits. So it's related to yesterday's post, but kind of the inverse effect. It accents similarity as opposed to difference in adjacent images in the portrait stack.

Mutation 5

This image was generated by using the temporal motion rank filter with the spatially normalized stack or portrait virus paintings. So it analyzes the stack of images and displays the pixel with the maximum motion at each spatial location in the output canvas. Obviously the ordering of the images in the stack is going to influence the result of this kind of processing. So by shuffling the ordering of the images in the stack sequence you could generate different variations of the effect.

Mutation 4

This image is an example that was built from a spatially normalized stack of portrait virus images. By spatial normalization i mean that every painted portrait was spatially warped to a standard portrait facial position. The standard position was just one portrait that i randomly picked out of the 2010 Siggraph portrait virus set. Every other portrait was spatially warped to match the standard position. I'll discuss the mechanics of that process at a later time. The resulting image was generated with the temporal rank filter and can be thought of as the average of all of the portrait virus images. So you could think of it as the generic portrait derived from the entire set.

I've generated another image below that uses the same process, but with the original non-spatially normalized portraits as the input stack. Note that it's much more of a blur and doesn't capture as much of the facial distinctions and detailing of the overall set of portraits.



It does tell us something about the overall set of original portraits (from the standpoint of visualization analysis), mainly that in average the faces do tend to be centered. And for some temporal processing effects that spatial variation of the individual portraits can lead to visually interesting results. But for other processing effects working with the spatially normalized image set can lead to very different results.

Mutation 3

Thought i'd try an experiment with eye detection. The stack of portraits are processed with a PASeq that auto-selects the painted eyes and then paints those into the canvas. So the eyes build up over time.

So this one is a little different than the stack filter effects that use temporal image processing. Building a PASeq that processes a stack of images to build up a single canvas can be thought of as a different approach to stack filtering. We'll get back to time-based processing tomorrow, but it's important to realize that there's many different approaches you can take to build stack filtering effects.

Again, i'm using the original series of 2010 Siggraph portrait virus images for this example. We will be discussing building a source image series spatially warped to a standard image. If we had used that sequence for this particular effect all of the eyes would be in the same position, so it would not be as effective. However, for other kinds of time based processing the standardized sequence can be very useful. We will show an example of that tomorrow.

Mutation 2

Generated with temporal rank neighborhood filtering in Studio Artist 4.

Again, we're working with the original stack of portrait virus images as the input for this effect.

Mutation 1

Generated by processing a stack of portrait virus images with a StudioArtist temporal image processing effect.

This first series is based on the original portrait virus images generated at Siggraph 2010. Later we'll be getting into modified image stacks based on spatially warping the stack of images to a 'standard' image.

What is Portrait Virus Mutations?

Michael Wright has been running his Portrait Virus project for several years now at various Siggraph shows. All of the portrait virus images are created using Studio Artist. The snapshot above shows Michael standing in front of a collection of portrait virus images generated during a previous Siggraph show in the old Guerilla Studio. (There's also a portrait of me buried in there if you look closely). Examples of the Portrait Virus from Siggraph Studio 2010 are available online here.

Michal runs Studio Artist on a portable computer in the Siggraph studio. He uses Studio Artist's video source capture option to grab a video still of someone in the gallery, and then uses Studio Artist's paint synthesizer in real time with a Wacom tablet to generate a painted digital portrait of that person. Michael also has a daily portrait virus post page available here that shows a new Studio Artist generated self-portrait every day. I'm a big believer in this kind of 'daily art post' project because i think the discipline of generating a new art image and posting it every day helps stimulate creativity in many different ways.

I thought it would be fun to try generating my own portrait virus mutation project. All of the source imagery in this initial portrait virus mutation project is generated from collections of Michael's portrait virus imagery. Part of the point of the mutation project is that all of the mutated images are derived from multiple input images. As opposed to processing a single input image. There are a number of different approaches one can use to work with collections of images with the goal of generating single output images from multiple input images. Photo mosaic imagery is one example of this kind of approach. Another approach is what i call 'stack filtering'. 'Stack' refers to a stack of images.

Studio Artist 4 has a number of different temporal image processing operations. Temporal image operations are image processing effects that work off of multiple frame input images at different times to generate a single output image. They are really designed for video processing. But there's no reason why you can't take any stack of images and run them through a temporal processing effect. So stack filtering is referring to taking a stack of images and processing them with a temporal video processing effect.

I'll be posting different portrait virus mutations on this blog, one a day, until the project runs it's course. The initial posts are going to focus on the use of Studio Artist temporal image operations to process stacks of Michael's original portrait virus images, so again a collection of multiple images are processed to generate a single processed or mutated output image. Hopefully this project will give you some ideas for using stack filtering in your own artistic endeavors. The creative possibilities are really endless.

A big thank you to Michael for being kind enough to allow me to mutate his original portrait virus imagery and infect it with a new digital virus. Michael is a long time pioneer in digital art, and you can learn more about his body of work here.