Chroma key explained: how software decides what is background

Chroma keying is one of the few pieces of image processing simple enough to understand completely in a few minutes, and understanding it changes how you use it: the sliders stop being trial and error and start being decisions. If you have not set a screen up yet, the green screen guide is the place to start; this is what happens to the picture afterwards.

The basic idea

Every pixel has a colour. You nominate a colour: the backing. For each pixel, measure how far its colour is from the backing colour. Turn that distance into an alpha value, which is the fourth number in each pixel that says how opaque it is. Pixels that are the backing colour get alpha zero, pixels that are nothing like it get alpha 255, and pixels in between get something in between.

That is the whole algorithm. What separates a keyer that works from one that does not is entirely in the phrase “how far its colour is from the backing colour”, and in what happens to the pixels that survive.

The alpha channel covers what that fourth number is and how it is stored, which is worth reading alongside this if the idea of partial transparency is new.

Measuring distance badly

The obvious measurement is straight geometric distance in red, green and blue. Take the backing colour, take the pixel’s colour, treat both as points in a three dimensional cube, measure between them.

It works on the lit part of the screen and fails on everything else, and the failure is always the same. Consider a green backing at red 30, green 200, blue 40. Now consider a shadow across that backing, at red 11, green 70, blue 14. To your eye these are the same colour, one lit and one not. In the cube they are 137 units apart, which is further than green is from grey.

So a naive key removes the lit wall and keeps the shadow, and you get a cutout with a grey lump attached to it. This is far and away the most common complaint about free chroma key tools, and it is not the fabric’s fault or the lighting’s fault. It is the measurement.

Measuring distance well

The fix is to take brightness out of the comparison before measuring.

Split each pixel into a brightness and two colour axes, which is exactly what video has done since colour television: one number for how much light there is, and two for where the colour sits. Then divide the two colour numbers by the brightness. Now a colour is described by its hue and saturation relative to how much light is falling on it, and the dark green shadow and the lit green wall produce nearly identical numbers.

Measure the distance in that space, scale it so that the backing colour itself gives zero and a neutral grey gives one, and you have a number between zero and one that means “how much of the backing colour does this pixel have”. A shadow on the screen gives almost zero, which is correct. Skin gives one. A grey jumper gives one. A dark olive jacket gives about 0.75, comfortably clear of the key.

This one change is the difference between a key you fight with and a key that works on the first try, and it is why the tool on this site gets shadows on the backing right without a second slider for it.

Tolerance and softness

With a distance between zero and one, transparency is a ramp with two thresholds.

Tolerance is where the ramp starts. Everything at or below it becomes fully transparent. Raise it to remove more of the backing, including creases, noise and unevenly lit patches. Raise it too far and it starts eating the parts of the subject closest to the backing colour.

Softness is the width of the ramp above the tolerance. Everything above tolerance plus softness stays fully opaque, and in between the alpha rises smoothly. A wider band gives a gentler edge, which helps with hair and with a noisy image, and eventually makes the whole subject look slightly milky at its border.

The ramp should be eased rather than straight, so there is no visible step where it starts and ends. A straight ramp shows as a faint line on gradients.

Good defaults depend on the picture, which is why the tool here measures the spread of colours along the border of your frame when the file arrives and sets the thresholds from that. An evenly lit studio screen gets a tight key. A creased bedsheet under one lamp gets a looser one. Both are still yours to adjust.

Spill suppression, and the wrong way to do it

A large coloured surface throws coloured light. Every subject in front of a green screen has green on it: along the shoulders, under the chin, and right through loose hair.

The pixels really are greenish. The camera was not wrong. So the question is what to do with them, and there are two answers.

The wrong one is to reduce the saturation of the affected area. That removes the green and also the colour underneath it, so the spill becomes a grey smear. Hair treated this way looks like steel wool.

The right one is to treat spill as a push in one direction in the colour plane. The backing colour defines that direction. For each pixel, work out how far it has been pushed along it, and subtract that push while leaving the brightness exactly as the camera recorded it. The pixel lands where it would have been without the wall. Hair keeps its shape and loses its cast.

An amount control is still worth having, because a subject genuinely wearing olive needs less suppression than a subject in white. Green screen spill covers both preventing it in the room and fixing it afterwards.

Edge adjustments

Two more controls are usually offered, and they are not part of the key itself.

Edge shift, sometimes called choke or shrink, moves the boundary in or out by a pixel or two. Pulling it in by one pixel is a common way to remove a thin remaining fringe. Pushing it out recovers an edge that was keyed too tightly.

Feather blurs the alpha channel slightly, which softens the boundary. It is useful on a noisy image and on a composite where the new background is much softer than the subject. Both should be used sparingly, because both throw away real edge information.

Picking the key colour

Most keyers offer to find the backing colour for you and also let you click it. Both are useful and they fail in different ways.

Finding it automatically works by looking where the backing almost always is: the border of the frame. Take the most common saturated colour in that band and you have the key, and you have it without the user having to understand anything. It breaks when the subject runs off the bottom of the frame, when there is a light stand in shot, or when the screen only covers the middle of the picture.

Clicking to set it is exact and has its own trap: one pixel is noisy. A single pixel on a cheap sensor can be several units away from its neighbours, and a key built on an outlier is a key built on the wrong colour. Averaging a small patch around the click, rather than reading the single pixel, removes the problem entirely and costs nothing.

It is also worth knowing that the key colour does not have to be the exact colour of the screen anywhere in particular. What matters is the direction it points in the colour plane, because that is what both the distance and the spill suppression are built on. Clicking a well lit part of the screen is better than clicking a shadowed part, not because the shadow is the wrong colour, but because it has less colour in it to point with.

Why a key still beats a model on hair

An AI background remover looks at the picture and decides, from what it has learned, which pixels belong to the subject. It has no way of knowing how much of a particular boundary pixel is hair and how much is background, so it estimates.

A key does know, because the backing is visible through the gap. A pixel that is 40 percent hair and 60 percent green screen sits 60 percent of the way toward the backing colour, and that is exactly the alpha the key produces. The information is genuinely in the picture, which is the entire reason green screens still exist in an age of good models. Green screen versus AI background removal compares them properly.

Questions

What does chroma key mean?

Chroma means colour, and key means the signal used to decide transparency. A chroma key is therefore a transparency decision made from colour: pixels close to a chosen colour become transparent and the rest stay.

What do tolerance and softness do?

Tolerance is how close to the backing colour a pixel has to be before it disappears completely. Softness is the width of the band above that, where pixels become partly transparent. Widening tolerance removes more backing and risks eating the subject; widening softness makes the edge gentler and can make it look milky.

Why does my keyer leave the shadow behind?

Because it measures raw colour distance, and a shadow on a green wall is a long way from bright green. A keyer that divides colour by brightness before measuring treats the shadow as the same colour as the lit screen, and removes it with the rest.

What is spill suppression?

The removal of the backing colour's hue from the pixels that stay, which is needed because light bouncing off a large coloured surface lands on the subject. Done well it keeps the brightness the camera recorded and only removes the cast. Done badly it desaturates the area and leaves a grey smear.

Is chroma key the same as background removal?

No. A chroma key uses colour and needs a backing of a known colour. An AI background remover uses a trained model to guess which pixels belong to the subject, and works on pictures that were never planned. They are good at different things.

Can a chroma key produce a soft edge?

Yes, and it must. A pixel on the boundary of a hair is genuinely a mixture of hair and backing, so its correct alpha is somewhere between transparent and opaque. A key that only produced fully transparent or fully opaque pixels would look cut out with scissors.