Matte (filmmaking)

For the technique of creating backgrounds, see matte painting. For other uses, see Matte (disambiguation).

Mattes are used in photography and special effects filmmaking to combine two or more image elements into a single, final image. Usually, mattes are used to combine a foreground image (such as actors on a set, or a spaceship) with a background image (a scenic vista, a field of stars and planets). In this case, the matte is the background painting. In film and stage, mattes can be physically huge sections of painted canvas, portraying large scenic expanses of landscapes.

In film, the principle of a matte requires masking certain areas of the film emulsion to selectively control which areas are exposed. However, many complex special-effects scenes have included dozens of discrete image elements, requiring very complex use of mattes, and layering mattes on top of one another. For an example of a simple matte, we may wish to depict a group of actors in front of a store, with a massive city and sky visible above the store's roof. We would have two images—the actors on the set, and the image of the city—to combine onto a third. This would require two masks/mattes. One would mask everything above the store's roof, and the other would mask everything below it. By using these masks/mattes when copying these images onto the third, we can combine the images without creating ghostly double-exposures. In film, this is an example of a static matte, where the shape of the mask does not change from frame to frame. Other shots may require mattes that change, to mask the shapes of moving objects, such as human beings or spaceships. These are known as traveling mattes. Traveling mattes enable greater freedom of composition and movement, but they are also more difficult to accomplish. Compositing techniques that remove all areas of a certain color from a recording - colloquially known as "bluescreen" or "greenscreen" after the most popular colors used - are probably the best-known and most widely used modern techniques for creating traveling mattes, although rotoscoping and multiple motion control passes have also been used in the past. Computer-generated imagery, either static or animated, is also often rendered with a transparent background and digitally overlaid on top of modern film recordings using the same principle as a matte - a digital image mask.

History

Mattes are a very old technique, going back to the Lumière brothers. Originally, the matte shot was created by filmmakers obscuring their backgrounds with cut-out cards. When the live action portion of a scene was filmed, the background portion of the film wasn’t exposed. Once the live action was filmed, a different cut-out would be placed over the live action. The film would be rewound, and the filmmakers would film their new background. This technique was known as the in-camera matte and was considered more a novelty than a serious special effect during the late 1880s.[1] A good early American example is seen in The Great Train Robbery (1903) where it is used to place a train outside a window in a ticket office, and later a moving background outside a baggage car on a train 'set'. Around this time, another technique known as the glass shot was also being used. The glass shot was made by painting details on a piece of glass which was then combined with live action footage to create the appearance of elaborate sets. The first glass shots are credited to Edgar Rogers.[1]

The first major development of the matte shot was the early 1900s by Norman Dawn ASC. Dawn had seamlessly woven glass shots into many of his films: such as the crumbling California Missions in the movie Missions of California,[2] and used the glass shot to revolutionize the in-camera matte. Now, instead of taking their live action footage to a real location, filmmakers would shoot the live action as before with the cut-out cards in place, then rewind the film and transfer it to a camera designed to minimize vibrations. Then the filmmakers would shoot a glass shot instead of a live action background. The resulting composite was of fairly high quality, since the matte line – the place of transition from the live action to the painted background – was much less jumpy. In addition, the new in-camera matte was much more cost effective, as the glass didn’t have to be ready the day the live action was shot. One downside to this method was that since the film was exposed twice, there was always the risk of accidentally overexposing the film and ruining the footage filmed earlier.

The in-camera matte shot remained in use until the film stock began to go up in quality in the 1920s. During this time a new technique known as the bi-pack camera method was developed. This was similar to the in-camera matte shot, but relied on one master positive as a backup. This way if anything was lost, the master would still be intact. Around 1925 another method of making a matte was developed. One of the drawbacks of the old mattes was that the matte line was stationary. There could be no direct contact between the live action and the matte background. The traveling matte changed that. The traveling matte was like an in-camera or bi-pack matte, except that the matte line changed every frame. Filmmakers could use a technique similar to the bi-pack method to make the live action portion a matte itself, allowing them to move the actors around the background and scene – integrating them completely. The Thief of Bagdad (1940) represented a major leap forward for the traveling matte and the first major introduction of the bluescreen technique invented by Larry Butler when it won the Academy Award for Best Visual Effects that year, though the process was still very time-intensive, and each frame had to be hand-processed.

Computers began to aid the process late in the 20th century. In the 1960s, Petro Vlahos refined the use of motion control cameras in bluescreen and received an Academy Award for the process. The 1980s saw the invention of the first digital mattes and bluescreening processes, as well as the invention of the first computerized non-linear editing systems for video. Alpha compositing, in which digital images could be made partially transparent in the same way an animation cel is in its natural state, had been invented in the late 1970s and was integrated with the bluescreen process in the 1980s. Digital planning began for The Empire Strikes Back in 1980, for which Richard Edlund received the Academy Award for his work to create an aerial-image optical printer for combining mattes, though this process was still analog. The first fully digital matte shot was created by painter Chris Evans in 1985 for Young Sherlock Holmes for a scene featuring a computer-graphics (CG) animation of a knight leaping from a stained-glass window. Evans first painted the window in acrylics, then scanned the painting into LucasFilm’s Pixar system for further digital manipulation. The computer animation blended perfectly with the digital matte, something a traditional matte painting could not have accomplished.[3] In the modern era, nearly all modern mattes are now done via digital video editing and the compositing technique - a digital generalization of the bluescreen - is now possible even on home computers.

Technique

In-camera matte shot

The in-camera matte shot, also known as the Dawn Process[4] is created by first mounting a piece of glass in front of the camera. Black paint is applied to the glass where the background will be replaced. The actors are then filmed with minimal sets. The director shoots several minutes of extra footage to be used as test strips. The matte painter then develops a test strip (with the blacked out areas in the shot) and projects a frame of the 'Matted' shot onto the easel mounted glass. This test footage clip is used as the reference to paint the background or scenery to be matted in on a new piece of glass. The live action part of the glass is painted black, more of the test footage is then exposed to adjust and confirm color matching and edge line up. Then the critical parts of the matted live action scene (with the desired actions and actors in place) are threaded up for burning the painted elements into the black areas. The flat black paint put on the glass blocks light from the part of the film it covers, preventing double exposure over the latent live action scenes from occurring.

Bi-pack process

To begin a bipack matte filming, the live action portion is shot. The film is loaded and projected onto a piece of glass that has been painted first black, then white.[5] The matte artist decides where the matte line will be and traces it on the glass, then paints in the background or scenery to be added. Once the painting is finished the matte artist scrapes away the paint on the live action portions of the glass.[5] The original footage and a clean reel are loaded into the bi-pack with the original threaded so it passes the shutter in front of the clean film. The glass is lit from behind, so that when the reels are both run, only the live action is transferred to the clean film. The reel of original footage is then removed and a piece of black cloth is placed behind the glass. The glass is lit from the front and the new reel is rewound and run again. The black cloth prevents the already exposed footage from being exposed a second time; the background scenery has been added to the live action.

Digitally extracting mattes from a single image

Often, it is desirable to extract two or more mattes from a single image. This process, dubbed "matting" or "pulling a matte," is most commonly used to separate the foreground and background elements of an image, and these images are often individual frames of a video file. Compositing techniques are a relatively simple way of pulling a matte - the foreground from a greensceen scene could be imposed on an arbitrary background scene, for instance. Attempting to matte an image that doesn't use this technique is significantly more difficult. Several algorithms have been designed in an effort to address this challenge.[6]

Ideally, this matting algorithm would separate an input video stream Irgb into three output streams: a full-color, foreground-only stream αFrgb with a pre-multiplied alpha (alpha compositing)), a full-color background stream Brgb, and a single-channel stream of partial coverage a of the pixels in the foreground stream. This ideal algorithm can take any arbitrary video as input, including video where the foreground and background are dynamic, there are multiple depths in the background, there exist overlaps between background and foreground share the same color and no texture, and other various features that such algorithms traditionally have some difficulty in dealing with. Unfortunately, achieving this algorithm is impossible due to the loss of information that occurs when translating a real-world scene into a two-dimensional video.[6] Smith and Blinn formally proved this in 1996.[7]

Matting also has some other fundamental limitations. The process cannot reconstruct parts of the background that are occluded by the foreground, and any sort of approximation will be limited. Additionally, the foreground and background of an image still have an effect on each other due to shadows being cast and light being reflected between them. When compositing an image or video from mattes of different origin, missing or extra shadows and other details of light can ruin the impact of the new image.

The process of matting itself is a difficult process to solve. It has been under research since the 1950's, and yet its most popular use - filmmaking - resorts to the classic but constrained compositing method. Another digital matting approach was proposed by McGuire et al. It makes use of two imaging sensors along the same optical axis, and uses data from both of them. (There are various ways to achieve this, such as using a beam-splitter or per-pixel polarization fiters.) The system simultaneously captures two frames that differ by about half the dynamic range at background pixels but are identical at foreground pixels. Using the differences between the backgrounds of the two images, McGuire et al. are able to extract a high-resolution foreground matte from the scene. This method still retains some of the shortcomings of compositing techniques - namely, the background must be relatively neutral and uniform - but it introduces several benefits, such as precise sub-pixel results, better support for natural illumination, and allowing the foreground to be the color that a compositing technique would identify as part of the background matte. However, this means that intentionally masking something in the foreground by coating it in the same color as the background is impossible.[8]

A third approach to digital matting is using three video streams with different focusing distances and depths of field. As with the previous method, all three image sensors share a common optical axis, though now the algorithm uses information about what part of the image is in focus in which video feed to generate its foreground matte. With this technique, both the foreground and background can have dynamic content, and there are no restrictions on what colors or complexity the background has.[9]

All of these approaches share one notable weakness: they cannot take arbitrary videos as inputs. In video, distinct from film, Chroma key requires the background of the original video to be a single color. The other two techniques require more information in the form of synchronized but slightly different videos.[10]

Mattes and widescreen filming

Another use of mattes in filmmaking is to create a widescreen effect. In this process, the top and bottom of a standard frame are matted out, or masked, with black bars, i.e. the film print has a thick frame line. Then the frame within the full frame is enlarged to fill a screen when projected in a theater.

Thus, in "masked widescreen" an image with an aspect ratio of 1.85:1 is created by using a standard, 1.37:1 frame and matting out the top and bottom. If the image is matted during the filming process it is called a hard matte due to its sharp edge. In contrast, if the full frame is filled during filming and the projectionist is relied upon to matte out the top and bottom in the theater, it is referred to as a soft matte, as the aperture plate is not on the focal plane and causes a soft edge.

In video, a similar effect is often used to present widescreen films on a conventional, 1.33:1 television screen. In this case, the process is called letterboxing. However, in letterboxing, the top and bottom of the actual image are not matted out. The picture is "pushed" farther back on screen and thus made "smaller", so to speak, so that, in a widescreen film, the viewer can see, on the left and right of the picture, what would normally be omitted if the film were shown fullscreen on television, achieving a sort of "widescreen" effect on a square TV screen. In letterboxing, the top of the image is slightly lower than usual, the bottom is higher, and the unused portion of the screen is covered by black bars. For video transfers, transferring a "soft matte" film to a home video format with the full frame exposed, thus removing the mattes at the top and bottom, is referred to as an "open matte transfer." In contrast, transferring a "soft matte" film to a home video format with the theatrical mattes intact is referred to as a "closed matte transfer."

Garbage matte

A "garbage matte" is often hand-drawn, sometimes quickly made, used to exclude parts of an image that another process, such as bluescreen, would not remove. The name stems from the fact that the matte removes "garbage" from the procedurally produced image. "Garbage" might include a rig holding a model, or the lighting grid above the top edge of the bluescreen.

Mattes can also force inclusion of parts of the image that might otherwise have been removed by the keyer, such as too much blue reflecting on a shiny model ("blue spill"), but in this case, technically, they should not be called "garbage" mattes (though they can be created using the same tool).

See also

References

  1. 1 2 The Saga of Special Effects, Fry and Fourzon, pp. 22-23
  2. ↑ The Invisible Art: The Legends of Movie Matte Painting by Mark Cotta Vaz and Craig Barron, Chronicle Books, 2002; p. 33
  3. ↑ The Invisible Art, Cotta Vaz/Barron, pp. 213, 217
  4. ↑ (3)http://entertainment.howstuffworks.com/digital-matte-painter1.htm
  5. 1 2 How Movies Work – Bruce F. Kawin pg 424-425
  6. 1 2 McGuire, M. 2005.Computational Videography with a Single Axis, Multi-Parameter Lens Camera.
  7. ↑ Alvy Ray Smith and James F. Blinn. Blue screen matting. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pages 259–268. ACM Press, 1996.
  8. ↑ McGuire, Matusik, and Yerazunis. Practical, Real-Time Studio Matting using Dual Imagers. EGSR 2006, July, 2006.
  9. ↑ McGuire, Matusik, Pfister, Hughes, and Durand. Defocus Video Matting. ACM Transactions on Graphics (SIGGRAPH 2005), August 2, 2005.
  10. ↑ https://www.smpte.org/top-standards

Books

This article is issued from Wikipedia - version of the 11/17/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.