Digital cinematography

Digital cinematography is the process of capturing motion pictures as digital images, rather than on film. Digital capture may occur on tape, hard disks, flash memory, or other media which can record digital data. As digital technology has improved, this practice has become increasingly common. Several mainstream Hollywood movies have now been shot digitally, and many vendors have brought products to market, including traditional film camera vendors like Arri and Panavision, new vendors like RED and Silicon Imaging, and companies which have traditionally focused on consumer and broadcast video equipment, like Sony and Panasonic. The benefits and drawbacks of digital vs. film acquisition are still hotly debated, but digital cinematography cameras sales have surpassed mechanical cameras in the classic 35mm format.
Technology
Digital cinematography captures motion pictures digitally, in a process analogous to digital photography. While there is no clear technical distinction that separates the images captured in digital cinematography from video, the term "digital cinematography" is usually applied only in cases where digital acquisition is substituted for film acquisition, such as when shooting a feature film. The term is not generally applied when digital acquisition is substituted for analog video acquisition, as with live broadcast television programs.
Sensors
Digital cinematography cameras capture images using CMOS or CCD sensors, usually in one of two arrangements.
High-end cameras designed specifically for the digital cinematography market often use a single sensor (much like digital photo cameras), with dimensions similar in size to a 35mm film frame or even (as with the Vision 65) a 65mm film frame. An image can be projected onto a single large sensor exactly the same way it can be projected onto a film frame, so cameras with this design can be made with PL, PV and similar mounts, in order to use the wide range of existing high-end cinematography lenses available. Their large sensors also let these cameras achieve the same shallow depth of field as 35 or 65mm motion picture film cameras, which is important because many cinematographers consider selective focus an essential visual tool.
Prosumer and broadcast television cameras typically use three 1/3" or 2/3" sensors in conjunction with a prism, with each sensor capturing a different color. Camera vendors like Sony and Panasonic, which have their roots in the broadcast and consumer camera markets, have leveraged their experience with these designs into three-chip products targeted specifically at the digital cinematography market. The Thomson Viper also uses a three-chip design. These designs offer benefits in terms of color reproduction, but are incompatible with traditional cinematography lenses (though new lines of high-end lenses have been developed with these cameras in mind), and incapable of achieving 35mm depth of field unless used with depth-of-field adaptors, which can lower image sharpness and result in a loss of light.
CMOS Sensor
Complementary metal–oxide–semiconductor (CMOS), is a major class of integrated circuits. CMOS technology is used in microprocessors, microcontrollers, static RAM, and other digital logic circuits. CMOS technology is also used for a wide variety of analog circuits such as image sensors, data converters, and highly integrated transceivers for many types of communication. Frank Wanlass got a patent on CMOS in 1967 (US Patent 3,356,858).
CMOS is also sometimes referred to as complementary-symmetry metal–oxide–semiconductor. The words "complementary-symmetry" refer to the fact that the typical digital design style with CMOS uses complementary and symmetrical pairs of p-type and n-type metal oxide semiconductor field effect transistors (MOSFETs) for logic functions.
Two important characteristics of CMOS devices are high noise immunity and low static power consumption. Significant power is only drawn when the transistors in the CMOS device are switching between on and off states. Consequently, CMOS devices do not produce as much waste heat as other forms of logic, for example transistor-transistor logic (TTL) or NMOS logic, which uses all n-channel devices without p-channel devices. CMOS also allows a high density of logic functions on a chip.
The phrase "metal–oxide–semiconductor" is a reference to the physical structure of certain field-effect transistors, having a metal gate electrode placed on top of an oxide insulator, which in turn is on top of a semiconductor material. Instead of metal (usually aluminum in the very old days), current gate electrodes (including those up to the 65 nanometer technology node) are almost always made from a different material, polysilicon, but the terms MOS and CMOS nevertheless continue to be used for the modern descendants of the original process. Metal gates have made a comeback with the advent of high-k dielectric materials in the CMOS process, as announced by IBM and Intel for the 45 nanometer node and beyond.
CCD Sensor or Charge-coupled device
A charge-coupled device (CCD) is an analog shift register, enabling analog signals (electric charges) to be transported through successive stages (capacitors) controlled by a clock signal. Charge coupled devices can be used as a form of memory or for delaying analog, sampled signals. Today, they are most widely used for serializing parallel analog signals, namely in arrays of photoelectric light sensors. This use is so predominant that in common parlance, "CCD" is (erroneously) used as a synonym for a type of image sensor even though, strictly speaking, "CCD" refers solely to the way that the image signal is read out from the chip.
The capacitor perspective is reflective of the history of the development of the CCD and also is indicative of its general mode of operation, with respect to readout, but attempts aimed at optimization of present CCD designs and structures tend towards consideration of the photodiode as the fundamental collecting unit of the CCD. Under the control of an external circuit, each capacitor can transfer its electric charge to one or other of its neighbors. CCDs are used in digital photography and astronomy (particularly in photometry), sensors, electron microscopy, medical fluoroscopy, optical and UV spectroscopy and high speed techniques such as lucky imaging.
Acquisition Formats
While many people make movies with MiniDV camcorders and other consumer and prosumer products that have lower resolutions or shoot interlaced video, cameras marketed as digital cinematography cameras typically shoot in progressive HDTV formats such as 720p and 1080p, or in higher-end formats created specifically for the digital cinematography market, such as 2K and 4K.
To date, 1080p has been the most common format for digitally acquired major motion pictures. In the summer of 2007, director Steven Soderbergh began shooting "The Argentine" and "Guerrilla", due for release in 2008, with prototype Red One 4K camera, making these the first two major motion pictures shot in the 4K format.
Data Storage
Tape vs. Data-Centric
Broadly, there are two paradigms used for data storage in the digital cinematography world.
Many people, particularly those coming from a background in broadcast television, are most comfortable with video tape based workflows. Data is captured to video tape on set. This data is then ingested into a computer running non-linear editing software, using a deck. Once on the computer, the footage is edited, and then output in its final format, possibly to a film recorder for theatrical exhibition, or back to video tape for broadcast use. Original video tapes are kept as an archival medium. The files generated by the non-linear editing application contain the information necessary to retrieve footage from the proper tapes, should the footage stored on the computer's hard disk be lost.
Increasingly, however, digital cinematography is shifting toward "tapeless" workflow, where instead of thinking about digital images as something that exists on a physical medium like video tape, digital video is conceived of as data in files. In tapeless workflow, digital images are usually recorded directly to files on hard disk or flash memory based "digital magazines". At the end of a shooting day (or sometimes even during the day), the digital files contained on these digital magazines are downloaded, typically to a large RAID connected to an editing system. Once data is copied from the digital magazines, they are erased and returned to the set for more shooting. Archiving is accomplished by backing up the digital files from the RAID, using standard practices and equipment for data backup from the Information Technology industry, often to data tape.
Compression
Digital cinema cameras are capable of generating extremely large amounts of data; often hundreds of megabytes per second.To help manage this huge data flow, many cameras or recording devices designed to be used in conjunction with them offer compression. Prosumer cameras typically use high compression ratios in conjunction with chroma subsampling. While this allows footage to be comfortably handled even on fairly modest personal computers, the convenience comes at the expense of image quality.
High-end digital cinematography cameras or recording devices typically support recording at much lower compression ratios, or in uncompressed formats. Additionally, digital cinematography camera vendors are not constrained by the standards of the consumer or broadcast video industries, and often develop proprietary compression technologies that are optimized for use with their specific sensor designs or recording technologies.
Lossless vs. lossy compression
A lossless compression system is capable of reducing the size of digital data in a fully reversible way -- that is, in a way that allows the original data to be completely restored, byte for byte. This is done by removing redundant information from a signal. Digital cinema cameras rarely use only lossless compression methods, because much higher compression ratios (lower data rates) can be achieved with lossy compression. With a lossy compression scheme, information is discarded to create a simpler signal. Due to limitations in human visual perception, it is possible to design algorithms which do this with little visual impact.
Chroma subsampling
Chroma subsampling is the practice of encoding images by implementing more resolution for luma information than for chroma information. It is used in many video encoding schemes—both analog and digital—and also in JPEG encoding.
Rationale
Because of storage and transmission limitations, there is always a desire to reduce (or compress) the signal. Since the human visual system is much more sensitive to variations in brightness than color, a video system can be optimized by devoting more bandwidth to the luma component (usually denoted Y'), than to the color difference components Cb and Cr. The 4:2:2 Y'CbCr scheme for example requires two-thirds the bandwidth of (4:4:4) R'G'B'. This reduction results in almost no visual difference as perceived by the viewer.
How subsampling works
Because the human visual system is less sensitive to the position and motion of color than luminance, bandwidth can be optimized by storing more luminance detail than color detail. At normal viewing distances, there is no perceptible loss incurred by sampling the color detail at a lower rate. In video systems, this is achieved through the use of color difference components. The signal is divided into a luma (Y') component and two color difference components (chroma).
Chroma subsampling deviates from color science in that the luma and chroma components are formed as a weighted sum of gamma-corrected (tristimulus) R'G'B' components instead of linear (tristimulus) RGB components. As a result, luminance and color detail are not completely independent of one another. There is some "bleeding" of luminance and color information between the luma and chroma components. The error is greatest for highly-saturated colors and can be somewhat noticeable in between the magenta and green bars of a color bars test pattern (that has chroma subsampling applied). This engineering approximation (by reversing the order of operations between gamma correction and forming the weighted sum) allows color subsampling to be more easily implemented.
Bitrate
Video and audio compression systems are often characterized by their bitrates. Bitrate describes how much data is required to represent one second of media. One cannot directly use bitrate as a measure of quality, because different compression algorithms perform differently. A more advanced compression algorithm at a lower bitrate may deliver the same quality as a less advanced algorithm at a higher bitrate.
Intra- vs. Inter-frame compression
Most compression systems used for acquisition in the digital cinematography world compress footage one frame at a time, as if a video stream is a series of still images. Inter-frame compression systems can further compress data by examining and eliminating redundancy between frames. This leads to higher compression ratios, but displaying a single frame will usually require the playback system to decompress a number of frames that precede it. In normal playback this is not a problem, as each successive frame is played in order, so the preceding frames have already been decompressed. In editing, however, it is common to jump around to specific frames and to play footage backwards or at different speeds. Because of the need to decompress extra frames in these situations, inter-frame compression can cause performance problems for editing systems. Inter-frame compression is also disadvantageous because the loss of a single frame (say, due to a flaw writing data to a tape) will typically ruin all the frames until the next keyframe occurs. In the case of the HDV format, for instance, this may result in as many as 6 frames being lost with 720p recording, or 15 with 1080i recording
Digital vs. film cinematography
Technical Considerations
Predictability
When shooting on film, response to light is determined by what film stock is chosen. A cinematographer can choose a film stock he or she is familiar with, and expose film on set with a high degree of confidence about how it will turn out. Because the film stock is the main determining factor, results will be substantially similar regardless of what camera model is being used. However, the final result cannot be controlled when shooting with mechanical cameras until the film negative has been processed at a laboratory. Therefore, damage to the film negative, scratches which are generated by faulty camera mechanics etc can not be controlled.
In contrast, when shooting digitally, response to light is determined by the CMOS or CCD sensor(s) in the camera and recorded and "developed" directly. This means a cinematographer can measure and predict exactly how the final image will look by eye if familiar with the specific model of camera being used or able to read a vector/waveform.
On-set monitoring allows the cinematographer to see the actual images that are captured, immediately on the set, which is impossible with film. With a properly calibrated high-definition display, on-set monitoring, in conjunction with data displays such as histograms, waveforms, RGB parades, and various types of focus assist, can give the cinematographer a far more accurate picture of what is being captured than is possible with film. However, all of this equipment may impose costs in terms of time and money, and may not be possible to utilize in difficult shooting situations.
Film cameras do often have a video assist that captures video though the camera to allow for on-set playback, but its usefulness is largely restricted to judging action and framing. Because this video is not derived from the image that is actually captured to film, it is not very useful for judging lighting, and because it is typically only NTSC-resolution, it is often useless for judging focus.
Portability
Ultra-lightweight and extremely compact digital cinematography cameras, as the SI:2K mini, are much smaller and lighter than mechanical film cameras. Other High-end digital cinema cameras can be quite large, and some models require bulky external recording mechanisms (though in some cases only a small strand of optical fiber is necessary to connect the camera and the recording mechanism).
Compact 35mm film cameras that produce the full 35mm film resolution and accept standard 35mm lenses cannot be sized down below a certain size and weight, as they require at least space for the film negative and basic mechanics.
Smaller form-factor digital cinema cameras such as the Red One and SI-2K have made digital more competitive in this respect. The SI-2K, in particular, with its detachable camera head, allows for high-quality images to be captured by a camera/lens package that is far smaller than is practically achievable with a 35mm film camera and is used in many scenarios to replace film - especially for stereoscopic productions.
Dynamic Range
The sensors in most high-end digital video cameras have less exposure latitude (dynamic range) than modern motion picture film stocks. In particular, they tend to 'blow out' highlights, losing detail in very bright parts of the image. If highlight detail is lost, it is impossible to recapture in post-production. Cinematographers can learn how to adjust for this type of response using techniques similar to those used when shooting on reversal film, which has a similar lack of latitude in the highlights. They can also use on-set monitoring and image analysis to ensure proper exposure. In some cases it may be necessary to 'flatten' a shot, or reduce the total contrast that appears in the shot, which may require more lighting to be used.
Many people also believe that highlights are less visually pleasing with digital acquisition, because digital sensors tend to 'clip' them very sharply, whereas film produces a 'softer' roll-off effect with over-bright regions of the image. Some more recent digital cinema cameras attempt to more closely emulate the way film handles highlights, though how well they achieve this is a matter of some dispute. A few cinematographers have started deliberately using the 'harsh' look of digital highlights for aesthetic purposes. One notable example of such use is Battlestar Galactica.
Digital acquisition typically offers better performance than film in low-light conditions, allowing less lighting and in some cases completely natural or practical lighting to be used for shooting, even indoors. This low-light sensitivity also tends to bring out shadow detail. Some directors have tried a "best for the job" approach, using digital acquisition for indoor or night shoots, and traditional film for daylight exteriors.
Resolution
Substantive debate over the subject of film resolution vs. digital image resolution is clouded by the fact that it is difficult to meaningfully and objectively determine the resolution of either.
Unlike a digital sensor, a film frame does not have a regular grid of discrete pixels. Rather, it has an irregular pattern of differently sized grains. As a film frame is scanned at higher and higher resolutions, image detail is increasingly masked by grain, but it is difficult to determine at what point there is no more useful detail to extract. Moreover, different film stocks have widely varying ability to resolve detail.
Determining resolution in digital acquisition seems straightforward, but is significantly complicated by the way digital camera sensors work in the real world. This is particularly true in the case of high-end digital cinematography cameras that use a single large bayer pattern CMOS sensor. A bayer pattern sensor does not sample full RGB data at every point; each pixel is biased toward red, green or blue, and a full color image is assembled from this checkerboard of color by processing the image through a demosaicing algorithm. Generally with a bayer pattern sensor, actual resolution will fall somewhere between the "native" value and half this figure, with different demosaicing algorithms producing different results. Additionally, most digital cameras (both bayer and three-chip designs) employ optical low-pass filters to avoid aliasing. Such filters reduce resolution.
In general, it is widely accepted that film exceeds the resolution of HDTV formats and the 2K digital cinema format, but there is still significant debate about whether 4K digital acquisition can match the results achieved by scanning 35mm film at 4K, as well as whether 4K scanning actually extracts all the useful detail from 35mm film in the first place. However, as of 2007 the majority of films that use a digital intermediate are done at 2K because of the costs associated with working at higher resolutions. Additionally, 2K projection is chosen for almost all permanent digital cinema installations, often even when 4K projection is available.
One important thing to note is that the process of optical duplication, used to produce theatrical release prints for movies that originate both on film and digitally, causes significant loss of resolution. If a 35mm negative does capture more detail than 4K digital acquisition, ironically this may only be visible when a 35mm movie is scanned and projected on a 4K digital projector.
Grain & Noise
Film has a characteristic grain structure, which many people view positively, either for aesthetic reasons or because it has become associated with the look of 'real' movies. Different film stocks have different grain, and cinematographers may use this for artistic effect.
Digitally acquired footage lacks this grain structure. Electronic noise is sometimes visible in digitally acquired footage, particularly in dark areas of an image or when footage was shot in low lighting conditions and gain was used. Some people believe such noise is a workable aesthetic substitute for film grain, while others believe it has a harsher look that detracts from the image.
Well shot, well lit images from high-end digital cinematography cameras can look almost eerily clean. Some people believe this makes them look "plasticky" or computer generated, while others find it to be an interesting new look, and argue that film grain can be emulated in post-production if desired.
Since most theatrical exhibition still occurs via film prints, the super-clean look of digital acquisition is often lost before moviegoers get to see it, because of the grain in the film stock of the release print.
Digital Intermediate Workflow
The process of using digital intermediate workflow, where movies are color graded digitally instead of via traditional photochemical finishing techniques, has become common, largely because of the greater artistic control it provides to filmmakers. In 2007, all of the 10 most successful movies released used the digital intermediate process.
In order to utilize digital intermediate workflow with film, the camera negative must be processed and then scanned. High quality film scanning is time consuming and expensive. With digital acquisition, this step can be skipped, and footage can go directly into a digital intermediate pipeline as digital data.
Some filmmakers have years of experience achieving their artistic vision using the techniques available in a traditional photochemical workflow, and prefer that finishing process. While it would be theoretically possible to use such a process with digital acquisition by creating a film negative on a film recorder, in general digital acquisition is not a suitable choice if a traditional finishing process is desired.
Sound
Films are traditionally shot with dual-system recording, where picture is recorded on camera, and sync sound is recorded to a separate sound recording device. In post-production, picture and sound are synced up.
Many cameras used for digital cinematography can record sound internally, already in sync with picture. This eliminates the need for syncing in post, which can lead to faster workflows. However, most sound recording is done by specialist operators, and the sound will likely be separated and further processed in post-production anyway. Also, recording sound to the camera may require running additional cables to the camera, which may be problematic in some shooting situations, particularly in shots where the camera is moving. Wireless transmission systems can eliminate this problem, but are not suitable for use in all circumstances.
Archiving
Many people feel there is significant value in having a film negative master for archival purposes. As long as the negative does not physically degrade, it will be possible to recover the image from it in the future, regardless of changes in technology. In contrast, even if digital data is stored on a medium that will preserve its integrity, changes in technology may render the format unreadable or expensive to recover over time. For this reason, film studios distributing digitally-originated films often make film-based separation masters of them for archival purposes.
Economics
Low-budget / Independent Filmmaking
For the last 25 years, many respected filmmakers like George Lucas have predicted that electronic or digital cinematography would bring about a revolution in filmmaking, by dramatically lowering costs.
For low-budget and so-called "no-budget" productions, digital cinematography on prosumer cameras clearly has cost benefits over shooting on 35mm or even 16mm film. The cost of film stock, processing, telecine, negative cutting, and titling for a feature film can run to tens of thousands of dollars according to From Reel to Deal, a book on independent feature film production by Dov S-S Simens. Costs directly attributable to shooting a low-budget feature on 35mm film could be $50,000 on the low side, and over twice that on the high side. In contrast, obtaining a high-definition prosumer camera and sufficient tape stock to shoot a feature can easily be done for under $10,000, or significantly less if, as is typically the case with 35mm shoots, the camera is rented.
If a 35mm print of the film is required, an April 2003 article in American Cinematographer found the costs between shooting film and video are roughly the same. The benefit to shooting video is that the cost of a film-out is only necessary should the film find a distributor to pick up the cost. When shooting film, the costs are upfront and cannot be deferred in such a manner. On the other hand, the same article found 16mm film to deliver better image quality in terms of resolution and dynamic range. Given the progress digital acquisition, film recording, and related technologies have seen in the last few years, it is unclear how relevant this article is today.
Most extremely low-budget movies never receive wide distribution, so the impact of low-budget video acquisition on the industry remains to be seen. It is possible that as a result of new distribution methods and the long tail effects they may bring into play, more such productions may find profitable distribution in the future. Traditional distributors may also begin to acquire more low-budget movies as better affordable digital acquisition eliminates the liability of low picture quality, and as they look for a means to escape the increasingly drastic "boom and bust" financial situation created by spending huge amounts of money on a relatively small number of very large movies, not all of which succeed.
Hollywood
On higher budget productions, the cost advantages of digital cinematography are not as significant, primarily because the costs imposed by working with film are simply not major expenses for such productions. Two recent films, Sin City and Superman Returns, both shot on digital tape, had budgets of $40 million and close to $200 million respectively. The cost savings, though probably in the range of several hundred thousand to over a million dollars, were negligible as a percentage of the total production budgets in these cases.
Rick McCallum, a producer on Attack of the Clones, has commented that the production spent $16,000 for 220 hours of digital tape, where a comparable amount of film would have cost $1.8 million. However, this does not necessarily indicate the actual cost savings. The low incremental cost of shooting additional footage may encourage filmmakers to use far higher shooting ratios with digital. The lower shooting ratios typical with film may save time in editing, lowering post-production costs somewhat.
Shooting in digital requires a digital intermediate workflow, which is more expensive than a photochemical finish. However, a digital intermediate may be desirable even with film acquisition because of the creative possibilities it provides, or a film may have a large number of effects shots which would require digital processing anyway. Digital intermediate workflow is coming down in price, and is quickly becoming standard procedure for high-budget Hollywood movies.
Digital cinematography cameras
Professional cameras include the Sony HDCAM Series, RED One, Panavisions Genesis, SI-2K, Thomson Viper, Vision Research Phantom, Weisscam, GS Vitec noX, and the Fusion Camera System. Independent filmmakers have also pressed low-cost consumer and prosumer cameras into service for digital filmmaking.
Industry acceptance of digital cinematography
For over a century, virtually all movies have been shot on film and nearly every film student learns about how to handle 16mm and 35mm film.
Today, digital acquisition accounts for the vast majority of moving image acquisition, as most content for broadcast is shot on digital formats. Most movies destined for theatrical release are still shot on film, however, as are many dramatic TV series and some high-budget commercials. High-end digital cinematography cameras suitable for acquiring footage intended for theatrical release are on the market since 1999/2000, and have meanwhile gained widespread adoption.