How to Avoid Crappy 3D: Technology Insights from an Industry Insider

(avatar is the highest grossing movie of all time)

(avatar is the highest grossing movie of all time)

I was in the 3D business for a six years. I’ve worked with all the major studios, consumer electronics companies, engineers, producers, directors, DPs, enthusiasts and even a few neurosurgeons.

I’ve gotten a lot of the same questions about 3D from all these groups because there’s a lot of confusion about 3D. The web doesn’t help.

Conflicting descriptions of formats, resolution, brightness, and display methods only make it more confusing, especially when there are multiple elements to each topic.

This post is my attempt at providing a basic introduction to 3D technology while clearing up some long-standing questions, and hopefully it’s helpful to you as a reference. You’re welcome to email me any time with questions. Enjoy.

Stereoscopic (3D) – An Overview

Stereoscopic visualization has been around since before humans, and is used by humans and animals alike. Our paired eyes, positioned parallel to each other, receive two sets of information – right eye and left eye information – which our brain converges into a single image that has perceived depth.

As humans are curious, we sought to figure out a way to recreate what we see and did so through drawings and paintings, eventually utilizing photographic methods. The results are the means by which our society now communicates vast amounts of information: visually, utilizing 2D imagery. Though, no matter how high the resolution, these2D images do not contain the data of a comparable 3D image, but it wasn’t until 1838 when Sir Charles Wheatstone discovered and studied the depth sense referred to as stereopsis. Since then, many stereo-presentation objects, from the parlor stereoscope to the timeless View-Master, have been invented and used for entertaining.  Now, several major organizations, such as NASA expend great effort to visualize in 3D, because of the enhanced ability to use that information for mission critical decisions.

The current trend toward stereoscopic is in some ways a step back in time to a fundamental technology. Images with depth contain more information than 2D images and that information can be used to clearly and adeptly communicate. It is shown that 3D imagery is more memorable and learning occurs much faster when stereoscopic tools are used in place of similar non-stereoscopic tools. Why else would we have evolved with stereoscopic sight if it were not to our advantage?

 

Stereoscopic Methods

It is fairly easy to create a 3D presentation device. Hundreds, if not thousands of methods exist. The idea is simple: we capture a right eye image and a left eye image and present each in a way that allows the right and left to see only respective images. This can be achieved using two separate lenses seeing separate images, as is done with a stereoscope or a View-Master, or using eyewear that block the R eye when the L eye is shown or block the L eye when the R eye is shown.

The methods for ‘filtering’ the proper R/L image into the proper eye are several, all requiring some type of eyewear, but a few are the most common: color separation (we all know the red and blue 3D images, also known as anaglyph), light polarization (similar to color separation but instead of changing the color of the R/L image and using colored lenses to ‘filter’ the images, the state of the light is changed and the eyewear lenses prevent the wrong  light from entering the wrong eye) and physical blocking (eyewear actually ‘black-out’ one or the other eye, physically preventing the image from being seen by the opposite eye).

There are eyewear-free methods such as lenticular, parallax barrier and others, but there are complications that prevent these methods from becoming broadly adopted. For the purposes of this paper, eyewear-free 3D will only be mentioned as a promising but still immature technology not readily available.

The most common method has been color separation (anaglyph), but viewing most red and blue 3D or color separation technologies (excluding color notching), can cause eye strain and headaches due to the color disparity between right and left eyes. Much more impressive are the polarized and physical blocking techniques since they allow full-color images to be seen easily and are currently enabling the majority of 3D displays and projectors available to the consumer and commercial markets. Since anaglyph is essentially limited in its useful application, it will be mentioned minimally.

Polarized technology is also referred to as ‘passive’ and physical blocking technology is referred to as ‘active’. These terms refer to the way the eyewear separate images for the left or right eye. Passive 3D glasses have polarized film as lenses, such as the lightweight eyewear you may receive when you attend a cinema from companies like RealD; the lenses do not physically activate. Active eyewear, also known as ‘shutter glasses’, have lenses that actually change between clear and opaque, physically blocking an image from entering the eye. In other words, active eyewear lenses are electronically powered whereas passive eyewear are not.

(active 3D glasses are going away)

(active 3D glasses are going away)

Active Eyewear – an overview

We can mimic the way active eyewear work by closing our L eye, then closing the R eye while opening the L eye, doing this in succession so you are blinking L,R,L,R, etc. Typical active eyewear use liquid crystal cells, like in a watch, that achieve an opaque (black) state when an electric charge is passed through the cell, similar to a number on an LCD watch face. The better the cell in the eyewear lens, the faster the lens can change from clear to black and the more black the lens will become preventing light reaching the eye, and the more accurate the color of the image when the lens is translucent. The extinction ratio is the degree to which the lens will prevent light from passing through and the contrast ratio is the ratio of the brightest color to the darkest color. Generally, a high extinction ratio and a high contrast ratio within a fast lens switching speed will create an excellent pair of active eyewear. Tinted lenses and poor switching speeds can negatively impact contrast and extinction.

Extinction, contrast, switching speed and lens tint are key factors impacting how effective shutterglasses operate. Generally, poor-performing shutterglasses have lenses that may not become opaque (low extinction), have low transmission of light when open (low contrast) or a low switching speed (slow). As battery power in active eyewear wears out, eyewear performance generally declines.

Shutterglasses will remain more expensive than passive glasses simply due to the electronics within the eyewear. Passive glasses have no electronics or lenses that switch electronically.

(a pair of passive RealD 3D glasses)

(a pair of passive RealD 3D glasses)

Passive Eyewear- an overview

Whereas active eyewear are electronic, passive eyewear are not, and can be made from inexpensive materials such as paper or plastic, with special polarized plastic lenses or lenses coated with polarized film. The lens of passive glasses work like a child’s toy that requires matching the shape of an object with a corresponding hole. The passive lens will only allow light of the same ‘shape’ to pass through the lens and reach the eye, filtering the unwanted light. Light from the opposite eye image is prevented from reaching the incorrect eye and therefore by orienting the light of the R/L eye images in a presentation device (display, projector, etc), the R/L eye images can be seen by the proper eye using polarizing lenses.

Obviously there will be sensitivity within passive lenses: lenses need to be held rigid and properly oriented toward the light source for the best filtering effect. The best passive eyewear will firmly hold the lens in place, flatly in front of the eye with a large viewing window and prevent lens distortion, which causes unwanted light penetration, unless proper technical methods are used to curve the lens surface while retaining polarization filtering capabilities. Passive lenses are measured by contrast and extinction capabilities, the best lens material having high contrast and extinction ratios. The best lens material is more expensive and requires a high degree of quality control for consistent performance. Low-quality lens material does not properly filter the light into the eye and can be a cause of ghosting/poor 3D effect.

Displaying 3D – The 3D Elements

The eyewear is only one piece in the chain of elements required for viewing 3D:

  1. Content – created in 3D (avoiding the topic of real-time 2D-3D conversion for now)
  2. Host Device- A device that hosts the content (Blu-ray® player, computer, cable/satellite, server, etc)
  3. Presentation Device – 3D presentation device, such as a display (TV, computer monitor, etc), projection onto a surface, or a scope
  4. Eyewear – Active or Passive eyewear
  5. Pair of eyes capable of seeing stereoscopic

The numerous elements above all rely on each other like chain links; any problem in any point of the element chain will impact the downstream element. Some examples:

A)     Eyewear problem –, eyewear unable to filter the image into the proper eye will prevent 3D or cause excessive ‘ghosting’, or bleed from right into left eye and vice versa

B)      Presentation problem –, if the presentation device does not work properly, poor stereo 3D will occur, for example if dual projectors are used and the projectors are not aligned properly, or have color or brightness disparity

C)      Content problem – poorly created content can cause physical discomfort such as eyestrain, headaches and nausea. There are five particular traits of stereo content that can cause physical discomfort. The five primary traits of stereo content that will cause physical discomfort include: negative parallax edge violations, R/L eye vertical disparity, R/L eye color disparity, divergence, excessive parallax.Stereo content itself does not cause discomfort unless a viewer has a preexisting condition causing daily visual discomfort.

Hopefully this better explains how important it is to maintain quality in each link of a 3D content presentation, from content to display. One rule is that 3D should never be uncomfortable. If it is, one of the links in the chain is faulty.

Active and Passive Presentation

Now let’s assume that we have content, a host device and eyewear all without any problems. The presentation device can be of any kind (projection or display), but the presentation device must match the eyewear type. Active eyewear will only work with an active presentation device and passive eyewear will only work with a passive presentation device, since it is the state of the light (polarized/non-polarized) or capability of the presentation device that will determine if passive or active eyewear must be used.

An active presentation device will present images in a sequential mode- L,R,L,R,L,R, etc whereby the shutterglasses will open and close the lens so the open lens coordinates with the eye image being shown by the presentation device. In theory this is simple, but in practice it is quite difficult. The active eyewear must match perfectly the ‘frame rate’ of the images being shown. Presentation devices best present 3D content at frame rates above ‘110hz’ or 110 frames per second (55 frames per eye). 120hz (60 frames per eye) is the standard frame rate whereby a frame is presented to the eye for such a short time that the frame switch is unperceivable and therefore avoids the common problem of ‘flicker’ or a jittery image. This requires active lenses to open and close 60 times in each eye in less than 1/60th of a second with a ‘sync signal’ from the presentation device telling the eyewear which lens to open or close at which time. This is done using a variety of sync signals including radio frequencies, infrared (IR) signals or a lesser known light pulse method (used by Texas Instruments). The eyewear receive the signal and respond accordingly, like a TV remote tells a TV to change a channel.

Passive eyewear do not require a ‘sync signal’, but they do require light waves to be oriented properly when each image is presented to the eye. Only the R image should be polarized to match the R eye polarized lens and vice versa. As light is emitted from a display device, the polarization is generally random (except in LCD & LCOS presentation devices but for simplicity sake, we will avoid this for now). A ‘polarization filter’ is placed in front of the light source so that the corresponding image can be polarized and voila, the passive eyewear filters the light and we see 3D. Easy enough if it weren’t for the challenge of how to polarize light.

Light polarization can occur two ways: 1) a static polarized film in front of a light source will permanently polarize the light from the source into a single polarized state (i.e. it sits there and as light passes through it arranges the light into a certain pattern), 2) switching polarizing element in front of a light source that can alternate between one polarizing state and an opposite polarizing state (i.e. electronic signals cause the switching element to polarize the light one direction, then alternate the element quickly to polarize the light the opposite direction).  A static polarized film is used in some of the LCD monitors and projectors available today and switching polarized elements are used in front of DLP® projectors in cinemas. A switching polarizer can be set to switch each time a frame is projected from a projector, so when a L frame is projected the polarization can match the L eyewear lens and vice versa when the R frame is projected. An active switch has the advantage of being coupled with a single projector, and therefore avoids the issues associated with dual-projector setups. Further, a solid-state switching element requires no mechanical movement and is therefore more reliable than mechanically switching polarizers, and has fewer components. A solid state polarizer looks like a tinted window in front of a projector lens while a mechanical switching polarizer will look like a wheel or some other form.

There are two primary forms of polarization: Circular and Linear. The names describe the structure of the polarized light: circular polarization structures light in a clockwise or counterclockwise circular (corkscrew) form while linear polarization structures light in a linear (straight line) structure. The advantage of circular polarization is that viewers have more freedom of head movement – they can tilt their heads to the side – and the eyewear will still filter the image. Linear polarization does not rotate, so viewers cannot tilt their heads beyond about seven degrees. While watching content, viewers must keep their heads straight to avoid linear polarization ghosting.

(a side-by-side 3D format)

(a side-by-side 3D format)

3D Formats

There are a variety of 3D ‘formats’, the ‘package’ in which 3D content is stored, transmitted, or presented. Another way to think of this is looking at a picture: the ‘format’ is the size and orientation of the picture. Since stereoscopic images consist of 2 pictures (R,L), both pictures must be considered when storing the information. There are several ways to store 3D pictures, each called a format. One 3D format can be converted to another format, depending on what the presentation or transmission hardware requires.

The most common format is frame-sequential. This format is like the pages in a book and how an image can be drawn on each of the corners so when the corner is flapped from end to beginning of the book, your hundreds of line drawings will appear to move. A 3D frame-sequential format just has a R eye movie and a L eye movie so when you are watching 3D content you are actually watching 2 movies simultaneously – one designed with a R eye view and one with a L eye view being presented like the corner pages in a book as R,L,R,L, etc. This format is referred to as ‘full resolution’ when the original image is being shown without pixel loss in its original state – you see 100% of the image pixels.

(an over-under 3D format)

(an over-under 3D format)

Other formats include side-by-side, over-under, horizontal-interlace and checkerboard whereby separate L/R frames are combined in a single frame, but each image is ‘squeezed’ or broken up and weaved together so the L and R information is contained in ½ the page. Each of these formats lose pixels and therefore are less than full resolution. Basic side-by-side is just two images horizontally squeezed (compressed) such that the L image is on the L side and R image is on the R side but only ½ the original horizontal size. Over-under is the same but instead of horizontal squeezing, the images are squeezed vertically so the R image is above the L image but each is only ½ the original vertical size. Horizontal-interlace and checkerboard formats are just as they sound: checkerboard is as if all the red squares of a checkerboard were from the L image and the black squares are the R image so ¼ the horizontal and ¼ the vertical resolution is missing. Horizontal-interlace is as if you were to cut thin strips of each image the long way, throw away ½ the strips and arrange the other ½ the strips for the L/R images as L eye strip on the top, followed by a R eye strip, all the way down a display so that every other line is one eye and the opposite line is the other eye, kind of like rows of in a corn field where the corn is the R eye image and the space between the corn is the L eye image.

 

Each format, when played back on a presentation device, can be converted to that presentation device’s required format and presented. If a side by side is converted to frame sequential for presentation, the L ½ is just expanded to take up the whole screen presenting the L eye image (and therefore the image looks normal) and vice versa when the R ½ is shown for the R eye.

Presentation Devices and Content Resolution

Each presentation device displays a 3D image in a different manner, using one kind of format. Many projectors utilize a frame-sequential format, as do most of the 3D LCD TV’s available today. The key to frame sequential format presentation is speed. A frame sequential device must be capable of playing content at a minimum of 120hz (120 frames per second) for a high quality 3D image. This converts to 60hz per eye. However, content going into a frame sequential device can be of any format. If the content source is not full resolution (i.e. side-by-side, over-under, checkerboard or horizontal-interlace), though the content will be shown in frame-sequential, it will still be lower resolution than if the source were full resolution frame-sequential. Further, if a content source is full resolution but it is presented in horizontal-interlace or checkerboard, it will be presented in a resolution lower than the content source.

Rear projection TV’s such as those from Mitsubishi use checkerboard to present 3D content while some models of displays from JVC, Hyundai, LG, Vizio, Toshiba and Sony use horizontal-interlace (though other models of LCD tv present content in frame-sequential such as Samsung Active 3D TVs).

All of the monitors available today have internal conversion capability where the format received is converted to the display format for the TV.

Content received from a 3D Blu-ray player will be frame-sequential, full-resolution (up to 1920x1080p) whereas content received through a set-top-box or cable feed is commonly side-by-side. The reason for the varying formats is that a full-resolution format requires 2 pictures (one per eye) and therefore has a larger amount of data to transmit between source and presentation device than a single-frame format such as side-by-side. Side-by-side is a more efficient way of broadcasting 3D and can use existing hardware infrastructure whereas full-resolution frame-sequential requires hardware with more bandwidth. The new 3D Blu-ray specification is a larger pipeline capable of carrying frame-sequential full-resolution images. Side-by-side and other formats can use existing bandwidth, but at lower resolution. It is possible to master a DVD or existing Blu-ray with SbS content and play it through legacy bandwidth (i.e. HDMI 1.3, analog or other). As long as the presentation device can convert the format to its display format, the host device is irrelevant and could be anything – DVD, Blu-ray, solid state drive, PC, MAC, set-top-box, etc. For the time being, we will not cover the topic of bitrate, etc.

Which format is best? Generally more resolution is preferred, but in the case of broadcast infrastructure, it would not be economically viable to upgrade all the hardware for higher bandwidth to carry frame-sequential 3D, so compressed formats are used. Side-by-side is the current standard adopted by most cable service providers since it has resolution advantages over other formats, though this may change in the future and over-under may be preferred for some types of content.

Viewing Content

Content, format, display device and viewing method are all separate but interrelated. Content can be any number of formats transferred into a presentation device that will convert the content into the display format. The presentation device has to be enabled for either active or passive viewing. Active and passive viewing are essentially independent (I will avoid the grey areas of this topic, since they do exist), and primarily require hardware on the presentation device to enable one or the other. If a presentation device is enabled for active eyewear it generally cannot be seen using passive glasses and vice versa. This is the reason that passive glasses from a cinema will only work on other presentation devices enabled with the same passive technology. So if you take your black ‘Ray-ban’ style glasses home from the theater and look at your tv, they will not work, even if the image is being presented as a 3D image, unless your tv is capable of presenting polarized images, i.e. the presentation device is capable of polarizing the light.

Even more complicated is that each pair of active eyewear and each presentation device has its own sync mechanism or signal and timing for the eyewear. Active eyewear used on one presentation device will generally not work on other devices unless eyewear can sync to the different sync signals. A frame-sequential projector will have a different sync timing and sync signal than an LCD TV or even rear-projection TV. Opponents to active eyewear argue that the lack of compatibility among the various presentation devices limits the ability to use the device. An example is that every person viewing the device would have to have eyewear compatible with the same signal. If a person has purchased a pair of glasses that do not match the presentation device signal, they will not be able to watch the content in 3D. Compounding this issue is that presentation device manufacturers have not agreed on a single standard sync method or signal and therefore will each offer their own eyewear. Active eyewear, being electronic, also are typically more expensive than passive eyewear causing concern about scalability – the ability of large audiences gathering to watch 3D content.

Proponents of passive technology argue that passive eliminates the electronic architecture requirement and required sync signal compatibility therefore enabling cross-compatibility among display devices and eyewear. Theoretically, if all presentation devices were enabled with passive technology in a polarization that matches the majority of movie theaters, one pair of eyewear could be used at all 3D-enabled locations and on passive 3D presentation devices. Additionally, the cost of a basic pair of passive eyewear is much lower than active and do not require a power source, which allow passive eyewear to be much more scalable and reliable than active eyewear.

There are a few reasons why passive systems dominate theaters and active presentation devices dominate consumer electronics. In a theater, one projector can be outfitted with passive polarization technology to enable hundreds of people in an audience to see 3D with inexpensive eyewear. Because each pair of eyewear does not require batteries or electronics, if one person in the audience is seeing 3D, the whole audience is receiving the same signal. Projectionists can quality control a show by simply watching the first few seconds and knowing 3D is working. The cost of each pair of passive eyewear reduces risk to theaters (loss, breakage, failure) and improves reliability for the audience – there is no chance that passive eyewear will ‘die’ in the middle of a screening since the projector is the only electronic piece of equipment. An audience of 200 people, each with active eyewear, are likely to have several failures (batteries, sync, etc) and several stolen/lost/broken for every presentation, and therefore also require ushers to handle audience eyewear failures and monitor outgoing customers to collect eyewear. Generally, passive eyewear can be 100% recycled – if they are broken or scratched, they can be melted and reformed and if they are not broken, they can be sanitized and redistributed. Active eyewear, if broken, cannot be 100% recycled and have dangerous battery waste throughout their life.

Passive polarization projection technology exists today and is the dominant format in cinemas around the world. Consumer electronics companies chose active eyewear as the dominant 3D introductory technology for TV’s because active 3D technology on televisions is much easier and less expensive than passive technology for TV’s and monitors. Active eyewear and matching TV’s allow the CE companies to continue developing reduced-cost passive solutions for TV’s and monitors. Though active eyewear are not universally compatible and can be expensive, several sets of active eyewear and an active 3D TV can still be less expensive for the consumer, but is not scalable.

Commercial 3D technology is more cost effective using passive technology because of the scalability, reliability, recyclability and low cost. When passive 3D presentation devices are broadly available in the consumer market, there may be a day when everyone owns a pair of passive 3D glasses and they may look just like your sunglasses and be used in both cinemas and in your home.

Conclusion

A review of stereoscopic methods, the basic elements of 3D, common formats, interpretation of eyewear technologies, eyewear relationship to presentation devices, and commercial trends is provided as general reference information for achieving a basic understanding of the new 3D ecosystem. It’s no surprise that technological barriers have prevented widespread adoption of stereoscopic media until now. Confusion around format vs. presentation technology and content quality add to the mystery and reluctance of 3D adoption. Antiquated technologies and physical pain associated with poor content and poor content presentation require time for obsolescence and domination of quality. Consistent standards-based 3D technologies, content guidelines and compatibility are driving 3D adoption and improving consumer attitudes. Broad application of 3D technology has only recently begun in the commercial markets  and seems like it is finally here to stay.

Advertisements