Tag Archive for Google Glass

AR/MR Optics for Combining Light for a See-Through Display (Part 1)

combiners-sample-cropIn general, people find the combining of an image with the real world somewhat magical; we see this with heads up displays (HUDs) as well as Augmented/Mixed Reality (AR/MR) headsets.   Unlike Starwars R2D2 projection into thin air which was pure movie magic (i.e. fake/impossible), light rays need something to bounce off to redirect them into a person’s eye from the image source.  We call this optical device that combines the computer image with the real world a “combiner.”

In effect, a combiner works like a partial mirror.  It reflects or redirects the display light to the eye while letting light through from the real world.  This is not, repeat not, a hologram which it is being mistakenly called by several companies today.  Over 99% people think or call “holograms” today are not, but rather simple optical combining (also known as the Pepper’s Ghost effect).

I’m only going to cover a few of the more popular/newer/more-interesting combiner examples.  For a more complete and more technical survey, I would highly recommend a presentation by Kessler Optics. My goal here is not to make anyone an optics expert but rather to gain insight into what companies are doing why.

With headsets, the display device(s) is too near for the human eye to focus and there are other issues such as making a big enough “pupil/eyebox” so the alignment of the display to the eye is not overly critical. With one exception (the Meta 2) there are separate optics  that move apparent focus point out (usually they try to put it in a person’s “far” vision as this is more comfortable when mixing with the real word”.  In the case of Magic Leap, they appear to be taking the focus issue to a new level with “light fields” that I plan to discuss the next article.

With combiners there is both the effect you want, i.e. redirecting the computer image into the person’s eye, with the potentially undesirable effects the combiner will cause in seeing through it to the real world.  A partial list of the issues includes:

  1. Dimming
  2. Distortion
  3. Double/ghost images
  4. Diffraction effects of color separation and blurring
  5. Seeing the edge of the combiner

In addition to the optical issues, the combiner adds weight, cost, and size.  Then there are aesthetic issues, particularly how they make the user’s eye look/or if they affect how others see the user’s eyes; humans are very sensitive to how other people’s eye look (see the EPSON BT-300 below as an example).

FOV and Combiner Size

There is a lot of desire to support a wide Field Of View (FOV) and for combiners a wide FOV means the combiner has to be big.  The wider the FOV and the farther the combiner is from the eye the bigger the combiner has to get (there is not way around this fact, it is a matter of physics).   One way companies “cheat” is to not support a person wearing their glasses at all (like Google Glass did).

The simple (not taking everything into effect) equation (in excel) to computer the minimum width of a combiner is =2*TAN(RADIANS(A1/2))*B1 where A1 is the FOV in degrees and and B1 is the distance to farthest part combiner.  Glasses are typically about 0.6 to 0.8 inches from the eye and the size of the glasses and the frames you want about 1.2 inches or more of eye relief. For a 40 degree wide FOV at 1.2 inches this translates to 0.9″, at 60 degrees 1.4″ and for 100 degrees it is 2.9″ which starts becoming impractical (typical lenses on glasses are about 2″ wide).

For, very wide FOV displays (over 100 degree), the combiner has to be so near your eye that supporting glasses becomes impossible. The formula above will let your try your own assumptions.

Popular/Recent Combiner Types (Part 1)

Below, I am going to go through the most common beam combiner options.  I’m going to start with the simpler/older combiner technologies and work my way to the “waveguide” beam splitters of some of the newest designs in Part 2.  I’m going to try and hit on the main types, but there are many big and small variations within a type

gg-combinerSolid Beam Splitter (Google Glass and Epson BT-300)

These are often used with a polarizing beam splitter polarized when using LCOS microdisplays, but they can also be simple mirrors.  They generally are small due to weight and cost issues such as with the Google Glass at left.  Due to their small size, the user will see the blurry edges of the beam splitter in their field of view which is considered highly undesirable.  bt-300Also as seen in the Epson BT-300 picture (at right), they can make a person’s eyes look strange.  As seen with both the Google Glass and Epson, they have been used with the projector engine(s) on the sides.

Google glass has only about a 13 degree FOV (and did not support using a person’s glasses) and about 1.21 arc-minutes/pixel angular resolution with is on the small end compared to most other headset displays.    The BT-300 about 23 degree (and has enough eye relief to supports most glasses) horizontally and has dual 1280×720 pixels per eye giving it a 1.1 arc-minutes/pixel angular resolution.  Clearly these are on the low end of what people are expecting in terms of FOV and the solid beam quickly becomes too large, heavy, and expensive at the FOV grows.  Interesting they are both are on the small end of their apparent pixel size.

meta-2-combiner-02bSpherical/Semi-Spherical Large Combiner (Meta 2)

While most of the AR/MR companies today are trying to make flatter combiners to support a wide FOV with small microdisplays for each eye, Meta has gone in the opposite direction with dual very large semi-spherical combiners with a single OLED flat panel to support an “almost 90 degree FOV”. Note in the picture of the Meta 2 device that there are essentially two hemispheres integrated together with a single large OLED flat panel above.

Meta 2 uses a 2560 by 1440 pixel display that is split between two eyes.  Allowing for some overlap there will be about 1200 pixel per eye to cover 90 degrees FOV resulting in a rather chunkylarge (similar to Oculus Rift) 4.5 arc-minutes/pixel which I find somewhat poor (a high resolution display would be closer to 1 a-m/pixel).

navdy-unitThe effect of the dual spherical combiners is to act as a magnifying mirror that also move the focus point out in space so the use can focus. The amount of magnification and the apparent focus point is a function of A) the distance from the display to the combiner, B) the distance from the eye to the combiner, and C) the curvature.   I’m pretty familiar with this optical arrangement since the optical design it did at Navdy had  similarly curved combiner, but because the distance from the display to the combiner and the eye to the combiner were so much more, the curvature was less (larger radius).

I wonder if their very low angular resolution was as a result of their design choice of the the large spherical combiner and the OLED display’s available that they could use.   To get the “focus” correct they would need a smaller (more curved) radius for the combiner which also increases the magnification and thus the big chunky pixels.  In theory they could swap out the display for something with higher resolution but it would take over doubling the horizontal resolution to have a decent angular resolution.

I would also be curious how well this large of a plastic combiner will keep its shape over time. It is a coated mirror and thus any minor perturbations are double.  Additionally and strain in the plastic (and there is always stress/strain in plasic) will cause polarization effect issues, say whenlink-ahmd viewing and LCD monitor through it.   It is interesting because it is so different, although the basic idea has been around for a number of years such as by a company called Link (see picture on the right).

Overall, Meta is bucking the trend toward smaller and lighter, and I find their angular resolution disappointing The image quality based on some on-line see-through videos (see for example this video) is reasonably good but you really can’t tell angular resolution from the video clips I have seen.  I do give them big props for showing REAL/TRUE video’s through they optics.

It should be noted that their system at $949 for a development kit is about 1/3 that of Hololens and the ODG R-7 with only 720p per eye but higher than the BT-300 at $750.   So at least on a relative basis, they look to be much more cost effective, if quite a bit larger.

odg-002-cropTilted Thin Flat or Slightly Curved (ODG)

With a wide FOV tilted combiner, the microdisplay and optics are locate above in a “brow” with the plate tilted (about 45 degrees) as shown at left on an Osterhout Design Group (ODG) model R-7 with 1280 by 720 pixel microdisplays per eye.   The R-7 has about a 37 degree FOV and a comparatively OK 1.7 arc-minutes/pixel angular resolution.

odg-rr-7-eyesTilted Plate combiners have the advantage of being the simplest and least expensive way to provide a large field of view while being relatively light weight.

The biggest drawback of the plate combiner is that it takes up a lot of volume/distance in front of the eye since the plate is tilted at about 45 degrees from front to back.  As the FOV gets bigger the volume/distance required also increase.
odg-horizons-50d-fovODG is now talking about a  next model called “Horizon” (early picture at left). Note in the picture at left how the Combiner (see red dots) has become much larger. They claim to have >50 degree FOV and with a 1920 x 1080 display per eyethis works out to an angular resolution of about 1.6 arc-minutes/pixel which is comparitively good.

Their combiner is bigger than absolutely necessary for the ~50 degree FOV.  Likely this is to get the edges of the combiner farther into a person’s peripheral vision to make them less noticeable.

The combiner is still tilted but it looks like it may have some curvature to it which will tend to act as a last stage of magnification and move the focus point out a bit.   The combiner in this picture is also darker than the one in the older R-7 combiner and may have additional coatings on it.

ODG has many years of experience and has done many different designs (for example, see this presentation on Linked-In).  They certainly know about the various forms of flat optical waveguides such as Microsoft’s Hololens is using that I am going to be talking about next time.  In fact,  that Microsoft’s licensed Patent from ODG for  about $150M US — see).

Today, flat or slightly curved thin combiners like ODG is using probably the best all around technology today in terms of size, weight, cost, and perhaps most importantly image quality.   Plate combiners don’t require the optical “gymnastics” and the level of technology and precision that the flat waveguides require.

Next time — High Tech Flat Waveguides

Flat waveguides using diffraction (DOE) and/or holographic optical elements (HOE) are what many think will be the future of combiners.  They certainly are the most technically sophisticated. They promise to make the optics thinner and lighter but the question is whether they have the optical quality and yield/cost to compete yet with simpler methods like what ODG is using on the R-7 and Horizon.

Microsoft and Magic Leap each are spending literally over $1B US each and both are going with some form of flat, thin waveguides. This is a subject to itself that I plan to cover next time.


HMD – A huge number of options

HMD montageThere have been a number of comments on this blog that I am very negative about Head Mounted Displays (HMDs), but I believe I am being realistic about the issues with HMDs.   I’m an engineer by training and profession and highly analytical.  Part of building successful products is to understand the issues that must be solved for the product to work.  I have seen a lot of “demo ware” through the years that demo’s great but fails to catch on in the market.

It’s rather funny the vitriolic response from some of the people posting in the comments (some of which are so foul they have been blocked).  Why do they feel so threaten by bringing up issues with HMD?   There must have been over 100 HMDs go to market, some of these with big name companies behind them, over the last 40 years and none of them have succeed in making the break through to a consumer product.   Clearly the problem is harder than people think and it is not from a lack of trying by smart people.  Why is the “101st” attempt going to succeed?

I know there is a lot of marketing and hype going on today with HMDs, I’m trying to cut through that hype to see what is really going on and what the results will be.    I also have seen a lot of group think/chasing each other where a number of companies are researching the same general technology and the panic to get to market after another company has made a big announcement.  Many feel this is going on now with HMD in response to Google Glass.

Designers of HMDs have a huge number of design decision to make and invariably each of these choices result in pro’s and con’s.   Invariably they have to make trade-offs that tend to make the HMD good for some applications and worse for others.    For example, making a display see-through may be a requirement for augmented reality, but it makes them more expensive and worse for watching moves or pictures.    For immersive Virtual Reality the design may want a wide field of view (FOV) optics which for a given display resolution and cost means that you will have low angular resolution making it bad for information displays.

To begin with I would like to outline just some of the basic display modality options:

  1. See-through – Usually for augmented reality.  It has the drawback that it is poor for watching movies, pictures, and seeing detail content because whatever is visible in the real world becomes the “black.”   The optics tend to cost more and end up trading image quality for the ability to see through.   Also while they may be see-through, they invariably have to affect the view of the real world.
  2. Monocular (one-eye) – A bit harder for people to get used to but generally less expensive and easier to adjust.   People usually have one “dominant eye” and/or good eye where the one display should be located.   A non-see-through monocular can provide a bit of a see-through effect, but generally the display dominates.   Monocular HMDs support much more flexible mounting/support options as they don’t have to be precisely located in front center of the eyes.
  3. Binoculars (both eyes) – Generally supports better image quality than monocular. Can more than double the cost and power consumption of the display system (two of most everything plus getting them to work together).  Can support 3-D stereoscopic vision.   The two displays have to be centered properly for both eyes or will cause problems seeing the image and/or eye strain.  More likely to cause disorientation and other ill effects.
  4. Centered Vertically – While perhaps the obvious location, it means that the display will tend to dominate (or in the case of non-see through totally block) the user’s vision of the real world.   Every near eye display technology to at least some extent negatively affect the view of the real world; even see-though displays will tend to darken and/or color change, and/or distort the view.
  5. Above and Below – Usually monocular displays are located above or below the eye so that they don’t impair forward vision when the user looks straight forward.  This is not optimal for extensive use and can cause eye strain.   Generally the above and below position are better for “data snacking” rather than long term use.

Within the above there are many variations and options.   For example, with a see through display you could add sunglasses to darken or totally block the outside light either mechanically or with electronic shutters (which have their own issues), but they will still not be as optimal as a purpose built non-see through display.

Then we have a huge number of issues and choices beyond the display modality that all tend to interact with each other:

  1. Cost – Always an issue and trade-off
  2. Size and Weight – A big issue forHMDs as theyare worn on the head.  There are also issues with how the weightis distributed from front to back and side to side
    1. Weight on the person’s nose – I call this out because it is a particularly problem, any significant weight on the nose will build up and feel worse over time (anyone that has had glasses with glass rather than plastic lenses can tell you).    Therefore there is generally a lot of effort to minimize the weight on the person nose by distributing the force elsewhere, but this generally makes the device more bulky and has issues with messing up the user’s hair.   The nose bridge when use is generally used to center and stabilize the HMD.    Complicating this even more is the wide variety of shapes of the human head and specifically the nose.   And don’t kid yourself thinking that light guides will solve everything, they tend to be heavy as well.
  3. Resolution – Obviously more is better, but it comes at a cost both for the display and optics.  Higher resolution also tends to make everything bigger and take more power.
  4. Field of View (FOV)  – A wider FOV is more immersive and supports more information, but to support a wide FOV with good angular resolution throughout and support high acuity would require an extremely high resolution display with extremely good optics which would be extremely expensive even it possible.   So generally a display either as a wide FOV with low angular resolution or a narrower FOV with higher angular resolution.  Immersive game like application generally chose wider FOV while more informational based displays go with a narrower FOV.
  5. Exit Pupil Size – Basically this means how big the sweet spot is for viewing the image in the optics.  If you have every used an HMD or binoculars you will notice how you have to get them centered right or you will only see part of the image with dark ring around the outside.  As the FOV and Eye relief increase it becomes more and more difficult and expensive to support a reasonable exit pupil.
  6. Vision Blocking (particularly peripheral vision) ­– This can be a serious safety consideration for something you think would wear by walking and/or driving.  All these devices to a greater or less or extend block vision even if the display itself it off.    Light guide type displays are not a panacea in this respect either.  While light guide displays block less in front of the user, they have the image coming in from the sides and end up blocking a significant amount of a person’s side peripheral vision which is use to visually sense things coming toward the person.
  7. Distortion and Light Blocking – Any see-through device by necessity will affect the light coming from the real world.   There has to be a optical surface to “kick” the light toward the eye and then light from the real world has to go through that same surface and is affected.
  8. Eye relieve and use with Glasses ­ – This is an issue of how far the last optical element is away from the eye.   This is made very complicated by the fact that some people wear glasses and that faces have very different shapes.   This is mostly an issue for monocular displays where they often use a “boom” to hold the display optics.    As you want more eye relief, the optics have to get bigger for the same FOV, which means more weight which in turn makes support more of a problem.   This was an issue with Google Glass as they “cheated” by having very little eye relief (to the point that they said they were not meant to be used with glasses
  9. Vision correction – Always and issue as many people don’t have perfect vision and generally the HMD optics want to be in the same place as a person’s glasses.  Moving the HDM optics further away to support glasses makes them bigger and more expensive.   Building corrective lenses in to the HMD itself will have a huge impact on cost (you have to have another set of prescription lenses that a specially fit into the optics).   Some designs have included diopter/focus adjustment some many people also have astigmatism.
  10. Adjustment/Fit – This can be a big can of worms as the more adjustable the device is the better it can be made to fit, but then the more complex it gets to fit is properly.  With binocular displays you then have to adjust/fit both eye which may need moving optics.      
  11. Battery life (and weight) – Obvious issue and they are made worse dual displays.  At some point the battery has to be move either to the back of the head (hope you don’t have a lot of hair back there) or via a cable to someplace other than the head.
  12. Connection/cabling – Everyone wants wireless, but then this means severe compromises in terms of power, weight, support on the head, processing power (heat, battery power, and size).
  13. How it is mounted (head bands, over the head straps, face goggles) – As soon as you start putting much stuff on the head a simple over the ears with a noise bridge is not going to feel comfortable and you start to have to look to other ways to support the weight and hold it steady.   You end up with a lot of bad alternatives that will at a minimum mess with people’s hair.
  14. Appearance ­– The more youtry and do on the head, bigger and bulkier and uglier it is going to get.
    1. Look of the eyes – I break this out separately because human’s are particularly sensitive to how people’s eye’s look.  Many of the HMD displays make the eyes look particularly strange with optical elements right in front of the eyes (see below).  Epson eyes
  15. Storage/fragility – A big issue if this is going to be a product you wear when you go out.   Unlike you cell phone that you can slip in your pocket, HMD don’t generally fold up into a very small form factor/footprint and they are generally too fragile to put in your pock even if they (and with all their straps and cables they may have) would fit.
  16. Input – A very big topic I will save for another day.

 If you take the basic display types with all the permutations and combinations of all the other issues above (and the list is certainly not exhaustive) you get a mind boggling number of different configurations and over the last 40 years almost every one of these has been tried to a greater or lesser extent.  And guess what, none of these have succeeded in the consumer market.   Almost every time you try and improve one of the characteristics above, you hurt another.    Some devices have found industrial and military used (it helps if there is not a lot of hair on the user’s head, they are strong enough to carry some weight on their head, they don’t care what they look like and they are ordered to do the training).

In future posts, I plan on going into more detail on some of the options above.

One last thing on the comment section; I’m happy to let go through comments that disagree with me as it helps both me and others understand the subject.  But I am not going to put up with foul language and personal attacks and will quickly hit the “trash” button so don’t waste your time.   I put this under; if you can’t argue the facts then you trash the person category and I will file it accordingly.