HMD – A huge number of options

HMD montageThere have been a number of comments on this blog that I am very negative about Head Mounted Displays (HMDs), but I believe I am being realistic about the issues with HMDs.   I’m an engineer by training and profession and highly analytical.  Part of building successful products is to understand the issues that must be solved for the product to work.  I have seen a lot of “demo ware” through the years that demo’s great but fails to catch on in the market.

It’s rather funny the vitriolic response from some of the people posting in the comments (some of which are so foul they have been blocked).  Why do they feel so threaten by bringing up issues with HMD?   There must have been over 100 HMDs go to market, some of these with big name companies behind them, over the last 40 years and none of them have succeed in making the break through to a consumer product.   Clearly the problem is harder than people think and it is not from a lack of trying by smart people.  Why is the “101st” attempt going to succeed?

I know there is a lot of marketing and hype going on today with HMDs, I’m trying to cut through that hype to see what is really going on and what the results will be.    I also have seen a lot of group think/chasing each other where a number of companies are researching the same general technology and the panic to get to market after another company has made a big announcement.  Many feel this is going on now with HMD in response to Google Glass.

Designers of HMDs have a huge number of design decision to make and invariably each of these choices result in pro’s and con’s.   Invariably they have to make trade-offs that tend to make the HMD good for some applications and worse for others.    For example, making a display see-through may be a requirement for augmented reality, but it makes them more expensive and worse for watching moves or pictures.    For immersive Virtual Reality the design may want a wide field of view (FOV) optics which for a given display resolution and cost means that you will have low angular resolution making it bad for information displays.

To begin with I would like to outline just some of the basic display modality options:

  1. See-through – Usually for augmented reality.  It has the drawback that it is poor for watching movies, pictures, and seeing detail content because whatever is visible in the real world becomes the “black.”   The optics tend to cost more and end up trading image quality for the ability to see through.   Also while they may be see-through, they invariably have to affect the view of the real world.
  2. Monocular (one-eye) – A bit harder for people to get used to but generally less expensive and easier to adjust.   People usually have one “dominant eye” and/or good eye where the one display should be located.   A non-see-through monocular can provide a bit of a see-through effect, but generally the display dominates.   Monocular HMDs support much more flexible mounting/support options as they don’t have to be precisely located in front center of the eyes.
  3. Binoculars (both eyes) – Generally supports better image quality than monocular. Can more than double the cost and power consumption of the display system (two of most everything plus getting them to work together).  Can support 3-D stereoscopic vision.   The two displays have to be centered properly for both eyes or will cause problems seeing the image and/or eye strain.  More likely to cause disorientation and other ill effects.
  4. Centered Vertically – While perhaps the obvious location, it means that the display will tend to dominate (or in the case of non-see through totally block) the user’s vision of the real world.   Every near eye display technology to at least some extent negatively affect the view of the real world; even see-though displays will tend to darken and/or color change, and/or distort the view.
  5. Above and Below – Usually monocular displays are located above or below the eye so that they don’t impair forward vision when the user looks straight forward.  This is not optimal for extensive use and can cause eye strain.   Generally the above and below position are better for “data snacking” rather than long term use.

Within the above there are many variations and options.   For example, with a see through display you could add sunglasses to darken or totally block the outside light either mechanically or with electronic shutters (which have their own issues), but they will still not be as optimal as a purpose built non-see through display.

Then we have a huge number of issues and choices beyond the display modality that all tend to interact with each other:

  1. Cost – Always an issue and trade-off
  2. Size and Weight – A big issue forHMDs as theyare worn on the head.  There are also issues with how the weightis distributed from front to back and side to side
    1. Weight on the person’s nose – I call this out because it is a particularly problem, any significant weight on the nose will build up and feel worse over time (anyone that has had glasses with glass rather than plastic lenses can tell you).    Therefore there is generally a lot of effort to minimize the weight on the person nose by distributing the force elsewhere, but this generally makes the device more bulky and has issues with messing up the user’s hair.   The nose bridge when use is generally used to center and stabilize the HMD.    Complicating this even more is the wide variety of shapes of the human head and specifically the nose.   And don’t kid yourself thinking that light guides will solve everything, they tend to be heavy as well.
  3. Resolution – Obviously more is better, but it comes at a cost both for the display and optics.  Higher resolution also tends to make everything bigger and take more power.
  4. Field of View (FOV)  – A wider FOV is more immersive and supports more information, but to support a wide FOV with good angular resolution throughout and support high acuity would require an extremely high resolution display with extremely good optics which would be extremely expensive even it possible.   So generally a display either as a wide FOV with low angular resolution or a narrower FOV with higher angular resolution.  Immersive game like application generally chose wider FOV while more informational based displays go with a narrower FOV.
  5. Exit Pupil Size – Basically this means how big the sweet spot is for viewing the image in the optics.  If you have every used an HMD or binoculars you will notice how you have to get them centered right or you will only see part of the image with dark ring around the outside.  As the FOV and Eye relief increase it becomes more and more difficult and expensive to support a reasonable exit pupil.
  6. Vision Blocking (particularly peripheral vision) ­– This can be a serious safety consideration for something you think would wear by walking and/or driving.  All these devices to a greater or less or extend block vision even if the display itself it off.    Light guide type displays are not a panacea in this respect either.  While light guide displays block less in front of the user, they have the image coming in from the sides and end up blocking a significant amount of a person’s side peripheral vision which is use to visually sense things coming toward the person.
  7. Distortion and Light Blocking – Any see-through device by necessity will affect the light coming from the real world.   There has to be a optical surface to “kick” the light toward the eye and then light from the real world has to go through that same surface and is affected.
  8. Eye relieve and use with Glasses ­ – This is an issue of how far the last optical element is away from the eye.   This is made very complicated by the fact that some people wear glasses and that faces have very different shapes.   This is mostly an issue for monocular displays where they often use a “boom” to hold the display optics.    As you want more eye relief, the optics have to get bigger for the same FOV, which means more weight which in turn makes support more of a problem.   This was an issue with Google Glass as they “cheated” by having very little eye relief (to the point that they said they were not meant to be used with glasses
  9. Vision correction – Always and issue as many people don’t have perfect vision and generally the HMD optics want to be in the same place as a person’s glasses.  Moving the HDM optics further away to support glasses makes them bigger and more expensive.   Building corrective lenses in to the HMD itself will have a huge impact on cost (you have to have another set of prescription lenses that a specially fit into the optics).   Some designs have included diopter/focus adjustment some many people also have astigmatism.
  10. Adjustment/Fit – This can be a big can of worms as the more adjustable the device is the better it can be made to fit, but then the more complex it gets to fit is properly.  With binocular displays you then have to adjust/fit both eye which may need moving optics.      
  11. Battery life (and weight) – Obvious issue and they are made worse dual displays.  At some point the battery has to be move either to the back of the head (hope you don’t have a lot of hair back there) or via a cable to someplace other than the head.
  12. Connection/cabling – Everyone wants wireless, but then this means severe compromises in terms of power, weight, support on the head, processing power (heat, battery power, and size).
  13. How it is mounted (head bands, over the head straps, face goggles) – As soon as you start putting much stuff on the head a simple over the ears with a noise bridge is not going to feel comfortable and you start to have to look to other ways to support the weight and hold it steady.   You end up with a lot of bad alternatives that will at a minimum mess with people’s hair.
  14. Appearance ­– The more youtry and do on the head, bigger and bulkier and uglier it is going to get.
    1. Look of the eyes – I break this out separately because human’s are particularly sensitive to how people’s eye’s look.  Many of the HMD displays make the eyes look particularly strange with optical elements right in front of the eyes (see below).  Epson eyes
  15. Storage/fragility – A big issue if this is going to be a product you wear when you go out.   Unlike you cell phone that you can slip in your pocket, HMD don’t generally fold up into a very small form factor/footprint and they are generally too fragile to put in your pock even if they (and with all their straps and cables they may have) would fit.
  16. Input – A very big topic I will save for another day.

 If you take the basic display types with all the permutations and combinations of all the other issues above (and the list is certainly not exhaustive) you get a mind boggling number of different configurations and over the last 40 years almost every one of these has been tried to a greater or lesser extent.  And guess what, none of these have succeeded in the consumer market.   Almost every time you try and improve one of the characteristics above, you hurt another.    Some devices have found industrial and military used (it helps if there is not a lot of hair on the user’s head, they are strong enough to carry some weight on their head, they don’t care what they look like and they are ordered to do the training).

In future posts, I plan on going into more detail on some of the options above.

One last thing on the comment section; I’m happy to let go through comments that disagree with me as it helps both me and others understand the subject.  But I am not going to put up with foul language and personal attacks and will quickly hit the “trash” button so don’t waste your time.   I put this under; if you can’t argue the facts then you trash the person category and I will file it accordingly. 

Desperately Seeking the Next Big Thing – Head Mounted Displays (HMDs) — Part 1

Untitled-2With Microsoft’s big announcement of HoloLens and spending a reported $150 million just for HMD IP from the small Osterhout Design Group, reports of Facebook spending about $2 billion for Oculus Rift, and the mega publicity surrounding Google Glass and the hundreds of millions they have spent, Head Mounted Displays (HMD) are certainly making big news these days.

Most of the articles I have seen pretty much just parrot the company press releases and hype up these as being the next big thing.   Many of the articles have, to say the least, dubious technical content and at worst give misinformation.   My goal is to analyze the technology and much of what I am seeing and hearing does not add up.

The question is whether these are lab experiments with big budgets and companies jumping the gun that are chasing each other or whether HMDs really are going to be big in terms of everyone using them?    Or are the companies just running scared that they might miss the next big thing after cell phones and tablets.   Will they reach numbers rivaling cell phone (or at least a significant fraction)?    Or perhaps is there a “consolation prize market” which for HMDs would be to take significant share of the game market?

Let me get this out-of-the-way:  Yes, I know there is a lot of big money and smart people working on the problem.   The question is whether the problem is bigger than what is solvable?  I know I will hear from all the people with 20/20 hindsight all the successful analogies (often citing Apple) but for every success there many more that failed to catch on in a big way or had minor success and then dived.   As an example consider the investment in artificial intelligence (AI) and related computing in the 1980’s and the Intel iAPX 432 (once upon a time Intel was betting the farm on the 432 to be replacement for the 8086 until the IBM PC took off).    More recently and more directly related, 3-D TV has largely failed.  My point here is that big companies and lots of smart people make the wrong call on future markets all the time; sometimes the problems is bigger than all the smart people and money can solve.

Let me be clear, I am not talking about HMDs used in niche/dedicated markets.  I definitely see uses for HMDs applications where hands-free use is a definite.  A classic example is military applications where a soldier has to keep his hands free, is already wearing a helmet that messes up their hair and they don’t care what they look like, and they spend many hours in training.   There are also uses for HMD in the medical field for doctors as a visual aid and for helping people with impaired vision.  What I am talking about is whether we are on the verge of mass adoption.

Pardon me for being a bit skeptical, but on the technical side I still see some tremendous obstacles to HMD.    As I pointed out on this blog soon after Google Glass was announced http://www.kguttag.com/2012/03/03/augmented-reality-head-mounted-displays-part-1-real-or-not/ HMDs have a very long history of not living up to expectations.

I personally started working on a HMD in 1998 and learned about many of the issues and problems associated with them.    There are the obvious measurable issues like size, weight, fit/comfort and can you wear them with your glasses, display resolution, brightness, ruggedness, storage, and battery life.   Then there are what I call the “social issues” like how geeky it looks, does it mess up a person’s hair, and taking video (a particularly hot topic with Google Glass).   But perhaps the most insidious problems are what I lump into the “user interface” category which include input/control, distraction/safety, nausea/disorientation, and what I loosely refer to “as it just doesn’t work right.”   These issues only just touch on what I sometime joking refer to as “the 101 problems with HMDs.”

A lot is made of the display device itself, be it a transmissive LCD, liquid crystal on silicon (LCOS), OLED, or TI’s DLP.    I have about 16 years of history working on display devices, particularly LCOS, and I know the pro’s and con’s on each one in some detail.   But as it turns out, the display device and its performance is among the least of the issues with HMDs, I had a very good LCOS device way back in 1998.   As with icebergs, the biggest problems are the ones below the surface.

This first article is just to set up the series.  My plan is to go into the various aspects and issue with HMDs trying to be as objective as I can with a bit of technical analysis.    My next article will be on the subject of “One eye, two eyes, transparent or not.”

I’m Back

Hi everyone,

Just a quick note today to say that I no longer at Navdy and I now have some time to get back to posting on this blog.   All the travel and work at a start-up was pretty all consuming.

I have some idea for some topics particularly related to the various head mounted display goings on from Google “Glass” to Microsoft’s recent “HoloLens.”   I also want to write more on the computer and game system history.   I like answering questions and would like suggestions for topics so please feel free to write.

The one thing I ask is that there be no questions directly related to Navdy as it would not be appropriate for me to answer them.   It is not going to make good reading for me to have to repeatedly reply with “no comment” or the like.   Almost everything else related to technology, projection, head mounted displays, lasers, computer/video-game history is fair game.

Oh yes one more thing, I am available again for consulting work.

Karl

Navdy Launches Pre-Sale Campaign Today

Bring Jet-Fighter Tech to Your Car with NavdyIts LAUNCH Day for Navdy as our presale campaign starts today. You can go to the  Navdy site to see the video.  It was a little over a year ago that Doug Simpson contacted me via this blog asking about how to make a aftermarket heads up display (HUD) for automobiels.     We went through an incubator program called Highway1 sponsored by PCH International that I discussed in my last blog entry.

The picture above is a “fancy marketing image” that tries to simulate what the eye sees (which is impossible to do with a camera as it turns out).   We figures out how to do some pretty interesting stuff and the optics works better than I thought was possible when we started.    The image image focuses beyond the “combiner/lens” to help with the driver seeing the images in the far vision is about 40 times brighter (for use in bright sunlight) than an iPhone while being very efficient.

Navdy Office

Being CTO at a new start-up has kept me away from this blog (a start-up is very time consuming).  We have raise some significant initial venture capital to get the program off the ground and the pre-sale campaign takes it to the next level to get products to market.  In the early days it was just me and Doug but now we have about a dozen people and growing.

Karl

Highway1 Incubator

Those that follow my blog are probably wondering what has happened to me these past months.   I have away from home for most of the last 4 months at an “incubator” program for start-ups called Highway1.   Navdy, for which I recently became CTO, was selected as one of 11 companies from over 100 applicants for the very first class of the Highway1  program sponsored by PCH International.

What makes Highway1 different from almost all other incubator programs these days is that it is totally focused on helping hardware start-ups.   Highway1 recognizes that hardware start-ups have special needs, are more difficult to get started, and have have to deliver a physical product unlike software companies.

The Highway1 office is in the Mission District of San Francisco where most of the time is spent, but the program also includes spending two weeks in Shenzhen China where many of the electronic products used around the world are made.   During the program companies are introduced to mentors from other companies and experts in the field as well as helped with introductions to angle and venture investment firms.

While in Shenzhen, the companies were introduced to manufacturers who could eventually be making their products.   Additionally our company received some very crucial support from PCH in Shenzhen in locating a company that could manufacture a critical component of our system.

Along the way, the people at the various 11 companies became friends and helped each other out.  Respecting each other was particularly important as the companies were cranking out prototypes sharing first on one and later two 3-D printers for making prototypes (as demo day neared, the those 3-D printers were pretty much running non-stop).   There was some incredible talent  technically, marketing, and business wise at these companies.

At the end of the program was “Demo Day” where more than 200 venture capitalists, investors, press, and technologist pack a large room at PCH’s U.S. Headquarters in San Francisco.  It was a chance for investors and the press to see what the companies had developed.   While Navdy presented, details of our product and plans were not released to the press because we are planning on launching our product later this year.  Navdy did receive serious interest from a number of VC’s with our demo after the formal presentations.

The whole Highway1 program was the dream of Liam Casey the founder and CEO of PCH, a company with over $700M in revenue.  You may not know the PCH name, but it is very likely that you have brand name products that they helped get to your home or office (be it anywhere in the world).   Liam was personally there to greet us at the beginning of the program and at key points along the way, and he told some great business stories.  The whole of the PCH team, be it the people from San Francisco, China, or Ireland, were always awesome to work with and incredibly nice reflecting PCH’s founder.

Comment: I don’t usually use the word “awesome” but the word was ubiquitous in San Francisco and it seemed to fit the people at PCH.

“If you haven’t tested it, it doesn’t work”

1994 Derek Roskell

Derek Roskell (circa 1994) of TI MOS Design Bedford, UK (Formal Photo – not how I remember him)

When I started this blog, I intended to write about more than displays and include some of my personal IC history.   Today’s story is about Derek Roskell of Texas Instrument’s who led the UK-based design teams I worked with between 1979 and 1997 on a number of the most complex I.C.s done up to that point including the 9995 16-bit CPU, 34010 and 34020 Graphics CPU’s, and the extremely complex 320C80 and 320C82 image processors with a 32-bit RISC CPU and 4 (C80) and 2 (C82) advanced DSP processors on one chip.  Every one of these designs quickly went from first silicon to product.

Having one successful design after the other may not seem so special in today’s era of logic synthesis and all the other computer tools, but back in 1979 we drew logic on paper and  transistors on sheets of frosted Mylar plastic with color pencils that then were then digitized by hand.  We then printed out large “composites” plots on giant flat-bed pen plotters (with each layer of the I.C. in a different color) and then verified all the circuitry by hand and eye (thank goodness by the mid 1980’s we got computer schematic verification).

In those days it all could go very wrong and it did for a 16-bit CPU call the 9940 and a spinoff version the 9985 that were design in Houston Texas in 1977-1978.   It went so bad that the both the 9940 and 9985 were never fully functional, causing the designer to be discredited (whether at fault or not) and many people to leave.

In the wake of the 9940/9985 disaster, in 1979 management pick me, the young hotshot only 1.5 years out of college, to lead the architecture and logic design of a new CPU, the TMS9995, to replace the failed TMS9985.   There was one hitch, they wanted to use a  TI design group in Bedford England.  So after some preliminary work, I packed up for a 6 month assignment in Bedford where I first met Derek Roskell.

Derek in circa 2010 DSCN1430

Derek more “In Character” but taken years later

To say Derek is a self-deprecating is a gross understatement.  The U.S. managers at TI at the time were more the self-assertive, aggressive, “shoot from the hip,” cut corners (which resulted in the 9940/9985 debacle) and generally didn’t take well to Derek’s “English working class” (said with great affection) style with the all too frequent laugh at the “wrong” time.

When I first met Derek he was this “funny old guy” who at had worked on “ancient” TTL technology.  He  was around 40 and seem like an old man in a world of engineers in their 20’s and early 30’s who he led.   As it turned out, Derek was the steady hand that guided a number of brilliant people who worked under him.   He made sure my “brilliant” architecture and logic design actually worked.  You don’t have one successful design after another, particularly back then, by accident.

Upper management  was always pressuring to get thing done faster which could only be accomplished by cutting corners.  They called Bedford a “country club” for resisting the pressure.  Derek was willing to take the heat and do things the “right way” because he understood the consequences of cutting corners.

For most engineers fun part of engineering is doing the original design work.  That is the “creative stuff” and the stuff that gets you noticed.   Also most engineers have big egos and think, “of course what I designed works.”  But when you are designing these massive I.C.’s with hundreds of thousand and later millions of transistors, even if 99.99% of the design is correct, there will be a hopeless number errors to debug and correct.  Most of what it takes to make sure a design works is the tedious process of “verification.”

A couple of months back I had a small reunion in Bedford with some friends from the old days including Derek.   Everyone remembered Derek for one thing he constantly chided the designers with, “If you haven’t tested it, it doesn’t work.”  Pretty good advice.

Epilog

TI, like most companies today, in their search for “shareholder value” closed the large Bedford UK site around 1995 but still kept Bedford MOS designers who had so many proven successes and moved them to a rental building Northhampton.   Through the years TI kept “consolidating/downsizing” and finally 2011 it shut down the last vestiges of their design operation in England and losing a number of extremely talented (and by then) senior people.

Below is a picture taken of the design team in Bedford that worked with me on the 320C80.

320C80 Bedford Team cropped and Sharpend

320C80 Bedford Design Team (1994)

Whatever happened to pico projectors embedding in phones?

iPad smBack around 2007 when I was at Syndiant we started looking at the pico projector market, we talked to many of the major cell phone as well as a number of PC companies and almost everyone had at least an R&D program working on pico projectors.  Additionally there were market forecasts for rapid growth of embedded pico projectors in 2009 and beyond.  This convinced us to develop small liquid crystal on silicon (LCOS) microdisplay for embedded pico projectors.  With so many companies saying they needed pico projectors, it seemed like a good idea at the time.  How could so many people be wrong?

Here we are 6 years later and there are almost no pico projectors embedded in cell phones or much else for that matter.   So what happened?   Well, just about the same time we started working on pico projectors, Apple introduced their first iPhone.    The iPhone overnight roughly tripled the size of the display screen of a smartphone such as a Blackberry.  Furthermore Apple introduced ways to control the screen (pinch/zoom, double clicking to zoom in on a column, etc.) to make better use of what was still a pretty small display.   Then to make matter much worse, Apple introduce the iPad and tablet market took off almost instantaneously.    Today we have larger phones, so called “phablets,” and small tablets filling in just about every size in between.

Additionally I have written about before, the use model for a cell phone pico projector shooting on a wall doesn’t work.   There is very rarely if ever a dark enough place with something that will work well for a screen in a place that is convenient.

I found that to use a pico projector I had to carry a screen (at least a white piece of paper mounted on a stiff board in a plastic sleeve to keep clean and flat) with me.   Then you have the issue of holding the screen up so you can project on it and then find a dark enough place that the image looks good.    By the time you carry a pico projector and screen with you, a thin iPad/tablet works better, you can carry it around the room with ease, and you don’t have to have very dark environment.

The above is the subjective analysis, and the rest of this article will give some more quantitative numbers.

The fundamental problem with a front projector is that it has to compete with ambient light whereas flat panels have screens that absorb generally 91% to 96% of the ambient light (thus they look dark when off).     While display makers market contrast number, these very high contrast numbers assume a totally dark environment, in the real world what counts is the net contrast, that is the contrast factoring in ambient light.

Displaymate has an excellent set of articles (including SmartPhone Brightness Shootout, Mobile Brightness Shootout 2, and Smartphone Shootout 2) on the subject of what they call “Contrast Rating for High Ambient Light” (CRHAL)  which they define as the display brightness per unit area (in candela’s per meter squared, also known as “nits”) of the display divide by the reflectivity of ambient light in percent by the display.

Displaymate’s CRHAL is not a “contrast ratio,” but it gives a good way to compare displays when in reasonable ambient light.  Also important, is that for a front projector it does not take much ambient light to end up dominating the contrast.  For a front projector even dim room light is “high ambient light.”

The total light projected out of a projector is given in lumens so to compare it to a cell phone or tablet we have to know how big the projected image will be and the type of screen.   We can then compute the reflected light in “nits”  which is calculated by the following formula Candelas/meter2 = nits = Gn x (lumens/m2)/PI (where Gn is the gain of the screen and PI = ~3.1416).   If we assume a piece of white paper with a gain of 1 (about right for a piece of good printer paper) then all we have to do is calculate the screen area in meters-square, multiply by the lumens and divide by PI.

A pico projector projecting a 16:9 (HDTV aspect ratio) on a white sheet of notebook paper (with a gain of say 1) results in 8.8-inch by 5-inch image with an area of 0.028 m2 (about the same area as an iPad2 which I will use for comparison).    Plugging a 20 lumen projector in to the equation above with a screen of 0.028 m2 and a gain of 1.0 we get 227 nits.  The problem is that same screen/paper will reflected (diffusing it) about 100% of the ambient light.   Using Displaymate’s CRHAL we get 227/100 = 2.27.

Now compare the pico projector numbers to an iPad2 of the same display area which according to Displaymate has 410 nits and only reflects 8.7% of the ambient light.   The CRHAL for the iPad2 is 410/8.7  = 47.   What really crushes the pico projector by about 20 to 1 with CRHAL metric is that the flat panel display reflects less than 10th of the ambient light where the pico projector’s image has to fight with 100% the ambient light.

In terms of contrast,to get a barely “readable” B&W image, you need at least 1.5:1 contrast (the “white” needs to be 1.5 brighter than the black) and preferably more than 2:1.   To have moderately good (but not great) colors you need 10:1 contrast.

A well lit room has about 100 to 500 lux (see Table 1 at the bottom of this article) and a bright “task area” up to 1500 lux.   If we take 350 lux as a “typical” room then for the sheet of paper screen there are about 10 lumens of ambient light in our 0.028 m2 image from used above.   Thus our 20 lumen projector on top of the 10 lumens of ambient has a contrast ratio of 30/10 or about 3 to 1 which means the colors will be pretty washed out but black on white text will be readable.  To get reasonably good (but not great) colors with a contrast ratio of 10:1 we would need about 80 lumens.   By the same measure, the iPad2 in the same lighting would have a contrast ratio of about 40:1 or over 10x the contrast of a 20 lumen pico projector.   And the brighter the lighting environment the worse the pico projector will compare.    Even if we double or triple the lumens, the pico projector can’t compete.

With the information above, you can plug in whatever numbers you want for brightness and screen size and no matter was reasonable numbers you plug in, you will find that a pico projector can’t compete with a tablet even in moderate lighting conditions.

And all this is before considering the power consumption and space a pico projector would take.   After working on the problem for a number of years it became clear that rather than adding a pico projector with its added battery, they would be better off to just make the display bigger (ala the Galaxy S3 and S4 or even the Note).   The microdisplay devices created would have to look for other markets such as near eye (for example, Google Glass) and automotive Heads Up Display (HUD).

Table 1.  Typical Ambient Lighting Levels (from Displaymate)

Brightness Range

Description

0 lux  –

100 lux  –

500 lux  –

1,000 lux  –

3,000 lux  –

10,000 lux  –

20,000 lux  –

50,000 lux  –

100,000 lux  –

100 lux

500 lux

1,500 lux

5,000 lux

10,000 lux

25,000 lux

50,000 lux

75,000 lux

    120,000 lux

Pitch black to dim interior lightingResidential indoor lighting

Bright indoor lighting:  kitchens, offices, stores

Outdoor lighting in shade or an overcast sky

Shadow cast by a person in direct sunlight

Full daylight not in direct sunlight

Indoor sunlight falling on a desk near a window

Indoor direct sunlight through a window

Outdoor direct sunlight

Himax FSC LCOS in Google Glass — Seeking Alpha Article

Catwig to Himax ComparisonThis blog was the first to identify that there was a Himax panel in an early Google Glass prototype and the first to identify that there was a field sequential color LCOS panel inside Google Glass.  Due to the connection it was a reasonable speculation but there was no proof that Himax was in Google Glass.

Then when Catwig published a teardown of Google Glass last week (and my inbox lit up with people telling me about the article) there were no Himax logos to be seen which started people to wondering if there was indeed a Himax display inside.   As a result of my prior exclusive finds on Himax, LCOS and Google Glass, I was ask to contribute to Seeking Alpha and I just published an article that details my proof that there is a Himax LCOS display inside the current Google Glass.   In that article, I also discounted some recent speculation that Google Glass was going to use a Samsung OLED microdisplay anytime soon.

 

 

 

 

Extended Temperature Range with LC Based Microdisplays

cookies and freezing

Extreme Car Temperatures

A reader, Doug Atkinson, asked a question about meeting extended temperature ranges with LC based microdisplays, particularly with respect to Kopin.    He asked the classic “car dash in the desert and the trunk in Alaska” question. I thought the answer would have broader interest so I decided to answer it it here.

Kopin wrote a good paper that is available on the subject in 2006 titled “A Normally Black, High Contrast, Wide Symmetrical Viewing Angle AMLCD for Military Head Mounted Displays (HMDs) and Other Viewer Applications”. This paper is the most detailed one readily available describing the how Kopin’s transmissive panels meet the military temperature and shock requirements.  It is not clear that Kopin uses this same technology for their consumer products as this paper is specifically addressing what Kopin did for military products.

With respect to LC microdisplays in general, it should realized that there is not a huge difference in the technical spec’s of the liquid crystals between the LC’s  most small panel microdisplays use and large flat panels in most cases. They often just use different “blends” of the very similar materials. There are some major LC differences including TN (twisted nematic), VAN (vertically aligned nematic), and others.   Field sequential color are biased to wanting faster switching “blends” of the LC.

In general, anywhere a large flat panel LC can go, a microdisplay LC can go. The issue is designing the seals and and other materials/structures to withstand the temperature cycling and mechanical shock which requires testing,  experimentation, and development.

The liquid crystals themselves generally will go through different phases from freezing (which is generally fatal) to heating up to the the “clearing point” where the display stops working (but is generally recoverable).  There is also a different spec for “storage temperature range” versus “operating temperature range.” Generally it is assumed the device only has to work in a temperature range in which a human could survive.

At low temperature the LC gets “sluggish” and does not operate well but this can be cured by various “heater mechanisms” including having heating mechanisms designed into the panel itself.  The liquid crystal blends are often designed/picked to work best at a higher temperature range because it is easier to heat than cool.

Field sequential color LCOS is more affected by temperature change because temperature affects not only the LC characteristics, but the switching speed. Once again, this can be dealt with by designing for the higher temperature range and then heating if necessary.

As far as Kopin’s “brightness” goes (another of Doug’s questions), a big factor is how powerful/bright the back light has to be. The Kopin panel blocks something like 98.5% of the light by their own spec’s. What you can get away with in a military headset is different than what you may accept in a consumer product in terms of size, weight, and power consumption. Brightness in daylight is a well known (inside the industry) issue for Kopin’s transmissive panels and one reason that near eye display makers have sought out LCOS.

[As an aside for completeness about FLC]  Displaytech which was sold the Micron and then sold to Citizen Finetech Miyota and the Kopin bought Forth Dimension Display (FDD) both use Ferro-electric LC (FLC / FLCOS) which does have a pretty dramatically different temperature profile that is very near “freezing” (going into a solid state) a little below 0C which would destroy the device. Displaytech claimed (I don’t know about FDD) that they had extended the low temperature range but I don’t know by how much. The point is that the temperature range of FLC is so different that meeting military spec’s is much more difficult.

AR Display Device of the Future: Color Filter, Field Sequential, OLED, LBS and other?

I’m curious what people think will be the near eye microdisplay of the future.   Each technology has its own drawbacks and advantages that are well known.   I thought I would start by listing summarizing the various options:

Color filter transmissive LCD – large pixels with 3 sub-pixels and lets through only 1% to 1.5% of the light (depends on pixel size and other factors).  Scaling down is limited by the colors bleeding together (LC effects) and light throughput.  Low power to panel but very inefficient use of the illumination light.

Color filter reflective (LCOS) – same as CF-transmissive but the sub-pixels (color dots) can be smaller, but still limited scaling due to needing 3 sub-pixels and color bleeding.  Light throughput on the order of 10%.  More complicated optics than transmissive (requires a beam splitter), but shares the low power to panel.

Field Sequential Color (LCOS) – Color breakup from sequential fields (“rainbow effect”), but the pixels can be very small (less than 1/3rd that of color filter).   Light throughput on the order of 40% (assuming a 45% loss in polarization).  Higher power to the panel due to changing fields.  Optical path similar to CF-LCOS, but to take advantage of the smaller size requires smaller but higher quality (low MTF) optics.   Potentially mates well with lasers for very large depth of focus so that the AR image is in focus regardless of where the user’s eyes are focused.

Field Sequential Color (DLP) – Color breakup form FSC but can go to higher field rates than LCOS to reduce the effects.   Device and control is comparatively high powered and has a larger optical path.  The pixel size it bigger than FSC LCOS due to the physical movement of the DLP mirrors.   Light throughput on the order of 80% (does not have the polarization losses) but falls as pixel gets smaller (gap between mirrors is bigger than LCOS).    Not sure this is a serious contender due to cost, power of the panel/controller, and optical path size, and nobody I know of has used it for near eye, but I listed it for completeness

OLED – Larger pixel due to 3 color sub-pixels.  It is not clear how small this technology will scale in the foreseeable future.  OLED while improving the progress has been slow — it has been the “next great near eye technology” for 10 years.   Has a very simple optical path and potentially high light efficiency which has made it seem to many like on technology with the best future, but it is not clear how it scales to very small sizes and higher resolution (the smallest OLED pixel I have found is still about 8 times bigger than the smallest FSC LCOS pixel) .    Also it is very diffuse light and therefore the depth of focus will be low.

Laser Beam Steering – While this one sounds good to the ill-informed, the need to precision combine 3 separate lasers beams tends to make it not very compact and it is ridiculously to expensive today due to the special (particularly green) lasers required.  Similar to field sequential color, there are breakup effects of having a raster scan (particularly with no persistence like a CRT) on a moving platform (as in a head mount display).   While there are still optics involved to produce an image on the eye, it could have a large depth of focus.   There are a lot of technical and cost issues that keep this from being a serious alternative any time soon, but it is in this list for completeness.

I particularly found it interesting that Google’s early prototype used a color filter LCOS and then they switched to field sequential LCOS.    This seems to suggest that they chose size over issues with the field sequential color breakup.    With the technologies I know of today, this is the trade-off for any given resolution; field sequential LCOS pixels are less than 1/3rd the size (a typically closer to 1/9th the size) of any of the existing 3-color devices (color filter LCD/LCOS or OLED).

Olympus MEG4.0

Olympus MEG4.0 – Display Device Over Ear

It should also be noted that in HMD, an extreme “premium” is put on size and weight in front of the eye (weight in front of the eye creates as series of ergonomic and design issues).    This can be mitigated by using light guides to bring the image to eye and locating a larger/heavier display device and its associate optics to a less critical location (such as near the ear) as Olympus has done with their Meg4.0 prototype (note, Olympus has been working at this for many years).  But doing this has trade-offs with the with the optics and cost.

Most of this comparison boils down to size versus field sequential color versus color sub-pixels.    I would be curious what you think.