Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
Desperately Seeking the Next Big Thing – Head Mounted Displays (HMDs) — Part 1
With Microsoft’s big announcement of HoloLens and spending a reported $150 million just for HMD IP from the small Osterhout Design Group, reports of Facebook spending about $2 billion for Oculus Rift, and the mega publicity surrounding Google Glass and the hundreds of millions they have spent, Head Mounted Displays (HMD) are certainly making big news these days.
Most of the articles I have seen pretty much just parrot the company press releases and hype up these as being the next big thing. Many of the articles have, to say the least, dubious technical content and at worst give misinformation. My goal is to analyze the technology and much of what I am seeing and hearing does not add up.
The question is whether these are lab experiments with big budgets and companies jumping the gun that are chasing each other or whether HMDs really are going to be big in terms of everyone using them? Or are the companies just running scared that they might miss the next big thing after cell phones and tablets. Will they reach numbers rivaling cell phone (or at least a significant fraction)? Or perhaps is there a “consolation prize market” which for HMDs would be to take significant share of the game market?
Let me get this out-of-the-way: Yes, I know there is a lot of big money and smart people working on the problem. The question is whether the problem is bigger than what is solvable? I know I will hear from all the people with 20/20 hindsight all the successful analogies (often citing Apple) but for every success there many more that failed to catch on in a big way or had minor success and then dived. As an example consider the investment in artificial intelligence (AI) and related computing in the 1980’s and the Intel iAPX 432 (once upon a time Intel was betting the farm on the 432 to be replacement for the 8086 until the IBM PC took off). More recently and more directly related, 3-D TV has largely failed. My point here is that big companies and lots of smart people make the wrong call on future markets all the time; sometimes the problems is bigger than all the smart people and money can solve.
Let me be clear, I am not talking about HMDs used in niche/dedicated markets. I definitely see uses for HMDs applications where hands-free use is a definite. A classic example is military applications where a soldier has to keep his hands free, is already wearing a helmet that messes up their hair and they don’t care what they look like, and they spend many hours in training. There are also uses for HMD in the medical field for doctors as a visual aid and for helping people with impaired vision. What I am talking about is whether we are on the verge of mass adoption.
Pardon me for being a bit skeptical, but on the technical side I still see some tremendous obstacles to HMD. As I pointed out on this blog soon after Google Glass was announced https://www.kguttag.com/2012/03/03/augmented-reality-head-mounted-displays-part-1-real-or-not/ HMDs have a very long history of not living up to expectations.
I personally started working on a HMD in 1998 and learned about many of the issues and problems associated with them. There are the obvious measurable issues like size, weight, fit/comfort and can you wear them with your glasses, display resolution, brightness, ruggedness, storage, and battery life. Then there are what I call the “social issues” like how geeky it looks, does it mess up a person’s hair, and taking video (a particularly hot topic with Google Glass). But perhaps the most insidious problems are what I lump into the “user interface” category which include input/control, distraction/safety, nausea/disorientation, and what I loosely refer to “as it just doesn’t work right.” These issues only just touch on what I sometime joking refer to as “the 101 problems with HMDs.”
A lot is made of the display device itself, be it a transmissive LCD, liquid crystal on silicon (LCOS), OLED, or TI’s DLP. I have about 16 years of history working on display devices, particularly LCOS, and I know the pro’s and con’s on each one in some detail. But as it turns out, the display device and its performance is among the least of the issues with HMDs, I had a very good LCOS device way back in 1998. As with icebergs, the biggest problems are the ones below the surface.
This first article is just to set up the series. My plan is to go into the various aspects and issue with HMDs trying to be as objective as I can with a bit of technical analysis. My next article will be on the subject of “One eye, two eyes, transparent or not.”
Karl, thanks for delving into this further. I hope you will touch on the Focus – Convergence problem as well with mass marketed 3D displays and 3D HMDs. It’s a problem that Walter Murch, a veteran Hollywood film editor, has written about and suggests it’s one of the reasons for the lack of large scale adoption of 3D film in theatres. People end up with headaches and blurry vision. HMD may fall into a similar camp.
Thanks,
I’m going to try and explain a number of the issues without sounding simply negative. As an engineer I know of a lot of issues and many of the issues are cross-linked in that improving one issue hurts others; you squeeze the balloon in one place and it pops out in another.
3-D in theaters is one of those concepts that has been tried again an again. Many people somehow thought that it was always with red and blue glasses (because that was the cheap way to do it in books and some early films) before the rebirth in the early 2000’s, but polarized glasses color 3-D has exited since 1952 or over 50 years before it was reborn with digital projectors. Some of the biggest problems were alleviated by not doing the “theme park” type 3D that pops the video out on the screen so much on long movies (which forces the person to look cross-eyed and causes eye strain). Even with the reducing in 3-D extremes, some people have issues. Personally while I’m OK watching 3-D it comes at the expense of brightness and color quality and rarely does it add much to the film.
3-D was of course a big flop in the home despite the push by manufacturers. 3-D TV at home ignores how most people normally watch TV, they don’t sit in the optimum location and orientation. They walk around, go to the fridge, and lay out on the couch, etc. all which play havoc with the stereo image. Some of the issues with 3-D TV are related to HMDs with the realities of how people live their lives.
Karl
Make sure you do your homework before you publish your articles. There’s a couple HMDs using VRD technology that are based on proven medical technologies.
You like to think there’s just not enough smart people and the problem is just to big to solve? We shall see señor Karl.
There can be a huge difference between what works in dedicated (and expensive) application (say medical or military) and what will work for a mass market product. The military had super sonic jets in the 1950’s, but as demonstrated by the financial failure of the SST, super sonic passenger jets proved to be impractical.
I think your burden of proof is misplaced. It is up to the new product concepts to prove that they are useful.
Also Vuzix’s patented waveguide technology has 56 degree FOV and projects an in focus image directly into the wearers eye. Vuzix is also working with Intel for their processor and the next generation of glasses that will replace the M100 will look like a normal pair of Oakley set to launch later this year. They already have the designs done and prototypes built.
Karl ,
Brenard Kress, Principal Optical Architect, Google [X] Labs , spoke at Stanford last October .
The subject: From Virtual Reality Headsets to Smart Glasses and Beyond
It was very comprehensive & informative . I thought you & your readers might be interested in what he has to say .
Go to :
http://talks.stanford.edu/scien/scien-colloquium-series/
Scroll down to :
SCIEN Colloquia 2014
Look in right hand column for:
From Virtual Reality Headsets to Smart Glasses and Beyond
Thanks for the link. It is a very interesting and informative presentation.
Karl
Karl, you are starting to sound like a real “Negative Nancy”.
The problem is that you have been away for too long from the heart of innovation and too distanced from the progress made with AR/VR, and “old” technologies such as Lcos, which itself has experienced disruptive upgrades, such as the likes of Himax’s front-lit Lcos.
2015 will be the year that AR/VR enters mass production and targets the consumer space. HTC’s/Valve’s Vive VR, Nvidia’s VR (rumored to be announced today), Hololens, Oculus Rift, Sony Project Morpheus, Google Glass 2.0 will all have a consumer launch within 1 year.
Why the push for AR/VR now? It’s quite simple really, the tech is finally mature and even more importantly, consumer want something new as the upgrades on the smartphones being released nowadays are not that revolutionary anymore. The smartphone experience is becoming static and consumers want the next “big thing”.
Admittedly, there is a growing number of AR/VR products being released in the short term, and not all will succeed. Some will take the lead, whilst others will either be scrapped or “rethinked”.
Best
Dan
Dan,
There is a lot of hype with respect to AR these days. I have no doubt that Himax is one of the main suppliers. Himax’s FrontLit is just using a lightguide to illuminate their LCOS.
Sure there are a lot of products being hyped, AR and 3-D glasses have by hyped for years and have not lived up to the hype. Two years ago it was Google Glass was going to be everywhere and now look at it.
AR/VR is being PUSH, but that does not mean there is a real PULL from consumers. I have been around for a lot of years and have seen they hype fest before. Just because manufactures want to sell it does not mean that the technology will meet user needs. There must have been well over 100 near eye products that have been released on the market in the last 20 years and they have all FAILED as CONSUMER products.
It takes very little to impress the so called technical press and blogs and they don’t seriously evaluate the technology. Tell me, if the technology is so great, why don’t we see any REAL pictures if the images rather than the results of Photoshop? There are some definite applications for some specific niche markets but from what I have seen so far, the technology is not ready for prime time consumer use.
As co-founder of Skully Helmets, you will see a practical, real world device that is useful and cutting edge. The technology is rather mainstream, but its implementation is revolutionary. Wait until this summer to get hold of a sample. Would like to see a technical analysis on this product.
Mike
Mike, thanks for you interest and comments. I would definitely like to see the display system in the Skully Helmet.
My understanding (based on what I have read on-line) is that Skully is using an LCOS display for its HUD system. I’m curious what Skully means by “infinite focus” per the marketing material. I’m also curious as to the spec’s (resolution, CD/M-sq, etc.).
Karl
Karl, did Mike @ Skully ever show you the tech info? They are using a Kopin transmissive display like Recon Jet and Snow goggles so I’m not sure how that works with infinite focus.
Whats the better application for true “augmented reality” – transmissive or reflective? I was looking at the Lumus waveguide and it seems pretty good. They use reflective LCoS now, but would there be any advantage to switch to transmissive like Citizen.
Do you know what the differences are between Citizen and Kopin and if either one has an advantage?
Thanks for your time,
Sam
Sam, That is quite a few interesting questions.
First, no I have not seen the technical details of Skully. There are pros and cons to all the technologies for near eye. What they generally do for most near eye optics is set the focus for “far vision”, but this can be problematic if the device is viewed below the eye level (as with Skully and Recon) with people that have bifocal/variable-focal glasses that have the reading vision but this can be fixed by having a diopter/focus adjustment (note that focus is different than where the virtual image appears to be).
Kopin’s transmissive technology uses color filters. It has advantages in that there is no color breakup as can occur with field sequential color (used with most LCOS and DLP near eye devices). Generally the downside of transmissive color filter is that the resolution is lower (limited by needing 3 sub pixels RGB, and the transistor and capacitor for each one) and perhaps more importantly is the blocking of light limiting brightness. But if you are “data snacking” (small amounts of data) as with a HUD or just using it for a view finder they can provide sufficient resolution and brightness.
Another option in the transmissive panel area is Epson but these tend to result in rather bulky devices for a given resolution.
The drawback of Transmissive comes at higher resolution. You either need a bigger device and/or block too much light and/or the liquid crystal effects blur the colors together as the RGB subpixels get too small. When you start wanting a wide field while maintaining high angular resolution (as with many VR/AR applications) you need a technology with smaller pixels that can only be supported by field sequential reflective technologies (LCOS and DLP) and generally LCOS is going to be lower power and more cost effective than DLP for near eye so it usually “wins” by a process of elimination.
I don’t know much about Citizen Fintech Miyota’s Transmissive technology. I assume it is a variation of the reflective technology they acquired from Displaytech/Micron which used “ferro-electic” LCOS or FLCOS. The spec sheet online says it has a field sequential rate of 120Hz which I would consider slow and prone to color breakup in headmount use. Generally when you go transmissive the depth of the LC doubles and the speed of the LC becomes about 4X slower which may be what is limiting their field sequential rate; it also limits how small they can make the pixels as the electric field controlling the pixels spreads through the thicker depth of LC.
So with Citizen, you are talking a relatively slow field sequential device versus a color filtered device both of which are transmissive. Without having done a detailed comparison, I would tend to pick the Kopin in this one-on-one. But if one needed higher resolution and/or smaller size, then the field sequential color LCOS would be my pick as their pixels are generally 3 to 6 times smaller than any transmissive technology can do.
Karl
This first article is just to set up the series. My plan is to go into the various aspects and issue with HMDs trying to be as objective as I can with a bit of technical analysis. My next article will be on the subject of “One eye, two eyes, transparent or not.”
Where is new article?
Bob,
That’s a fair question, I wish I had a better answer. I got busy with a startup working on high tech pet products and have neglected the blog.
There are literally many billions of dollars of investment going into AR/VR and so far the only ones making money are the people selling the technology to bigger companies desperate to get into it. I often make the analogy that to the San Francisco Gold rush, on average the miners made very little, the people that made all the money were the ones selling picks and shovels and Levi Strauss selling jeans. It certainly is a target rich environment for some sane analysis.
Is there something specific you would like to know about?
[…] secured a large contract with the U.S. Army, they could be considered a competitor to RAVN. I started covering Hololens several years before my involvement with RAVN. The views expressed in this article represent my own opinion and analysis and not that of […]