Lenovo’s STMicro Based Prototype Laser Projector (part 1)

Lenovo Tech World Projector 001At Lenovo at their Tech World on May 27th 2015 showed a Laser Beam Scanning (LBS) projector integrated into a cell phone prototype (to be clear, a prototype and not a product).   White there has been no announcement of the maker of the LBS projector, there is no doubt that is made by STM as I will show below (to give credit where it is due, this was first shown on a blog by Paul Anderson focused on Microvision )

ST-720p- to Lenove comparison 2The comparison at left is base on video by Lenovo that included an exploded views of the projector and pictures of STM’s 720p projector from an article from Picoprojector-info.com on Jan 18, 2013.   I have drawn lines comparing various elements such as the size and placement of connectors and other components, the size and placement of the 3 major I.C.’s, and even the silk screen “STM” in the same place on both the Lenovo video and the STM article’s photo (circled in yellow).

While there are some minor differences, there are so many direct matches that there can be no doubt that Lenovo is using STM.

The next interesting to consider is how this design compares to the LBS design of Microvision and Sony in the Celluon projector.   The Lenovo video shows the projector as being about 34mm by 26mm by 5mm thick.  To check this I took the a photo from the STM to CelluonTO SCALE  003Picoprojector.com
article and was able to fit the light engine and electronic into a 34mm by 26mm rectangle arranged as they are in the Lenovo video (yet one more verification that it is STM).   I then took a picture I took of the Celluon board to the same scale and show the same 34x26mm rectangle on it.   The STM optics plus electronics are 1/4 the area and 1/5th the volume (STM is 5mm thick versus Microvision/Sony’s 7mm).

The Microvision/Sony is has probably about double the lumens/brightness of the STM module due to have two green and two red lasers and I have not had a chance to compare the image quality.   Taking out the extra two lasers would make the Microvision/Sony engine optics/heat-sinking smaller by about 25% and have a smaller impact on the board space, but this would still leave them over 3X bigger than STM.   The obvious next question is why.

One reason is that the STM either has a simpler electronics design or is more integrated and/or some combination thereof.  In particular the Microvision/Sony design requires an external DRAM (large rectangular chip in the Microvision/Sony).    STM probably still needs DRAM, but it is likely integrated into one of their chips.

There are not a lot of details on the STM optics (developed by bTendo of Israel before being acquired by STM).   But what we do know is STM uses separate simpler and smaller horizontal and vertical mirrors versus Microvision significantly larger and more complex single mirror assembly.  Comparing the photos above, the Microvision mirror assembly alone is almost as big as STM’s entire optical engine with lasers.   The Microvision mirror assembly has a lot of parts other than the MEMs mirror including some very strong magnets.  Generally the optical path of the Microvision engine requires a lot of space to enter and exit the Microvision mirror from the “right” directions.

btendo optics

On the right I have captured two frames from the Lenovo video showing the optics from two directions.  What you should notice is that the mirror assembly is perpendicular to the incoming laser light.  There appears to be a block of optics (pointed to by the red arrow in the two pictures) that redirects the light down to the first mirror and then returning it to the second mirror.  The horizontal scanning mirror is clearly shown in the video but it is not clear (so I took an educated guess) as to the location of the vertical scanning mirror.

Also shown at the right is bTendo patent 8,228,579 showing the path of light for their two scanning mirror design.   It does not show the more complex block of optics required to direct the light down to the vertical mirror and then redirect it back down to the horizontal mirror and then out as would be required in the Lenovo design.    You might also notice that there is a flat clear glass/plastic output cover shown in the at the 21s point in the video, this is very different from the Microvision/Celluon/Sony design show below.

Microvision mirror with measurements

Microvision Mirror Assembly and Exit Lens

Shown at left is the Microvision/Celluon beam scanning mirror and the “Exit” Lens.   First notices the size and complexity of the scanning mirror assembly with magnets and coils.  You can see the single round mirror with its horizontal hinge (green arrow) and the vertical hinge (yellow arrow) on the larger oval yoke.   The single mirror/pivot point causes an inherently bow-tied image.  You can see how distorted the mirror looks through the Exit Lens (see red arrow); this is caused by the exit lens correcting for the bow-tie effect.  This significant corrective lens is also a likely source of chroma aberrations in the final image.

Conclusions

All the above does not mean that the Leveno/STM is going to be a successful product.   I have not had a chance to evaluated the Lenovo projector and I still have serious reservations about any embedded projector succeeding in a cell phone (I outlined my reasons in an August 2013 article and I think they still hold true).    Being less than 1/5th the volume of the Microvision/Sony design is necessary but I don’t think is sufficient.

This comparison only shows that the STM design is much smaller than Microvisions and Microvision has only made relatively small incremental progress in size since the ShowWX announced in 2009) and Sony so far has not improved on it much, at least so far.

IRIS HUD on Indiegogo Appears to be Repackaged Pioneer HUD(s)

The startup IRIS has started and Indiegogo presale campaign for not just one (a major challenge for a new company)  but two different HUD designs, one “laser” and one DLP based.    Their video and “story” talk about how they designed this HUD and even show some CAD pictures, 3-D printing (of what?), and a CNC milling machine (but not showing what is being made).

The problem is that this “new” unit looks almost identical at every point to Pioneer HUD announced shipped in Japan in 2012 (with a slightly updated version in 2013) see such as, “The Verge” article from May 2012.   Pioneer’s model was also a “Laser HUD” and used a Microvision beam scanning mirror and laser control electronics.

Pioneer then in late 2013 Pioneer introduced a less expensive model based on Texas Instrument’s DLP that I wrote about on Seeking Alpha.   And low and behold IRIS also has a DLP version.  Where the Laser version was sold with Pioneer’s proprietary navigation system, the DLP version was sold in Europe that connect to a smart-phone.

According to IRIS’s Indiegogo campaign,

This limited quantity of Laser (30) and DLP (300) units are being assembled and will be ready to ship at the end of the campaign.  

Assuming that if IRIS is actually going to be delivering these products (that is always a big “if” for a new high-tech product on Indiegogo), the only rational conclusion is that they are shipping Pioneer’s unsold inventory of at least Laser and DLP engines if not whole systems.

Below are a series of comparison photos with alternating photos of the IRIS HUD and the Pioneer Laser HUD.   I have draw lines connecting corresponding elements between the IRIS and Pioneer HUDs.   I will go into some more of the business issues after the photos.

IRIS Pioneer Comparison 003

IRIS does claim to be adding features that were not in the either the Laser or DLP based Pioneer systems, specifically they say they are adding “gesture recognition” and connection to the OBD (on-board diagnostics) port.   Being I think most generous, it could be that they are taking the old unsold Pioneer units and modifying them.   I could be OK with this, but I am always a bit distrustful when I catch someone fudging on what they did.

Pioneer DLP hud2Interestingly, the Pioneer DLP HUD (left) while it worked with smartphones, as does IRIS’s HUD, it looks quite different and it optically different in just about every way but the combiner.   The Pioneer Laser HUD rear projected on a screen behind a large plastic lens that is then viewed via the combiner (the “combiner” is that large curved plastic mostly transparent but slightly mirrored lens at the front of the unit).  The Pioneer DLP HUD front projects on a a screen that is then seen reflected in- and magnified by- the combiner.

Additionally, the Pioneer Laser HUD required you to remove your sun visor to mount the unit where their DLP HUD strapped to the sun visor (see the photo above).   This got me curious how they could be selling two radically different designs that also mounted differently while showing a single product so I posted the follow question and got the response below on IRIS’s Facebook page:

Karl Guttag‎, “Is the case and mounting the same for the Laser and the DLP versions of the product?

IRIS “Yes, Absolutely the same!

I guess it is possible that they took Pioneer Laser HUD cases and reworked/redesigned them to fit the DLP and added gesture recognition and OBD.   That would seem to me to be a pretty major effort for a small team with little known funding.

Yet they say they are going to ship units at the end of their Indiegogo campaign this month which would suggest they have them in-stock.   If they are so close to having real product, then I would have expected them to be out there demonstrating them to reviewers and not just showing the carefully staged video on Indiegogo.   Maybe they have something, but maybe it does not work very well.   Something just does not seem to add up.

BTW, I have had the opportunity to see both the Pioneer Laser and DLP based HUDs.  Frankly, neither one seems very practical.  The Laser HUD requires you to remove your sun visor to mount it and they give you a small sun visor that only goes up and down (can’t block your side window and and does not cover enough).  Additionally unless you are very short, the combiner tends to cut through your critical forward vision.   The DLP version was worse in that it mounted below the sun visor and totally blocks the forward vision if you are tall and/or your seat adjust to a high position.  The bottom line, there are reasons why the Pioneer units did not sell well.

Disclaimer:

I previously worked as CTO for Navdy which is also developing an aftermarket HUD product and could be seen as a competitor for IRIS.  I currently have no financial interest in Navdy.   Because of my prior position at Navdy and knowledge of non-public information, it is not appropriate for me to comment on their product.

Celluon LBS Analysis Part 2B – “Never In-Focus Technology” Revisit

Celluon alignment IMG_9775

After Alignment alignment target (click for bigger image)

I received concerns that the chroma aberrations (color fringes) seen in the photos in Part 2B were caused by poor alignment of the lasers.   I had aligned the lasers per Celluon’s instructions before running the test but I decided to repeat the alignment to see if there would be a difference.

After my first redo of the alignment I notice that the horizontal resolution got slightly better in places but the vertical resolution got worse.   The problem I identified is that the alignment procedure does not make aligning the pairs of red and green lasers easy.  The alignment routine turns all 5 lasers on a once which makes it very difficult to see pairs of lasers of the same color.

To improve on the procedure, I put a red color filter in front of the projector output to eliminate the blue and two green lasers and then aligned the two red laser to each other.  Then using a green color filter, I aligned the two green lasers.  I did this for both horizontally and vertically.   On this first pass I didn’t worry about the other colors.  On the next pass I moved the red pair by always the same amount horizontally and vertically and similarly for the green pair.  I went around this loop a few times trying for the best possible alignment (see picture of alignment image above).

After the re-alignment I did notice some slightly better horizontal resolution in the vertical lines (but not that much and not everywhere) and some very slight improvement in the vertical resolution.   There was still the large chroma aberrations, particularly on the left side of the image (much less so on the right side) that some had claimed were “proof” that the lasers were horribly aligned (which they were not before).   The likely cause of the chroma aberrations is the output lens and/or angle error in the mechanical alignment of the lasers.

Below shows the comparison before and after on the 72-inch diagonal image.laser alignment comparison 2

Note the overall effect (and the key point of the earlier article_ of the projected image going further out of focus at smaller image sizes.   Even at 72-inch diagonal the image is far from what should be considered sharp/in-focus even after the re-calibration.

Below shows the left and right side of the 72-in diagonal image.  The green arrows show that there is minimal chroma aberration on the right side but there is a significant issue on the left side.   Additionally, you may note the sets of parallel horizontal lines have lost all definition on the left and right side and the 1 pixel wide targets are not resolved (compare to the center target above).   This loss of resolution on the sides of the image is inherent in Microvision’s scanning process.

Celluon 72-in diag left-right targets

Center left and center right of 72-in diag. after re-alignment (click on thumbnail for full resolution image)

While the re-alignment did make some parts of the image a little more defined, the nature of the laser scanning process could not fully resolved other areas.   In future article I hope to get into this some more.

One other small correction from the earlier article, the images labeled “24-inch diagonal” are actually closer to 22-inches in diagonal.

Below are the high-resolution (20 megapixel) images for the 72-in, 22-in, and 12-in images after calibration.  I used a slightly different test patter which is also below (click on the various images for the high-resolution version).

Celluon 72-in diag  recalibrated IMG_9783

Celluon 72-in diag re-calibrated (click for full size image)

Celluon 22-in diag  recalibrated IMG_9864

Celluon 22-in diag re-calibrated (click for full size image)

Celluon 12-in diag recalibrated IMG_9807

Celluon 12-in diag re-calibrated (click for full size image)

 

 

 

 

interlace res-chart-720P G100A

Test Chart for 1280×270 resolution (click for full resolution)

Just to verify that my camera/lens combination was in no way limiting the visible resolution of the projected image, I also took some pictures of about 1/3 of the image (to roughly triple the resolution) and with an 85mm F1.8 “prime” (non-zoom) lens shot at F6.3 so it would show extremely find detail (including the texture of the white wall the image was projected onto).

Below are the images showing the Center-Left, Center and Center-Right resolution targets of the test chart above.   Among other things to notice how the resolution of the projected image drops from the center to the left and right and also how the chroma/color aberrations/fringes are most pronounce on the center-left image.

 

Celluon 72-in diag 85mm Center-Left 9821

85mm Prime Lens Center Left Target and Lines (click for full size image)

Celluon 72-in diag 85mm lens center  9817

85mm Prime Lens Center Target and Lines (click for full size image)

Celluon 72-in diag 85mm center-right 9813

85mm Prime Lens Center-Right Target and Lines (click for full size image)

Karl

Celluon Laser Beam Steering Analysis Part 2 – “Never In-Focus Technology”

June 6th 2015 – Note, I am in the process of updating this analysis with new photos.  The results are not dramatically different but I was able to improve the horizontal resolution slightly and now have some better pictures.    

Celluon image size comparison center cropsOne of the first things I noticed when projecting text pattern images with the Celluon PicoPro was that the images were very blurry.   I later found out that the smaller the image the blurrier it became.

To the left are high-resolution center crops of images taken with a 12-inch diagonal (about as big as you can get on a letter size sheet of paper, a 24-inch diagonal image (about as big as fits on a standard “B” size sheet of paper, and a 72-inch diagonal image I project on a wall.   For reference I have also included a the same portion of the source 3x magnified.

As you should notice the 12-in diagonal image is completely blurry even at 1/2 the stated resolution.  With the 24-inch diagonal you can start to see some “modulation” of the single pixel size lines horizontally but not vertically.  With the 72-inch diagonal the horizontal lines are pretty clear but still the vertical lines are still pretty much a blur (on close visual inspection there is a little modulation of the single pixel wide lines).

What is happening is that size of the laser beams is larger than the pixel size for small images.  The size of the beam diverges but at a slower rate than the size of the image grows so eventually the laser beam size is smaller than a “pixel” and you start to see separation between horizontal 1 pixel wide lines.

As for the horizontal resolution, whatever is driving the lasers in their horizontal sweep is not able to fully modulate them at single pixel resolution.

For the next set of 3 images (plus a 2x Magnified source) I have scale the images down so you can see more area.  Note you need to click on the image to see it at its intended size and to see the detail.  In these pictures you can see the ruler with both indicates the size of the image and shows that the camera was in-focus and could see the detail if it was in the projected image.

On the 24-inch diagonal and 72-in diagonal image I have drawn 3 ovals.  The left oval is around a set of 4 line pairs (see source image) of horizontal and vertical lines.   The middle and right ovals are each around 4 line pairs of vertical lines and two sets of 4 pairs of horizontal lines and where the horizontal and vertical lines cross is a set of 9 white pixels (never visible in any of the projected images).

Looking at the 72-inch image you may notice that you can barely make out the horizontal line pairs in the center oval but that they become blurry in the right oval.  This is due to the interlaced Lissajous scanning being done (for more detail on the Microvision interlaced scanning process see: http://www.kguttag.com/2012/01/09/cynics-guild-to-ces-measuring-resolution/).  The net effect of this scanning process is that vertical resolution is reduce from the center to the left and right sides.

Image Size Comparison

The 5 year old Microvision ShowWX having this blurring issue with small images.  In looking inside at the optics with the lasers on, I notice that the laser spot sizes were larger than expected.  I’m left wondering if the larger laser spot sizes were at least in part cause by efforts to reduce speckle or for some other reason.

Next time, I plan on giving a little “tour” of the optics.

Addendum – How the pictures were taken, full resolution images, and source pattern used

All the pictures were taken with a Canon 70D (5472 by 3648 pixel) DSLR.  By framing the pictures so that filled roughly 90% of the width, this meant there were roughly 4 camera pixel “samples” per pixel in the output image.   The ruler in the picture was both to keep track of the size of the image and to make sure the camera was in-focus and could resolve single pixels (if they were there).

I did selectively zoom in with the camera on smaller regions to see if it made any measurable difference in resolving features in the images and it did not.  I have included the test pattern I used and would welcome anyone using it to verify what I have shown.

By clicking on the thumbnails below you will bring up the full size image (depending on your browser it may not display full size until after you click on the magnifying glass).  You can then right click to download the images.   Each image is about 8 to 9 Megabytes and is stored in a high quality (low compression) JPG format.   The source test pattern is stored in loss-less PNG.

12-inch Diag Celluon_8572

12-in Diagonal Celluon Image (20 megapixels-click to see full size image)

24-inch Diag Celluon_8452

24-in Diagonal Celluon Image (20 megapixels click to see full size image)

72-inch Diag Celluon_8205

72-in Diagonal Celluon Image (20 megapixels click to see full size image)

Basic res-chart-720P

Test Pattern Source (1280×720 pixels PNG format, click for full size image)

Celluon Laser Beam Scanning Projector Technical Analysis – Part 1

Celluon Light Path w800 IMG_8087The Celluon PicoPro projector has been out for a few months now for about $359.   I have read a number of so-called “reviews” that were very superficial and did little more than turn on the projector and run a few pictures and maybe make a video.   But I have not seen any serious technical analysis or review that really showed the resolution or measured anything beyond the lumens.   So I am going to be doing a multi-part technical analysis on this blog (there is just too much to cover in one article).

In the photo at the top, I took a picture with the lasers on to more clearly see the various light paths.  A surprise to many is that they used 5 lasers and not just three which adds to the cost and complexity of the design.   They use two red and green lasers to get to the spec’ed (and measured) brightness of 32 lumens.   In future articles, I will get into more details on the optical path and what is going on (there are a few “tricks” they are using).

It is no secret by now that the Celluon engine uses a beam scanning mirror from Microvision and the optical engine and electronics are from Sony (the engine looks identical to the one Sony Announced February 20, 2014) .  Below I have taken the cover off the electrical part so you can see some of the chips. If you look carefully at the red arrows in the picture below, you can see the 3 clearly identified Sony ASICs used in the driver board (the 4th large chip is a Samsung SDRAM and the smaller device is a Texas Instruments power supply chip — there are more power supply chips on the backside of the board).

Sony Devices IMG_9737

I have used test charts to measure the resolution, check the color control , and measured the power consumption.   I have also taken a look inside to see how it is made (per the pictures above).    I have collected data and many images so the biggest problem for me to boil  this down into a manageable form for presentation on this blog.   I decide to start with just a bit about the resolution and a summary of some other issues.

Celluon claims the resolution is “1920 x 720″ pixels and not that is not a typo on my part, they really claim to have “1920” horizontal resolution with as claimed by Sony in a press release on the engine.  It is easily provable that the horizontal resolution is much less than 1920 or even 1280 pixels and the vertical resolution is not up to fully resolving 720 lines.   In fact the effective/measurable resolution of the Celluon engine is closer to 640 by 360 pixels than it is to 1280×720.

PC Magazine’s April 22, 2015 article on the Celluon PicoPro made the oxymoron statement “the image has a slight soft-focus effect.”  To me “soft-focus” means blurry and indeed the image is in fact both blurry and lower in resolution.   The article also stated “I also saw some reddish tinges in dark gray areas in some images, a problem that also showed up in a black-and-white movie clip“.   The image is definitely “off to the red” (white point at about 4000K) and it has very poor color control in the darker areas of the gray-scale.

Resolution is a big topic and I have a lot of photos, but to get things started, below I have taken a center crop of 1280×720 HDMI input into the Cellulon projector.   Below this image I have included the same crop of the text pattern in put zoomed in by 2X for comparison.   In the photo you will see a yellow measuring tape that was flush against the projection screen, this both shows the size of the projected image AND proves that the camera was focused well and had enough resolution to show pixels in the projected image.

Celluon test pattern comparison

720P Celluon Projected Image with Source Below It with key comparison point indicated by the red ovals

You might want to look at the various areas indicated by the red ovals corresponding to the same areas of the projected image and the test pattern.  What you can see is that there is effectively no modulation/resolution of the sets of 1 pixel wide vertical lines so the horizontal resolution is below 1280 (more like about half 1280).

There is some modulation, but not as much as you should get if this were truly 720p, of the horizontal lines center of the of the image but this will fade out towards the left and right side of the projected image (I will get into this more in a future article).

You may also notices that the overall Celluon image is blurry.  Yes, I know lasers are supposed to “always be in focus,” but the image is definitely out of focus.   It turns out that at the size of this image (12 inches vertical or 24 inches diagonal which is moderately big, the width of the scanned laser beams are wider than a pixel and thus overlap.

The image is even more blurry if the image is say about 7-inches high projected on a standard letter size sheet of paper (the image is very blurry).  The blurriness goes down if the image gets bigger but it is NEVER really sharp even with a 72-inch diagonal image.   In a future article I will post the same test pattern at different image sizes to show the effects of image size and blurriness/focus.  I have started to call this “never in-focus technology.”

Some summary observations (more to come on these subjects):

  1. Laser Speckle – much improved over previous Microvision ShowWX projectors.   It still is far from perfect an most annoying where there are large flat areas and text on a bright background.
  2. The Celluon eliminated the “bowtie” effect of earlier Microvision ShowWX product so that the image is rectangular
  3. The lost the 100% offset of the ShowWX meaning that this requires a “stand” and the image will either be keystone or the projector will be between the viewers eye and the image.  This is bad/wrong for a short throw projector.  There is no keystone correction supported by the product.
  4. Low effective resolution – absolutely nowhere close to 720p (see above, more on this in future articles).
  5. Blurry image – not the same per se as resolution.  The size of the laser beam appears to be bigger than a pixel until the image is very large.  Additionally there are issues with aligning the 5 lasers into a single “beam” and issue with the interlaced bi-directional scan process (see http://www.kguttag.com/2012/01/09/cynics-guild-to-ces-measuring-resolution/ for more on the scan process and how it hurts resolution).
  6. Class 3R laser product – This is a very serious problem as it is not safe for use with children (in fact laser safety glasses are recommended) but it this is not well marked.  The labels on the product are ridiculously tiny (particularly the one on the projector itself).  The EU is reported in the process of banning consumer products that emit 3R laser light (http://www.laserpointersafety.com/news/news/other-news_files/tag-european-union.php and http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32014D0059)
  7. Flicker – this is a serious problem with this product and I will discuss more about this in a later article.  About 1 in 7 people I showed the projector to said it gave them headaches or other problems (I had multiple people tell me to turn it off as it was painful to even be in the room with it).  The scan process is 60-hertz “interlaced” with no persistence (as with an old CRT).
  8. The power consumption is high taking about 2.6W to show a totally back image and 6.1W for a totally white 32 lumen image with the power consumption in between roughly proportional to the image content. Don’t let the lack of fans fool you, they are using heat spreading over the entire package to dissipate the heat from just the projector.  The device will quickly overheat if left on a tabletop (as opposed to the fan) as much of the heat is spread over the bottom of the package.  It will also overheat if a bright image is left on the screen for too long even if the device is floating in air.
  9. The color/gray scale control is pretty poor particularly with the darker parts of a gray ramp.  At the dark end of the gray scale the “gray” turns red.  Additionally there is “crosstalk” caused from the lasers heating or cooling based on the brightness on one part of the screen that affects the color/brightness on the other side of the screen.   In other words the content of the image in one area will affect the color in another area (particularly horizontally).

I have seen Microvision laser scanned projectors since the Microvision ShowWX came out in 2010 or 5 years ago and the Celluon unit has many of the same issues that I found with the ShowWX.  While the Celluon is much improved in terms of brightness and speckle, has better resolution (but not as near what is claimed) and it delivers about 3X the brightness for the about the same power (much of this is due to laser improvements over the last 5 years) the progress is very modest considering that 5 years have passed.

Frankly, I still consider this technology far from ready for “prime time” high volume and sill has some major and in many ways fatal flaws.  Being laser safety class 3R at only 32 lumens is chief among them.  The flicker I also consider to be a fatal problem for a consumer product but this perhaps could be solved by going to a higher refresh rate (which would require a much faster scanning mirror).   The power consumption is far too high for embedding into small portable products.

And then we come back to the issues with the “use model” that still exists with Pico Projectors (see my discussion from way back in 2011 about this).

On a final note, I know that Laser Beam Scanning has a very dedicated following with some people that vigorously defend it.   I will be providing test patterns and other information so people can duplicate my experiments and verify my results.   I am more than happy to discuss the technology and respond to dissenting opinions, but I won’t tolerate rude comments or personal attack in the discussion.

Addendum — Test Patterns

Below are some test patterns stored in lossless PNG format to try out on the Celluon or other 720p projector to see for yourself.

Right-Click on the given pattern download the original full size pattern. Note, they should be view at “100%” if not on a 720p monitor and should totally fill the screen on 720p projector.

The first one below is a resolution test with 9 “zone patterns” has well as sets of 1 pixel wide black and white horizontal and vertical lines.

interlace res-chart-720P G100A

 

Simple horizontal gray ramp.  This is totally neutral gray from 0 to 255.Horz 0 to 255 gray ramp

Below may look dark gray or even black but it a totally flat R=B=G=16 everyone (a flay gray of 16/255).   See how it looks on the Celluon.

gray 16

 

HMD – A huge number of options

HMD montageThere have been a number of comments on this blog that I am very negative about Head Mounted Displays (HMDs), but I believe I am being realistic about the issues with HMDs.   I’m an engineer by training and profession and highly analytical.  Part of building successful products is to understand the issues that must be solved for the product to work.  I have seen a lot of “demo ware” through the years that demo’s great but fails to catch on in the market.

It’s rather funny the vitriolic response from some of the people posting in the comments (some of which are so foul they have been blocked).  Why do they feel so threaten by bringing up issues with HMD?   There must have been over 100 HMDs go to market, some of these with big name companies behind them, over the last 40 years and none of them have succeed in making the break through to a consumer product.   Clearly the problem is harder than people think and it is not from a lack of trying by smart people.  Why is the “101st” attempt going to succeed?

I know there is a lot of marketing and hype going on today with HMDs, I’m trying to cut through that hype to see what is really going on and what the results will be.    I also have seen a lot of group think/chasing each other where a number of companies are researching the same general technology and the panic to get to market after another company has made a big announcement.  Many feel this is going on now with HMD in response to Google Glass.

Designers of HMDs have a huge number of design decision to make and invariably each of these choices result in pro’s and con’s.   Invariably they have to make trade-offs that tend to make the HMD good for some applications and worse for others.    For example, making a display see-through may be a requirement for augmented reality, but it makes them more expensive and worse for watching moves or pictures.    For immersive Virtual Reality the design may want a wide field of view (FOV) optics which for a given display resolution and cost means that you will have low angular resolution making it bad for information displays.

To begin with I would like to outline just some of the basic display modality options:

  1. See-through – Usually for augmented reality.  It has the drawback that it is poor for watching movies, pictures, and seeing detail content because whatever is visible in the real world becomes the “black.”   The optics tend to cost more and end up trading image quality for the ability to see through.   Also while they may be see-through, they invariably have to affect the view of the real world.
  2. Monocular (one-eye) – A bit harder for people to get used to but generally less expensive and easier to adjust.   People usually have one “dominant eye” and/or good eye where the one display should be located.   A non-see-through monocular can provide a bit of a see-through effect, but generally the display dominates.   Monocular HMDs support much more flexible mounting/support options as they don’t have to be precisely located in front center of the eyes.
  3. Binoculars (both eyes) – Generally supports better image quality than monocular. Can more than double the cost and power consumption of the display system (two of most everything plus getting them to work together).  Can support 3-D stereoscopic vision.   The two displays have to be centered properly for both eyes or will cause problems seeing the image and/or eye strain.  More likely to cause disorientation and other ill effects.
  4. Centered Vertically – While perhaps the obvious location, it means that the display will tend to dominate (or in the case of non-see through totally block) the user’s vision of the real world.   Every near eye display technology to at least some extent negatively affect the view of the real world; even see-though displays will tend to darken and/or color change, and/or distort the view.
  5. Above and Below – Usually monocular displays are located above or below the eye so that they don’t impair forward vision when the user looks straight forward.  This is not optimal for extensive use and can cause eye strain.   Generally the above and below position are better for “data snacking” rather than long term use.

Within the above there are many variations and options.   For example, with a see through display you could add sunglasses to darken or totally block the outside light either mechanically or with electronic shutters (which have their own issues), but they will still not be as optimal as a purpose built non-see through display.

Then we have a huge number of issues and choices beyond the display modality that all tend to interact with each other:

  1. Cost – Always an issue and trade-off
  2. Size and Weight – A big issue forHMDs as theyare worn on the head.  There are also issues with how the weightis distributed from front to back and side to side
    1. Weight on the person’s nose – I call this out because it is a particularly problem, any significant weight on the nose will build up and feel worse over time (anyone that has had glasses with glass rather than plastic lenses can tell you).    Therefore there is generally a lot of effort to minimize the weight on the person nose by distributing the force elsewhere, but this generally makes the device more bulky and has issues with messing up the user’s hair.   The nose bridge when use is generally used to center and stabilize the HMD.    Complicating this even more is the wide variety of shapes of the human head and specifically the nose.   And don’t kid yourself thinking that light guides will solve everything, they tend to be heavy as well.
  3. Resolution – Obviously more is better, but it comes at a cost both for the display and optics.  Higher resolution also tends to make everything bigger and take more power.
  4. Field of View (FOV)  – A wider FOV is more immersive and supports more information, but to support a wide FOV with good angular resolution throughout and support high acuity would require an extremely high resolution display with extremely good optics which would be extremely expensive even it possible.   So generally a display either as a wide FOV with low angular resolution or a narrower FOV with higher angular resolution.  Immersive game like application generally chose wider FOV while more informational based displays go with a narrower FOV.
  5. Exit Pupil Size – Basically this means how big the sweet spot is for viewing the image in the optics.  If you have every used an HMD or binoculars you will notice how you have to get them centered right or you will only see part of the image with dark ring around the outside.  As the FOV and Eye relief increase it becomes more and more difficult and expensive to support a reasonable exit pupil.
  6. Vision Blocking (particularly peripheral vision) ­– This can be a serious safety consideration for something you think would wear by walking and/or driving.  All these devices to a greater or less or extend block vision even if the display itself it off.    Light guide type displays are not a panacea in this respect either.  While light guide displays block less in front of the user, they have the image coming in from the sides and end up blocking a significant amount of a person’s side peripheral vision which is use to visually sense things coming toward the person.
  7. Distortion and Light Blocking – Any see-through device by necessity will affect the light coming from the real world.   There has to be a optical surface to “kick” the light toward the eye and then light from the real world has to go through that same surface and is affected.
  8. Eye relieve and use with Glasses ­ – This is an issue of how far the last optical element is away from the eye.   This is made very complicated by the fact that some people wear glasses and that faces have very different shapes.   This is mostly an issue for monocular displays where they often use a “boom” to hold the display optics.    As you want more eye relief, the optics have to get bigger for the same FOV, which means more weight which in turn makes support more of a problem.   This was an issue with Google Glass as they “cheated” by having very little eye relief (to the point that they said they were not meant to be used with glasses
  9. Vision correction – Always and issue as many people don’t have perfect vision and generally the HMD optics want to be in the same place as a person’s glasses.  Moving the HDM optics further away to support glasses makes them bigger and more expensive.   Building corrective lenses in to the HMD itself will have a huge impact on cost (you have to have another set of prescription lenses that a specially fit into the optics).   Some designs have included diopter/focus adjustment some many people also have astigmatism.
  10. Adjustment/Fit – This can be a big can of worms as the more adjustable the device is the better it can be made to fit, but then the more complex it gets to fit is properly.  With binocular displays you then have to adjust/fit both eye which may need moving optics.      
  11. Battery life (and weight) – Obvious issue and they are made worse dual displays.  At some point the battery has to be move either to the back of the head (hope you don’t have a lot of hair back there) or via a cable to someplace other than the head.
  12. Connection/cabling – Everyone wants wireless, but then this means severe compromises in terms of power, weight, support on the head, processing power (heat, battery power, and size).
  13. How it is mounted (head bands, over the head straps, face goggles) – As soon as you start putting much stuff on the head a simple over the ears with a noise bridge is not going to feel comfortable and you start to have to look to other ways to support the weight and hold it steady.   You end up with a lot of bad alternatives that will at a minimum mess with people’s hair.
  14. Appearance ­– The more youtry and do on the head, bigger and bulkier and uglier it is going to get.
    1. Look of the eyes – I break this out separately because human’s are particularly sensitive to how people’s eye’s look.  Many of the HMD displays make the eyes look particularly strange with optical elements right in front of the eyes (see below).  Epson eyes
  15. Storage/fragility – A big issue if this is going to be a product you wear when you go out.   Unlike you cell phone that you can slip in your pocket, HMD don’t generally fold up into a very small form factor/footprint and they are generally too fragile to put in your pock even if they (and with all their straps and cables they may have) would fit.
  16. Input – A very big topic I will save for another day.

 If you take the basic display types with all the permutations and combinations of all the other issues above (and the list is certainly not exhaustive) you get a mind boggling number of different configurations and over the last 40 years almost every one of these has been tried to a greater or lesser extent.  And guess what, none of these have succeeded in the consumer market.   Almost every time you try and improve one of the characteristics above, you hurt another.    Some devices have found industrial and military used (it helps if there is not a lot of hair on the user’s head, they are strong enough to carry some weight on their head, they don’t care what they look like and they are ordered to do the training).

In future posts, I plan on going into more detail on some of the options above.

One last thing on the comment section; I’m happy to let go through comments that disagree with me as it helps both me and others understand the subject.  But I am not going to put up with foul language and personal attacks and will quickly hit the “trash” button so don’t waste your time.   I put this under; if you can’t argue the facts then you trash the person category and I will file it accordingly. 

Desperately Seeking the Next Big Thing – Head Mounted Displays (HMDs) — Part 1

Untitled-2With Microsoft’s big announcement of HoloLens and spending a reported $150 million just for HMD IP from the small Osterhout Design Group, reports of Facebook spending about $2 billion for Oculus Rift, and the mega publicity surrounding Google Glass and the hundreds of millions they have spent, Head Mounted Displays (HMD) are certainly making big news these days.

Most of the articles I have seen pretty much just parrot the company press releases and hype up these as being the next big thing.   Many of the articles have, to say the least, dubious technical content and at worst give misinformation.   My goal is to analyze the technology and much of what I am seeing and hearing does not add up.

The question is whether these are lab experiments with big budgets and companies jumping the gun that are chasing each other or whether HMDs really are going to be big in terms of everyone using them?    Or are the companies just running scared that they might miss the next big thing after cell phones and tablets.   Will they reach numbers rivaling cell phone (or at least a significant fraction)?    Or perhaps is there a “consolation prize market” which for HMDs would be to take significant share of the game market?

Let me get this out-of-the-way:  Yes, I know there is a lot of big money and smart people working on the problem.   The question is whether the problem is bigger than what is solvable?  I know I will hear from all the people with 20/20 hindsight all the successful analogies (often citing Apple) but for every success there many more that failed to catch on in a big way or had minor success and then dived.   As an example consider the investment in artificial intelligence (AI) and related computing in the 1980’s and the Intel iAPX 432 (once upon a time Intel was betting the farm on the 432 to be replacement for the 8086 until the IBM PC took off).    More recently and more directly related, 3-D TV has largely failed.  My point here is that big companies and lots of smart people make the wrong call on future markets all the time; sometimes the problems is bigger than all the smart people and money can solve.

Let me be clear, I am not talking about HMDs used in niche/dedicated markets.  I definitely see uses for HMDs applications where hands-free use is a definite.  A classic example is military applications where a soldier has to keep his hands free, is already wearing a helmet that messes up their hair and they don’t care what they look like, and they spend many hours in training.   There are also uses for HMD in the medical field for doctors as a visual aid and for helping people with impaired vision.  What I am talking about is whether we are on the verge of mass adoption.

Pardon me for being a bit skeptical, but on the technical side I still see some tremendous obstacles to HMD.    As I pointed out on this blog soon after Google Glass was announced http://www.kguttag.com/2012/03/03/augmented-reality-head-mounted-displays-part-1-real-or-not/ HMDs have a very long history of not living up to expectations.

I personally started working on a HMD in 1998 and learned about many of the issues and problems associated with them.    There are the obvious measurable issues like size, weight, fit/comfort and can you wear them with your glasses, display resolution, brightness, ruggedness, storage, and battery life.   Then there are what I call the “social issues” like how geeky it looks, does it mess up a person’s hair, and taking video (a particularly hot topic with Google Glass).   But perhaps the most insidious problems are what I lump into the “user interface” category which include input/control, distraction/safety, nausea/disorientation, and what I loosely refer to “as it just doesn’t work right.”   These issues only just touch on what I sometime joking refer to as “the 101 problems with HMDs.”

A lot is made of the display device itself, be it a transmissive LCD, liquid crystal on silicon (LCOS), OLED, or TI’s DLP.    I have about 16 years of history working on display devices, particularly LCOS, and I know the pro’s and con’s on each one in some detail.   But as it turns out, the display device and its performance is among the least of the issues with HMDs, I had a very good LCOS device way back in 1998.   As with icebergs, the biggest problems are the ones below the surface.

This first article is just to set up the series.  My plan is to go into the various aspects and issue with HMDs trying to be as objective as I can with a bit of technical analysis.    My next article will be on the subject of “One eye, two eyes, transparent or not.”

I’m Back

Hi everyone,

Just a quick note today to say that I no longer at Navdy and I now have some time to get back to posting on this blog.   All the travel and work at a start-up was pretty all consuming.

I have some idea for some topics particularly related to the various head mounted display goings on from Google “Glass” to Microsoft’s recent “HoloLens.”   I also want to write more on the computer and game system history.   I like answering questions and would like suggestions for topics so please feel free to write.

The one thing I ask is that there be no questions directly related to Navdy as it would not be appropriate for me to answer them.   It is not going to make good reading for me to have to repeatedly reply with “no comment” or the like.   Almost everything else related to technology, projection, head mounted displays, lasers, computer/video-game history is fair game.

Oh yes one more thing, I am available again for consulting work.

Karl

Navdy Launches Pre-Sale Campaign Today

Bring Jet-Fighter Tech to Your Car with NavdyIts LAUNCH Day for Navdy as our presale campaign starts today. You can go to the  Navdy site to see the video.  It was a little over a year ago that Doug Simpson contacted me via this blog asking about how to make a aftermarket heads up display (HUD) for automobiels.     We went through an incubator program called Highway1 sponsored by PCH International that I discussed in my last blog entry.

The picture above is a “fancy marketing image” that tries to simulate what the eye sees (which is impossible to do with a camera as it turns out).   We figures out how to do some pretty interesting stuff and the optics works better than I thought was possible when we started.    The image image focuses beyond the “combiner/lens” to help with the driver seeing the images in the far vision is about 40 times brighter (for use in bright sunlight) than an iPhone while being very efficient.

Navdy Office

Being CTO at a new start-up has kept me away from this blog (a start-up is very time consuming).  We have raise some significant initial venture capital to get the program off the ground and the pre-sale campaign takes it to the next level to get products to market.  In the early days it was just me and Doug but now we have about a dozen people and growing.

Karl

Highway1 Incubator

Those that follow my blog are probably wondering what has happened to me these past months.   I have away from home for most of the last 4 months at an “incubator” program for start-ups called Highway1.   Navdy, for which I recently became CTO, was selected as one of 11 companies from over 100 applicants for the very first class of the Highway1  program sponsored by PCH International.

What makes Highway1 different from almost all other incubator programs these days is that it is totally focused on helping hardware start-ups.   Highway1 recognizes that hardware start-ups have special needs, are more difficult to get started, and have have to deliver a physical product unlike software companies.

The Highway1 office is in the Mission District of San Francisco where most of the time is spent, but the program also includes spending two weeks in Shenzhen China where many of the electronic products used around the world are made.   During the program companies are introduced to mentors from other companies and experts in the field as well as helped with introductions to angle and venture investment firms.

While in Shenzhen, the companies were introduced to manufacturers who could eventually be making their products.   Additionally our company received some very crucial support from PCH in Shenzhen in locating a company that could manufacture a critical component of our system.

Along the way, the people at the various 11 companies became friends and helped each other out.  Respecting each other was particularly important as the companies were cranking out prototypes sharing first on one and later two 3-D printers for making prototypes (as demo day neared, the those 3-D printers were pretty much running non-stop).   There was some incredible talent  technically, marketing, and business wise at these companies.

At the end of the program was “Demo Day” where more than 200 venture capitalists, investors, press, and technologist pack a large room at PCH’s U.S. Headquarters in San Francisco.  It was a chance for investors and the press to see what the companies had developed.   While Navdy presented, details of our product and plans were not released to the press because we are planning on launching our product later this year.  Navdy did receive serious interest from a number of VC’s with our demo after the formal presentations.

The whole Highway1 program was the dream of Liam Casey the founder and CEO of PCH, a company with over $700M in revenue.  You may not know the PCH name, but it is very likely that you have brand name products that they helped get to your home or office (be it anywhere in the world).   Liam was personally there to greet us at the beginning of the program and at key points along the way, and he told some great business stories.  The whole of the PCH team, be it the people from San Francisco, China, or Ireland, were always awesome to work with and incredibly nice reflecting PCH’s founder.

Comment: I don’t usually use the word “awesome” but the word was ubiquitous in San Francisco and it seemed to fit the people at PCH.