Head-up Cardboard

a see-through 3D display made with a phone, inspired by Google Cardboard · 2014

Prototype 1 with a phone on top, showing a planet in its window — Prototype 1, with the phone on top

In VR you are cut off from the real world. My Solar System for Google Cardboard mixed in the real world through the phone's camera, but with a single camera the real world looks flat. We already see the real world well without a camera, so why not project 3D content straight into our view of it? Like Google Glass, but with a phone, and in stereo.

The idea is simple: Google Cardboard, but with the stereo images reflected into your view by a semi-transparent mirror. You see the real world with hologram-like 3D content floating in it.

Prototype 1 13 December 2014

A proof of concept, to confirm that a phone can work as a stereoscopic head-up display. You need:

a piece of cardboard,
two lenses (mine are from a ColorCross VR headset),
a thin semi-transparent material. I used a CD case: not perfect, but good enough for a proof of concept.

Make a box from the cardboard and tape, open on one side, with two holes for the lenses about 6 cm apart, centre to centre. The box's height is the distance from the lenses to the phone's screen, which depends on the focal length of the lenses: hold a lens above some text and measure the distance at which the text looks sharpest. For my lenses it is 4 cm.
Cut the CD case into the pieces shown.
Assemble the pieces over the lenses, with the window at 45°. (In the photo the lenses are in the wrong way around.)
Turn the box over, lay the phone screen-down on its open side, and look straight ahead through the window.

Testing prototype 1 14 December 2014

To test it, I made a 3D scene and a test image. Both are drawn upside down, to cancel the flip of the mirror.

Run the 3D test scene on your phone

A planet floating above a desk, seen through prototype 1 — The test scene of orbiting planets, seen through prototype 1

Or open the static test image full screen on your phone:

The test image floating above a desk, seen through prototype 1 — The test image seen through prototype 1: the time and the text appear at slightly different distances.

Findings 14 December 2014

It sort of works. With the orbiting objects in the middle 20% of the screen, aligned and focused correctly, I saw them in stereoscopic 3D while still seeing the real world.
The lenses are not ideal; other lenses or curved surfaces may work better.
The field of view is much narrower than in Google Cardboard, which makes it hard to find content when looking around.
The CD case is not reflective enough: you need a dark background to see the content. Its back surface also adds a fainter ghost image.
A light floor washes out the image, because light from below reflects off the phone into view. A dark casing at the bottom would fix this.
The image is a mirror image of the screen, so apps must flip it, and Google Cardboard apps do not work. A second mirror could fix this.
The phone is held differently from Google Cardboard, so apps must interpret its movement differently.
The objects have stereoscopic depth, but not focal depth: your eyes focus at the same distance for everything, so the objects do not blend into the real world properly.

Improvements and variations 17 December 2014

The main needs are a wider field of view, compatibility with Google Cardboard, and a better image.

Wider field of view

Eyes closer to the lens: hardly possible, because the lens must stay out of the direct line of sight.
Bigger lenses: finding the right size and shape will be a challenge.
No lens: a longer optical path lets the eyes focus without a lens, but narrows the field of view. A curved mirror works like a lens, but is hard to make.

Compatibility with Google Cardboard

Two mirrors, arranged like a periscope, cancel each other's flip. The bottom one, the combiner, is semi-transparent; the top one is fully reflective.

Better image

First-surface mirrors and very thin glass reduce the ghost image.

Four concepts

Concept 2 — 2: a periscope of a semi-transparent and a fully reflective mirror. Works with Google Cardboard apps.

Concept 3 — 3: concept 2 reversed: the phone is seen directly, and the real world through the mirrors, from above the normal line of sight. Works with Google Cardboard apps.

Concept 4 — 4: a curved semi-transparent mirror that doubles as the lens.

Mirrors for concept 2 20 December 2014

Made with Tatjana J. van Vark's vacuum coating machine:

A fully reflective first-surface mirror: thin glass coated with aluminium.
The combiner, a semi-transparent mirror: thin glass with a very thin layer of aluminium on one side, and an anti-reflective layer of magnesium fluoride on the other. The aim was 50% transmission; the result is 44%.

The vacuum coating machine with its cage — The vacuum coating machine, made by Tatjana J. van Vark

The vacuum coating machine's bell jar — The vacuum coating machine, made by Tatjana J. van Vark

The semi-transparent mirror — The combiner

The fully reflective mirror — The first-surface mirror

Testing the mirrors 21 December 2014

I tested the mirrors in a flimsy Lego model of concept 2. First impressions are good: they blend the real world and the 3D content very well.

After this, the project was put on hold while I worked on other things.