Augmented Reality data displayed directly onto the real world – AnatOnMe

Augmented reality is pretty cool; data superimposed on the real environment giving viewers enhanced perception of their surrounding environment. The downside is that most AR systems require some kind of a display device, on which the magic happens. In other words, viewers have to look at a computer screen, or a smart phone, to see the augmented reality.

Wouldn’t it be awesome if AR can be displayed directly onto the real world! No more HMD, smartphone or glasses getting in the way.

This is exactly what Microsoft Research demonstrated at this month’s ACM CHI Conference on Human Factors in Computer Systems.

AnatOnMe is a device that projects images of the bones, muscles, nerves and ligaments inside the body onto the patient’s own skin, giving them an enhanced understanding of the injury they sustained.It was designed to help patients who are undergoing physiotherapy to better understand their injuries, encouraging them to stick to the prescribed exercise regime and reduce the degree of non-compliance that contributes the high failure rate of this type of therapy.

The system comprises of a pico projector, an ordinary digital camera and a near-infrared camera. The infrared camera contains a laser pointer and the control buttons. The prototype is fairly basic. Instead of using a complicated geometry correction system to map an image onto the patient, the alignment of the projector is done by eye. Despite the low tech approach, the system works surprisingly well, giving patients a good understanding of their anatomy and injuries which they have sustained.

However, a small number of test subjects had difficulty manipulating themselves into the position required for effective viewing of certain images. In a genuine injury situation, this could prove to be quite a stumbling block. Therapists all found that the current system kept both of their hands occupied during a consultation, preventing them for touching and manipulating the patient’s injured area.

Despite some short comings, the researchers were encouraged by the positive reception of AnatOnMe. Currently, no development plans have been announced. But as AR technology matures with more sophisticated tracking and geometry alignment techniques, we may expect to see this type of AR being deployed not only in medical application, but in other application arenas as well, such as maintenance trainer.

If you haven’t seen it, check out the video Toyota Auris Hybrid: ‘Get Your Energy Back’ 3D Projection Mapping here. Through clever animation and 3D projection mapping techniques, the Toyota Auris appears to come to live in an orchestra of lights and sounds. Similar technology can be employed to project the inner workings of the car onto either the real vehicle, or even onto a polystyrene model.

For more information on the AnatOnMe, visit

About editor

Editor of