Augmented Reality

The next big step

Augmented reality is here and there are applications that deliver real benefits. But how do you typically set about proceeding with a new AR project? And what expertise is needed to gain a foothold in this field? Find the answers in this article.

After the PC, the Internet and the rapid spread of mobile platforms the sector is again at the beginning of the next era: that of augmented and mixed reality platforms. This opens completely new opportunities for applications but many of them are still beyond our imagination. It is not a question of whether these platforms will become relevant, just when.

An overview of xR

The topics of Mixed Reality (MR), Augmented Reality (AR) and Virtual Reality (VR) are developing rapidly. We hear about new devices, services and applications almost every month. But what do the terms mean? Virtual reality is relatively clearly defined - full immersion in another world using a headset where you are no longer aware of the real world. One disadvantage is this lack of reference to the real world. The user therefore does not move or can only move to a limited extent.

Augmented reality is the field that is currently being developed most actively and is where many companies see the most potential. It is defined as an extension of reality, usually by visual and acoustic means. However further senses could be added in the future, such as touch.

A simple kind of augmented reality is the superimposition of 2D data such as the distance from a free kick to the goal during a football game. For more enhanced use, the space is recognized and 3D objects with physical models are embedded in it. Augmented reality has the advantage that the user can augment his or her real world, move normally in that world and communicate with other people.

The definition of mixed reality is not currently quite so clear. Back in 1994, Paul Milgram already described MR as the entire spectrum between virtual reality and true reality. This means that a 2D superimposition on a mobile phone or VR glasses, showing an additional camera image of reality, is also mixed reality. Another definition describes mixed reality as a subgroup of augmented reality. This advanced extension of the real world with the 3D capabilities of virtual reality supplies an additional understanding of the actual space. Objects are placed in the real space in 3D and interact with this space. The Microsoft Hololens is a good example of this. The following explanations are based on this definition. When we say AR we always also include MR.

It is still very difficult for many people to imagine that VR is the next big development after PCs, the Internet and mobile technology. There was a similar reaction at the end of the 1990s, when the first mobile phones with cameras were launched. Only a few people understood the benefit of having good cameras in smartphones. But once you have experienced augmented reality it becomes clear: this attractive form of extending reality is not just a passing fad, but the harbinger of something greater.

So it is not really surprising that all the big companies now have a strong AR strategy - Microsoft with the Hololens and Windows 10, Google with the Tango Project, Apple with the upcoming iOS 11 and the ARKit framework and Facebook, who this spring defined AR as one of its three strategic pillars. But what does a service provider need to do to take those first steps into the world of virtual reality?

Creating added value with AR

When a new technology or platform is in its infancy, a company does not generally change its fundamental processes but instead improves them in specific areas. Entire processes are only rethought once greater maturity and breadth is reached. Companies also first consider where there might already be a good basis that can be used in order to gain some experience in their normal working environment. Because of this, AR is typically first used in fields such as:

  • anywhere there is already a good base of 3D data
  • or where value can be added through use in the real world.

For example 3D data usually already exists in the medical sector. Today CT and MR scans can easily be processed into 3D models. The industrial sector also works frequently with CAD/CAM applications and so has a good 3D basis. There is also an industrial trend in the form of BIM (Building Information Modeling) within the field of architecture.

There are many situations in the real world where value can be added. In industry, for example, employees come face to face with machines about which they have little additional information. Superimposing this in a context-sensitive manner creates real added value. According to information from Microsoft, 50 to 100 service technicians at Thyssenkrupp and Tetrapak have already been equipped with a Hololens. An expert can go on Skype at any time to dial in and see the maintenance work, and sees the same image as the employee and can speak to them and give them instructions within their field of view. The employees do not wear the glasses all the time but put them on selectively for processes where they require assistance. This already substantially accelerates the diagnosis and solution process, improves the quality of the work and therefore means that the use of AR has already paid off for the company.

Another exciting area of application for AR is in the field of training. The benefit over VR: all the members of a class see the same objects in 3D and can move around them freely and explain them to each another.

Innovation methodology is essential

Often it cannot be foreseen in advance where the added value can be achieved. That is why it is worth using innovation methods, for example working iteratively, in a similar way to design thinking.

In order to recognize how augmented reality can add value, it is worth applying innovation methods along similar lines to the design thought process.

Using this kind of methodology it can be determined fairly rapidly which ideas and use cases can actually be implemented and provide benefits, and which cannot. Only paths that are worthwhile are pursued. It is also recommended to create an interdisciplinary project team. This should comprise at least:

  • Experts from the field who know or can identify the current pain points, processes and data quality.
  • Experts from AR development and 3D engineering who can quickly determine feasibility and can also develop prototypes.

Experts from the areas of law, data protection, data, 3D modelling or user experience can also be added depending on requirements and the type of project.

The advantage of this procedure is that you take a very limited risk. You can stop at practically any point if it is shown that something is still missing. Or you can go directly into product development if the potential is proven.

This is not possible without 3D expertise

Expertise is absolutely essential in 3D development in order to implement solutions using AR. You need to know all about the concepts of how to work in 3D and how to create data with 3D or optimize existing data. The application logic for creating scenery and developing user interfaces for them is completely different to the development of web or mobile apps. A web front end or back end developer may be able to program in an AR environment. But without prior knowledge he or she will spend a lot of time acquiring the skills that are needed for this area. That is why we recommend working with experts, as they are the only ones who can ask the important questions quickly and cost-efficiently and really clarify feasibility.

An example: A traditional developer at a company evaluates the Hololens based on a simple app that cannot deal efficiently with 3D data. Because he wants to load a complex object and the app makes this impossible he considers the Hololens to be incapable of processing complex objects. He rejects taking further steps as result of this experience. However this judgement is incorrect, because the Hololens can indeed deal with the complexity of up to half a million polygons. This requires knowledge of modelling tools such as Blender, 3D Studio Max, AutoCAD and development frameworks like Unity 3D. A concrete example of its performance is the representation of brain scans at Inselspital Bern using the Hololens, which loads them onto the glasses on-demand.

AR has arrived

It is now possible to realize AR applications that were unimaginable only a short time ago. And it is possible to determine and justify the benefit of possible applications efficiently and cost-effectively with a few dozen days of work. This is thanks to major advances in both the hardware and software stacks and the stable development framework. Development is progressing rapidly, for example with the appearance of the Apple ARKit in the autumn. The consumer market will see some very good AR apps in the next six months. Ikea, for example, has already announced an app that allows you to place furniture in your living space at home and to measure your living space simply using your telephone.

In the enterprise applications sector more glasses will follow the Hololens onto the marketplace and will drive the market forwards. Hololens already offers very many functions and a high level of stability and so can be used to carry out its own feasibility studies. Anyone who is already preparing now for the new AR platforms will be acquiring knowledge to further develop his or her business, get ahead of competitors or even be able to offer new services or products in this future field.

Source: Swiss IT Reseller, no. 09, September 2017

Talk to the author

More stories

On this topic