ATLANTIS: Designing living spaces with Augmented and Diminished Reality

Mr. Thallinger can you please give us a brief overview of where the ATLANTIS project currently stands?

Currently we have completed 9 months of the planned 20 months project duration. At the beginning there was a very comprehensive phase of collecting user requirements by our Swedish partner "Usability Partners". For this purpose we used a questionnaire on the one hand, but also conducted individual interviews with potential users. Our results are already summarized in a small series of reports on our website (under the menu item "Results").

Who are the potential users?

The possible users actually include anyone who wants to redecorate their living room - or any other room - or add something new. Subsequently, our tool is also interesting for furnishing consultants, for example in furniture stores. If you think even further, interior designers could also use the ATLANTIS tool. So this goes from very simple customers who are not particularly technology-savvy to people with a professional background from the direction of furnishing and design.

What have your interviews and surveys revealed: Is there a lot of interest?

Quite! There are also many wishes as to what a tool like this should be able to do. We can already cover part of them with our project. The other part gives a concrete knowledge about the direction in which we should then go further.

Please tell us more about the planned authoring tool!

The tool will probably be used as an app, primarily on tablets or smartphones. The idea is that the user can take a picture of the room - either as a detail or as a 360-degree panoramic shot, depending on the device. Afterwards, he or she can simply try out different equipment variants.

For example, furniture can simply be selected from a catalog, placed in the image - i.e. in the room - via augmented reality and viewed from different angles. A designer could take a picture of a room, then design different variants in his office and discuss them with his customer.

What if in the place where the piece of furniture is to be placed, there is already something else?

Then Diminished Reality comes into play. It will probably very often be the case that "space" must first be created. For this purpose it will be possible to recognize objects, remove them from the room and replace them with a plausible background - in front of which the new object can then be placed.

To make this possible, we work with Instance Segmentation. This is because the tool must first recognize that a table is a table - and what all belongs to this table.

What is the challenge here?

These are AI- or Deep Learning-based methods that always go through a training process. The tool must therefore "learn" which object classes exist. If you haven't trained lamps, for example - then you can't recognize, segment and remove them. So that's an important task that we're working on.

The other task is to increase the speed of these processes. Because if I'm in a room and want to try out different things, then I have to be able to move around. But that also means that the view of the objects is constantly changing - and the segmentation has to be constantly renewed in real time as the device moves. That's still a challenge.

What are the next steps in the project now?

In parallel to collecting the requirements and wishes of the potential users, we have already made the first experiments with instance segmentation. And also first steps towards inpainting, i.e. replacing objects.

As a project partner, Roomle has already begun to work more concretely on the tool and, for example, to create the first wire frames. Of course, this also involves usability issues: What does the tool look like? How can you do everything with it? How is the user interaction implemented?

What interests or excites you personally in particular about ATLANTIS?

For me, the innovative technologies and their concrete application are very exciting. Depending on the application, other needs and challenges arise. And it's interesting to think about how you can solve certain problems as well as possible in this or that case.

Thank you for the interview!