Next up in interaction

A new concept to navigate the interfaces of tomorrow

We’ve seen Google working on the Glass project for a few years, and with the iPod father Tony Fadell rowing the boat I’m convinced exciting things are on the horizon. Recently Microsoft engaged the whole world with their HoloLens project. The next paradigm of consumer technology is right around the corner and it’s mostly about seamlessly moving the screen interface to our surroundings as holograms and projections.

I keep seeing videos of people wearing glasses, experiencing holograms and interacting with them in a Don Quijote-esque fashion, waving hands in the air and struggling to feel something. I am a believer this technology, but I think we need other means of input to communicate with it.

An interesting concept have been taking form in my sketchbook and I would like to invite you into my take on the next iteration of mobile consumer technology. Remember that this is mostly about asking questions.

Every surface needs to be a trackpad

In ten years I don’t want to own a laptop, a phone, a tablet and a smart watch. I want one device – one device to rule them all, without making sacrifices. I want it to be wearable, seamless and beautiful. The device should adapt to us and our environment, not the other way around.

Watches missed, by an inch

If we are anatomically alike, the surface of your hand between the thumb and index finger is not only more glanceable, but more comfortable to interact with than any surface of your arm. It’s just the right angle, all the time and it provides a pretty big flat surface for content.

Aside from the holograms on in our field of view. – If we were to place a flexible oled touch display on that surface, we would have a superiorly interactable display to both todays watches and phones. We know how to interact with a touch screen, so let’s skip along. How do we interact with holograms?

How do we feel a something floating in mid air? How do we touch air? How can we tap a hologram on the shoulder?

What about text input?

A room full of people dictating messages is not the future, and that’s a promise. But if we can’t rely on voices as our primary method of input and don’t have access to a physical keyboard – we are going to need a worthy substitute (At least until mind control gets here).

Swipe keyboard

In air and on any surface

I’ve come up with a Harry Potter “Swish-and-Flip”-esque keyboard. We swipe a path with out finger and software figures out what we are trying to say. Don’t worry, this does not limit itself to an air-bound solution. If a table is nearby, we could simply project the keyboard on it and swipe directly on the table, using it to support our hand. If we are just typing out something quickly while on the move, we could just swish and flip in the air in front of us. Today we can see this technology in third party keyboards for our phones. A device attached to our hand would also provide superior motion tracking, as we could avoid the image processing (that is state of the art today) and cut straight to the data emitted from our hand.

Remember T9 on your old phone? I recall being able to type one-handed with excellent precision in class without the phone ever leaving my pocket and revealing my shenanigans. That possibility left us with the arrival of touch screens (though it was a nice trade), but I can see this happening again with this technology. Imagine some subtle doodling with your finger on the table in front of you – Secretly ordereding pizza and no one suspected a thing.

[ Warning, Nerdy ]How does it work?
The underlining technology here is pretty simple. A Markov chain is used to predict words based the likelihood of what will follow the previous ones to help with the swipe deduction. Probabalistic area, combination of letters in that area and the derivative of the swipe-turn would help form words. For example; If we were to drag and turn our finger to the intermediate points of [y,u,h,j], [e,r,d,f] and [y,u,h,j] in that order; A few computations would deduce we are trying to write ‘hey’. If we then feed our Markov model ‘hey’, it would generate an increased probability of the following words being e.g. “- how - are - you?”.

Psst! I’m thinking my next post will be starring a theoretical walkthrough and an implementation of such a model, as it is a fitting project for a course I’m taking in “Information Theory for Complex Systems”. Stay tuned for that! Now Let’s get back to our glove.

The future is organic

Technology is becoming a physical part of us. Today there is a vast difference of how my parents and me interact with technology. To them a computer is a tool, to me it is an augmentation of my body.

If there is one adjective I see technology is shifting towards it is “organic“. Device technology has always been hard, metal and cold (with an exception of my old laptop that bursted into flames), and as the future takes form, I can see how the shiny metal slowly turns into organic, matte, soft materials. Adapting to– and augmenting us.

Thank you for peeking into my sketchbook

Remember that these are just my ideas, perspectives and opinions. All illustrations are my own.

If I provoked any thoughts that you wish to discuss further, or just want to say hello – please ping me on Twitter.

You are also most welcome to shoot me an email. Remember to try and keep it short!

Be Social & Stay Tuned

Copyright © 2015 Patrik Göthe

Patrik Goethe