Re-imagining Apps for Touch

by August 9, 2012

I just kicked off a new video series for Intel Software Partners on re-imaging desktop application design. Many of the capabilities (touch, location detection, sensors) that got me excited and focused on mobile over the past several years are now making their way to new devices and creating different ways of thinking about and designing desktop software.

The Re-imagining Apps for Ultrabook™ video series not only introduces these new opportunities, but also offers practical design advice and guidelines to help you take advantage of them. The first video in the series looks at Touch Interfaces. Specifically, I outline the impact of new input methods in personal computing and walk through the top-level principles behind designing for touch.

Complete Transcript

Welcome to the re-imaging apps for ultrabook series with Luke Wroblewski.

Today we're going to talk about the opportunity touch provides for us to rethink our desktop applications and create new ways for people to interact with our software.

Specifically we'll look at the impact of touch and the opportunities it creates from both a business and software level. Then we'll walk through the principles of designing for touch and how our applications need to adapt to a world of direct manipulation and touch input.

Touch is a new input method that has been making its way into personal computing devices over the past few years. While that may not seem like a big deal on the surface, new forms of input have traditionally been very effective at shaking up the status quo in personal computing. That is when the mouse became part of the PC, when click wheel interactions became part of the mp3 player, and when multi-touch became part of the smartphone, disruptions in business models and software platforms quickly followed behind.

To illustrate this, let's look at the transition from keyboard-based input to mouse- based input in personal computers. You can see in the first 15 years of personal computing there were a number of companies trying to figure things out. The mouse wasn't a big player. The next generation of successful platforms x86 windows and apple were entirely based off a mouse and keyboard model. This change in input led to huge opportunity and new players emerged as leaders in the personal computing space.

Similar things happened in the mobile device market. Earlier generations of smartphones were predominantly controlled by keypads, scroll wheels, and trackpad interactions. When multi-touch interactions were introduced, we again saw a shifting of the guard. In this case, we’re looking at a chart of who is making mobile phone profits. And as you can see not only did new companies take over, once again new platforms arose.

From PCs to mobile devices, Each time a new input method was introduced, significant new opportunities emerged that allowed new players to take over. Being able to adapt to this change not only helps companies grow but it also allows them to better adapt their software to new consumer behaviors and expectations.

Companies that don't adapt risk being disrupted by new players that build with today's capabilities front and center. Consider the photo sharing service Instagram. While many desktop-based photo sharing services existed before instagram was released, the company built a uniquely mobile experience that capitalized the capabilities of mobile devices. The result was tremendous growth and a billion dollar acquisition from facebook shortly after.

Taking advantage of new forms of input isn’t just for consumer software. Here we're looking at an application called Hive that focuses on the sales in the enterprise. The way Hive works is a salesperson gets out of a meeting and moves to simple sliders with their thumb that indicate how far along a sales deal is and how big it could be. Looking at the analytics behind this experience, it turns out Hive is actually more accurate at detecting how sales forecasts will pan out. While at a cursory level, this simple touch-based interaction seems insignificant. In aggregate it works quite well.

Especially when you think about the sales rep that had to come home at the end of a day of sales calls, and start filling lots and lots of desktop based forms. Moving a slider or two with their thumb right after a meeting sure seems like a better overall user experience.

These touch-based interactions have traditionally been consigned to smartphone and tablet screens. But with the release of touch-enabled ultra books, touch is now becoming part of laptop screens as well. To better understand if people will use touch interactions on laptops, intel conducted a study that found touch was highly desirable for laptop users. In fact, many people moved to touchscreens for primary interactions instead of the mouse, keyboard, and trackpad. Participant's comments were very positive too. As one noted: having a laptop with touch is like having a laptop with an extra gear. Intel’s research illustrates people's desire for touch to go beyond the smartphone screen and make its way to all kinds of personal computing devices.

So what we see from these examples is that touch matters. Whether its the ability for new input models to drive new platforms and shift profits, or force companies to rethink consumer and enterprise software, touch is a big deal. Big enough that it's worth spending considerable time understanding how to design for touch. What can we do to Allow people to Interact with our software applications using touch?

Let's start at the top with design principles. One of the things that distinguishes touch from mouse, keyboard, or trackpad interactions is that its predominantly a direct manipulation interaction. That is you get to use your fingers to interact with the content you actually want to move, change, edit, or see more of. The content is the user interface. This is a significant change from traditional graphical user interface design. In our interfaces we need to strive to reduce the distance as much as possible between people and the content they want to interact with. That means reducing visuals that are not content: the windows icons menus, pointers of graphical user interfaces are a last resort for touch interfaces. These ui elements are often getting in the way of letting people directly interact with the content they want to use.

Let's look at an example. When someone is moving through a list of photos in apple's iPhoto application on the ipad, they don’t interact with controls. If they want to see what's in a pile of photos they just expand it. Move up, down, touch to view, push to side. In this entire series of interactions you never saw anyone interact with button, scrollbar or icon. Instead the content was the user interface. And that's the big difference. Touch is all about direct interactions with content.

The desktop applications we've be one used to follow the graphical user interface paradigm of using windows, icons, menus, and pointers to meet user needs and our business objectives. These elements don't account for the potential and constraints of touch. In fact, taken as they are they can be quite difficult to use with our fingers as we’ll see in the next video looking at touch target sizing.

We need a new paradigm to focus the design of touch interfaces. Instead of windows, icons, menus, Nd pointers for interactions. Think instead of content that can be directly manipulated with gestures that are reinforced through clear and meaningful feedback. At a high level this mode of thinking will help you get your desktop applications into a touch mindset. And your increasing number of touch-enabled users will thank you for it.

To quickly recap, we outlined the impact new forms of input can have on software and business. Touch is clearly one of these industry altering changes. It's mainstream adoption by consumers and increasingly enterprise users, makes it a key part of re- imaging desktop applications for the ultrabook platform. The first step toward this opportunity is understanding the principles that shape how to approach the design of touch-based software. but there's a lot more including accounting for touch target sizes and postures, understanding what's possible with touch gestures, and making sure touch interactions are discoverable. So stay tuned for the next videos in the re-imaging apps for ultrabook series where we dig into those topics and more.

I hope you'll join me then. Thanks.

More Soon...

Stay tuned for more video in the series coming soon...

Disclosure: I am a contracted vendor with Intel. Opinions expressed on this site are my own and do not necessarily represent Intel's position on any issue.