In his presentation at Breaking Development in Nashville TN, Josh Clark looked toward the future of devices and interaction design. Here's my notes from his talk on Beyond Mobile: Where No Geek Has Gone Before.
- Mobile devices are today's most personal computer. They give us superpowers through the sensors within them: GPS, camera, microphone, gyroscope, etc.
- People assume mobile is the companion experience but it is actually the primary experience. Mobile can do more than traditional PCs.
- When we think about sensors, we think about understanding our environment. But we should be focused on interpreting our environment or even changing it.
- Games and toys are often more forward-looking because they are free to play.
- Skinvaders is a game that overlays controls on top of people's faces using the front-facing sensors.
- Ikea allows you to scan pages of their catalog and position furniture in your house to see what it might look like in your apps.
- Layar was an augmented reality browser that pivoted into using print layouts as input. Just point your camera at a magazine and get digital information about it.
- Minimize input to maximize output. Make software design as smart as possible to minimize work on the part of users.
- Table Drum uses the microphone on a mobile device to turn sounds in the real world into sounds in the app.
- Don't force people to interact with machines. Design for sensors instead of the screen.
- Sometimes the best touch interface is no touch at all. Too much time on screens can be dangerous, we're distracted.
- Speech and 3D gestures are input types that are coming of age just like touch did a few years ago.
- All the ways we communicate with each other as humans are starting to be recognized by machines. What do we do with those inputs in an interface.
- HP's Envy has a touch screen, leap motion controller, keyboard, and trackpad. All these inputs can be used at any time in the applications.
- Anything that is trying to replace the mouse as a pointer is going to loose. Instead think about how controllers like Leap Motion can use three dimensions.
- The Leap motion controller can take input from physical objects and turn them into digital interfaces.
Designing for Sensors
- Services like Square put custom sensors in places like registers and smartphones to collect payment information.
- As we add sensors everywhere, we can collect information from anything. The cost of internet connections and sensors is dropping dramatically.
- Sensors turn anything into a controller.
- We've been spending the last few years making the digital world more physical. Today, more of the physical world is becoming digital. There's an interesting nexus here.
- People who are really serious about software should make their own hardware -Alan Kay. More companies are becoming these kinds of organizations.
- How do we make peace with all our devices when everything is connected?
- Remote control interfaces allow smart devices (like our phones) to control dumber devices like our TVs. The phone acts as a controller.
- Sensors, mirroring, and remote controls are our current ecosystem of devices. Migrating interfaces are next.
- Migrating interfaces allow us to move between devices and continue our tasks seamlessly. Example: take a call on your phone using car audio, then easily switch to phone handset.
- Displays can connect to devices that carry operating systems and apps for you.
- We can design the spaces between our devices. There's magic there.
- Gesture and speech create magic together. The issue is imagination not technical capabilities.
- Sifteo cubes are aware of each other and can transfer information and actions between each other.
- Passive interfaces just do their work without needing any interaction from us.
- We tend to think of technology as always getting smarter, bigger, more powerful. But we'll also have a lot of dumb devices around us.
- How do you design interfaces to reach all these devices? You need an API -that's your application. Its creative control in an uncertain world.
- Sensors, mirroring, remote control, passive and migrating interfaces create a cloud of social devices that are there when we need them.
- We need to think beyond single interfaces. They're just temporary containers for our services. Presentations deprecate.
Advice
- Push the use of sensors to minimize input and maximize output.
- Think social: how can we move things between the devices in our lives?
- Build an ecosystem for your content.
- We're all cloud developers now. Our content and behaviors need to converge in the cloud.
- Your application is the API. Think beyond a single presentation layer.
- New input methods are coming fast and furious, embrace them.
- The future is already here. We've got decades of material to work with right now.