An Event Apart: The New Design Material

by October 29, 2019

In his The New Design Material presentation at An Event Apart in Denver, Josh Clark outlined how designers can integrate Machine Learning and other new technologies into their product designs. Here's my notes from his talk:

  • Designers and front-end developers have a role to play in Machine Learning and new technologies overall. But how?
  • Sometimes we get fascinated with the making of the product instead of enabling the service of the product (the end user experience). We sometimes care more about using the latest frameworks or technologies than making meaningful experiences.
  • The last decade of digital design was shaped by mobile, the next one is already being shaped by machine learning. Machine learning is our new design material, how can/should it be used?
  • When you encounter a new design material, ask: what can it do? how does it change us (both makers and society)?
  • How can machine learning help us? If we could detect patterns in anything, how can we act on them? Recommendation (ranking results that match a context); Prediction (most likely result); Classification (grouping items into defined categorization); Clustering (discover patterns/categories based on item attributes); Generation (machines can make something).
  • Get comfortable with casual (almost mundane uses of machine learning) uses of machine learning. We can add a little intelligence to many of our products using these techniques.
  • While there are some early attempts at using machine learning to create Web designers, machines are really best at time-consuming, repetitive, detail-oriented, error-prone, and joyless tasks.
  • How can we let people do what they do best and let machines do what they do best. How do we amplify our potential with machines vs. trying to replace things that we can do? Machines can help us focus our time and judgement on what matters (via pattern matching and clustering).
  • What can machine learning amplify for us: be smarter with questions we already ask; ask entirely new kinds of questions; unlock new sources of data; surface invisible patterns.
  • The job of user experience designers and researchers is to point machine learning at problems worth solving.

Characteristics of Machine Learning

  • Machine learning is a different kind of design material. It has different characteristics we can learn.
  • Machines try to find patterns in what we do but we're unpredictable and do weird things, so sometimes the patterns machines find are weird. Yet these results can uncover new connections that would otherwise be invisible.
  • We need to design for failure and uncertainty because machine learning can find strange and sometimes incorrect results. This is different than designing for the happy path (typical design work), instead we need to design for uncertainty and cushion mistakes by setting the right expectations. Match language and manner to system ability.
  • It's better to be vague and correct than specific and incorrect. Machines focus on narrow domains and don't understand the complete world. It's not real intelligence but scaled "interns" or "infinite tem year olds".
  • Narrow problems don't have to be small problems. We can go deep on specific medical issue identification or identify patterns in climate change.
  • We don't always understand how machine learning works, the systems are opaque. To help people understand what signals are being used we can give people some feedback on what signals inform recommendations or clustering.
  • Because the logic is opaque, we need to signal our intention. Designers can help with adding clarity to our product designs. Make transparency a design principle.
  • Machine learning is probabilistic. Everything is a probability of correctness, not definitive. We can surface some of these confidence intervals to our end users. "I don't know" is better than a wrong answer.
  • Present information as signals, not as absolutes. Point people in a good direction so they can then apply their agency and insights to interesting insights.
  • What do we want form these systems? What does it require from us? Software has values embedded in it (from its makers). We don't want to be self-driven by technology, we want to make use of technology to amplify human potential.
  • We're inventing the future together. We need to do so intentionally.