In her Data Basics presentation at An Event Apart in Chicago, Laura Martini walked through common issues teams face when working with data and how to get around/work with them. Here's my notes from her talk:
- Today there's lots of data available to teams for making decisions but it can hard to know what to use and how.
- Data tools have gotten much better and more useful. Don't underestimate yourself, you can use these tools to learn.
- Google Analytics: The old way of looking at data is based on sessions are composed of page views and clicks with timestamps. The new way is looking at users with events. Events can be much more granular and cover more of people's behaviors than page views and clicks.
- Different data can be stored in different systems so it can be hard to get a complete picture of what is happening across platforms and experiences. Journey maps are one way to understand traffic between apps.
- You can do things with data that don't scale. Some visualizations can give you a sense of what is happening without being completely precise. Example: a quantified journey map can show you where to focus.
- Individual users can also be good data sources. Zooming in allows you to learn things you can't in aggregate. Tools like Fullstory replays exactly what people did on your Website. These kinds of human-centric sessions can be more engaging/convincing than aggregate measures.
- Data freshness changes how people use it in their workflows. Having real-time data or predictive tools allows you to monitor and adapt as insights come in.
- How do you know what questions to ask of your data? HEART framework: happiness, engagement, adoptions, retention, and task success. Start with your goals, decide what is an indicator of success of your goals, then instrument that.
- To decide which part of the customer journey to measure, start by laying it all out.
- There's a number of good go-to solutions for answering questions like: funnel analysis (shows you possible improvements) or focus on user groups and split them into a test & control (allows you to test predictions).
- The Sample Size Calculator gives you a way to determine what size audience you need for your tests.
- Quantitative data is a good tool for understanding what is happening but it won't tell you why. For that, you often need to turn to qualitative data (talking to people). You can ask people with in-context small surveys and similar techniques.
- Often the hardest part of using data is getting people on the same page and caring about the metrics. Try turning data insights into a shared activity, bet on results. Make it fun.
- Dashboards surface data people care about but you need to come together as a team to decide what is important. Having user-centric metrics in your dashboards shows you care about user behavior.
- Data can be used for good and bad. Proceed with caution when using data and be mindful where and how you collect it.