16:10 | 16:50
Keywords defining the session:
- Data products
- Product management
Takeaway points of the session:
- Making a well-aligned multidisciplinary team is tricky. It is not just sitting a bunch of people from different teams at the same table. It requires a mutual understanding of others’ background, so you have to train your team to achieve that.
- There are a lot of nuts and bolts involved in a good design for a data product. Using a ML-based algorithm is not plug-and-play. You have to design a whole new experience taking into account your models.
What makes a data product successful? This question is apparently simple. Probably most of our assistants would propose using better models or using bigger and new data sources to train them. In our experience we found out that a great data science work is only halfway to launching a successful data product. A data product involves a lot of interesting and multidisciplinary questions to solve that are not faced in other kinds of digital products This talk explores the most important questions that we faced in the development and launch of data products like: Design a good expectation management system to handle model’s uncertainty and error. Did you try to put yourself in your users’ shoes? It’s hard to understand probabilities and uncertainty if you are not familiar with statistics. Then is it impossible to manage the user expectation? Are all kinds of errors the same to the users’ eyes? Create good feedback loops in your product to feed your model training. We all love data, our models certainly love data. But how to gather it in an honest and accurate way? Can you use more complex feedback than a thumbs up and down icon? Is there only one way to disagree with a model output? Can you gather this with a thumbs up and down icon? Are your users honest when they give feedback? How can you solve this? Manage privacy and trust concerns What’s privacy by design? Do we make enough to protect our users privacy? What can we do? How to propel a corporate cultural change to a more data-like mind and survive it Create data products requires a cultural change in all stratums of your company. Understanding the significance of data, its opportunities and challenges should be part of the basic knowledge. How to align your design and DS teams (they have to work together!) and improve their mutual understanding Try a simple experiment: ask to your design team and DS team what is a recommendation and see if they agree. We firmly believe a good product must be the result of a good alignment and understanding within a multidisciplinary team. Train your models in a more human-aware way using other objective functions. Error is not always the most important thing. Maybe you can introduce other constraints to your model to make a better UX Risk management related with model training and research inside a typical product development cycle. New data based models and business generation Design your UX to give your users more control about their data and the behaviour of your algorithms How to consider, predict and avoid undesirable side effects of your algorithms Creating good KPIs to measure and understand the behaviour of your products Although we based some of this ideas in our experience we are trying to create and expose a general framework that could be applicable to different industries and products. Because we learned some of this problems the hard way (maybe the best one) we also want to share examples of what you should never (or rarely) do.