Interview with a Product Team: Segmentation & Personalization by Ryan Connolly on October 21, 2020

Meet the Team

Recently, your team had the opportunity to interface directly with a customer about some issues they were having, can you give us more context?

The customer, the web services department of a university in Barcelona, wanted to use experiences for a small project they were starting. While reading the documentation to figure out how to do what they wanted to do, they had some questions that they addressed to CS (Customer Support).

Why was the customer’s ticket escalated to your attention?

José Jiménez, from CS, answered the questions (unfortunately “it can’t be done right now”) and opened two feature requests to see if we were thinking about that kind of solution. Both of them were totally aligned with the roadmap.

At what point did you decide to do customer interviews?

José told us that this customer was very easy going and they were using experiences a lot, so in the case that we wanted to have some feedback they would be keen to help. And you can’t say no to a customer who wants to help!

What was your approach?

The team worked to further define the features proposed, so we could ask them for feedback. But the most important thing was getting closer to understanding their needs, rather than validating the solution.

How is customer feedback different from user testing?

When we do user testing we are validating hypotheses. We have some goals we need to clarify and we create tasks to see if the users are able to perform those tasks easily. So, we imagine the problems and see if the users can solve them. And if they don’t we search for a solution. Besides, for now, we are testing just with internal users, so we don’t have the real customers’ voices.

With the Customer Feedback we face a real use case. More insights and more situations may arise. It’s very enriching to hear how real clients are using the product and what challenges they face. So they are already facing those problems, and once detected we can think about possible solutions.

Yes, with customer feedback you listen to customer needs (caused by their challenges in business), while testing gives you an overview of the problems they find while working with your UI. They say little about problems to solve.

Why did you include both a designer and a product manager in your approach?

Susana and I usually work together in the ideation and definition stages of every epic. This time we were working on a couple of new features proposed by this customer, so it made total sense that we interviewed the customer together. If not so, there is a lot of information to be transferred from one to another afterwards, and a lot of details would have been lost.

That’s right! It’s important to work from the beginning together. Our end goal is to create a good product that helps users to do what they need. Each of us takes care of one part. I completely trust Julia on the product's alignment with the market and the roadmap and she trusts me to be the voice of the user and create easy to use features.

Moreover, we make decisions all the time, so it’s healthy to hear diverse opinions that improve what we build. I think that by working together we build better products and it’s so much more fun.

What is different about getting customer feedback for on-prem vs SaaS?

I would say that the main thing about working on an on-premise product is the delay. You are always working on the future, and you are struck by the fact that customers often deal with issues you left behind. That's a disadvantage of on-prem products, the feedback usually takes a long time to arrive.

Yes, absolutely agree, there's also a hard layer to overcome when talking about customer feedback for on-prem products. It’s difficult to get their consent, build the relationship, and share information.

It’s not uncommon to hear “there’s not enough time to talk to customers” — how would you respond?

It is not a matter of time, it is a matter of process. It is true that you need to distribute your time differently and the initial stages of ideation will be a bit longer, but that time is saved in the development stage. Which usually involves more money and wasted developer effort and frustration if we don't know if what we are building will be used and useful.

In fact, talking to customers saves time. Would you build a house without talking to the owners to learn their habits and needs? Or would you wait until the house is built and in use to demolish it and start over?

How did you prepare and execute?

We prepared a script for an informal interview having clear what we wanted to know. But at the end we didn’t use the script because the customers started sharing with us their context and their use case in an informal chat. So there was no reason to change the format and we let the conversation flow. Of course, we asked questions when needed to clarify or deepen in the matter.

Was there anything you intentionally avoided?

As we understood that we had missed their use case until that moment, we tried to avoid talking about the solution we had thought about - but it was too late, they asked about it anyway.

Can you share specific examples or insights you received that you wouldn’t otherwise have?

We were totally surprised that customers were using experiences in a completely different way that we were thinking. We could then understand the problem they were facing.

Yes, and listening to them we could better understand the context of their projects: the high staff turnover, the fragmentation of projects and stakeholders, the difficulty of establishing publication processes, for example.

Is there anything you would change about your approach?

Next time we will record the session. There are always more insights to extract from the interview and the chance to review is invaluable.