Doing research with artificial intelligence
Artificial intelligence (AI) does not replace us: it extends our capabilities. We explore how AI is automating analysis in order to free our time and make us more strategic.
It is impossible, today, to take part in a conversation about design or technology without the subject of artificial intelligence emerging. The background noise is considerable, and it oscillates between the promise of an efficiency never seen before and the fear that it will replace people. But what does AI really mean for those who, like our user experience agency, work on user research?
AI for user experience research, from the right point of view
AI can analyse thousands of pieces of feedback in a few minutes, transcribe interviews in real time and even simulate the behaviour of fictitious users. Silently, but quickly, it is redesigning our workflows.
But, despite all these changes, human understanding remains a fundamental element of user experience research. AI is changing how we do research, but not the why. Technology can provide data at scale, but empathy, intuition and the ability to catch a nuance in a pause during an interview remain, and will remain, profoundly human.
Let us look at what this transformation consists of, and how it is strengthening the work of user experience researchers.
Data, models and people: the new era of user experience research
This change marks the entry into a new phase for user experience research. If our work once consisted in meticulously collecting data and then spending weeks synthesising it, today AI promises to accelerate this process considerably.
AI excels at the analysis of large volumes of data (the what), freeing precious time that researchers can use to concentrate on context and on deep understanding (the why).
It is not a matter of saving time in order to do less, but of reinvesting that time in order to think more. It means finally having the space to connect the dots in a strategic way, to talk with stakeholders and to have a real impact on the product, instead of spending days reworking data.
Training with “digital twins”
What, in concrete terms, are the tools of this new era? One of the most fascinating (and debated) applications is simulations. So-called “digital twins” or synthetic users are generative AI models trained to imitate the behaviours, preferences and pain points of specific segments of users. We can interview them or ask them to carry out a task on a prototype, obtaining instant feedback.
It is very important, however, that AI simulations do not replace tests with real people.
This is because an AI model does not have a body, has not lived experiences, does not feel genuine frustration and does not possess that cultural and personal context that makes every person unique. It does not know what it means to have a bad day, to be interrupted by a notification, or to feel the anxiety of entering credit card details on an unfamiliar site. As NN/g puts it, an AI “doesn’t know what it is like to be human”.
What are AI simulations really for?
If they are used strategically, AI simulations help us explore and prepare the ground for research with real people. They allow us to:
- Generate research hypotheses — They are an excellent starting point for exploring an unknown domain or a new audience, allowing us to formulate initial hypotheses.
- Refine the research plan — Interviewing synthetic users can bring out questions we had not considered, helping us test and refine our guide for qualitative interviews with real people.
- Obtain feedback at embryonic stages — We can test mockups and ideas extremely quickly, in order to discover the most macroscopic problems and formulate better questions, before starting the costly process of recruitment.
- Explore “edge cases” — They can simulate limit scenarios or complex personas that are difficult to recruit in the real world.
We should think of digital twins as “mental training”, a way to accelerate the exploratory phase and arrive better prepared at the real and irreplaceable meeting with people.
AI tools for user experience research: opportunities and risks
If simulations are still a frontier being explored, AI tools for analysis are already an established reality. This is the transition we are living through now.
The market is flooded with solutions that promise to optimise the researcher’s workflow. These tools concentrate mainly on a few key areas:
- Transcription — Services that turn hours of interview audio into accurate text in a few minutes, often identifying the different speakers.
- Synthesis and thematic analysis — Platforms that read thousands of pieces of textual feedback (from surveys, app stores, social media) and group them automatically by theme, identifying recurring pain points and sentiment.
- Generation of partial insights — Some tools go further, attempting to generate “insights” ready for use from the raw data.
The efficient assistant and the risk of “noise”
AI becomes an indefatigable research assistant. Tasks that used to take days now take minutes. It is here, however, that the risk emerges. An algorithm does not possess critical thought.
AI can identify a correlation (for example, “people who click A also click B”), but it cannot explain causation (the why they do it). It can group the feedback that mentions “slowness”, but it will never catch the sarcasm in a comment such as: “Sure, your app is really fast…”.
Without human guidance, AI produces results that are of little use or that are misleading. Or worse, it can be actively frustrating.
A recent UXtweak study on the use of AI as an automatic moderator in usability tests showed that, if it is not guided, the AI ends up asking repetitive questions, plainly ignoring what the person has just said. The result is that the participant feels unheard, becomes irritated, and the quality of the whole test collapses.
Researchers must supervise, validate and, above all, filter. AI can aggregate, quantify data and find correlations, but only human analysis can interpret them, catching the meaning and the context that the machine does not see.
Augmented user experience researchers: the new professional profile
This brings us to the most significant change: the one that AI requires of us, as professionals. If AI takes care of the more mechanical parts of analysis, our role evolves. In our work, we do not limit ourselves to collecting data, but we also spend our time talking with the AI in order to guide it.
The new professional profile of the user experience researcher requires:
- Knowing how to ask the right questions (prompting) — The skill is no longer only knowing how to interview a person, but also knowing how to query the AI in order to obtain the most relevant data, avoiding bias and hallucinations of the model.
- Having a critical capacity — We must not accept the AI’s output passively. We are responsible for the final result. We must validate, challenge, question and integrate the AI’s output with our knowledge of the context.
- Amplifying one’s own curiosity — We can use the efficiency of AI to explore more deeply, to ask “What if…” questions that used to take too much time.
-
Integrating AI into the research process: three good practices
How, then, does one become an “augmented researcher”? It is not enough to buy software. The adoption of AI requires method, transparency and a critical approach. Here are three fundamental good practices that we have adopted, the fruit of our experience in the field.
Start from a real research problem
We do not use AI because “it is fashionable”. We integrate it when it serves to solve a specific problem. For example: “We have 2,000 pieces of feedback from customer care and we need to identify the three main problems by tomorrow.” In this case, AI for thematic analysis is an invaluable resource.
Always combine AI and human tests for validation
An effective approach is to use AI (such as simulations) for a first round of feedback, or to analyse quantitative data and generate hypotheses, which we will then validate with real people.
- Example: the AI analyses the navigation data and suggests that the checkout is confusing. This is an output.
- Action: we design a qualitative usability test with five real people in order to observe why it is confusing and where they get stuck. This is the insight.
Document and act with transparency
When we use AI, we must be transparent with the team and with stakeholders about how we have used it. In particular, we must:
Protect the data — This is a non-negotiable imperative. Never enter sensitive data or personally identifiable information (PII) of participants into a public AI model. Use only secure and private platforms.
Mitigate bias — Be aware that AI is trained on existing data and can amplify bias. We must actively ask it to consider different perspectives or accessibility needs.
Be transparent — Include a note in our reports specifying that the analysis was assisted by AI and validated by human interpretation.
Transparency is not only ethical: it is the basis of good scientific research.
The future of user experience research: artificial and human intelligence
Let us return to the initial question. Is AI really changing user experience research?
Yes, in a profound way. It is removing the labour of analysis at scale, it is opening new paths for simulation, and it is pushing those who do research to become more strategic, more critical and more curious.
Artificial intelligence is very good at generating, summarising and classifying data. But only human intelligence, or rather empathic intelligence, can understand a grimace, catch a hesitation, connect a behaviour to an emotional need and, finally, give meaning to that whole flow of information.
AI amplifies our capabilities; it does not act as a substitute. It is we who guide the process, ask the right questions and catch the nuances. The future of user experience research is not artificial; it is augmented.
This frees us from the complexity of analysing large quantities of data, in order to return us to the complexity — far more fascinating — of people and their experiences.
We also recommend
