UX research starts by defining the problem, identifying stakeholders, and deciding what the research should change before collecting data.
2
Researchers need close collaboration with engineers, designers, product teams, and leadership to produce specific findings that can guide decisions.
3
AI teams can use short, phased research projects to find early signals, understand developer needs, and avoid building tools that people will not use.
Summary
Lauren Kaplan describes UX research as learning how people interact with an experience, including their needs, pain points, motivations, and behavior. She explains that the work spans interviews, contextual inquiry, surveys, behavioral data, and secondary research. In AI and developer tools, research can clarify which workflows, frameworks, model capabilities, and use cases matter to people. Kaplan stresses that research only becomes useful when stakeholders agree on the problem and participate in shaping the questions. She also discusses the practical limits of industry research, where teams may have only days or weeks instead of months. Phased studies can provide early signals and refine later questions. The conversation covers bias in survey responses, including people using AI to complete paid surveys, and the difficulty of measuring research impact. Kaplan argues that researchers must connect user needs with business concerns such as efficiency, competitiveness, and product adoption. She also describes the organizational trust and communication needed for research findings to lead to action.
UX research examines how people experience a product, workflow, or tool
Lauren Kaplan defines UX research as gathering insight into how people interact with an experience. The work looks at users' needs, pain points, motivations, and behavior. It can involve contextual inquiry, where a researcher observes people using a product, as well as interviews and surveys. Researchers can also analyze behavioral data or use machine learning for tasks such as sentiment analysis. The method depends on the subject and the privacy expectations of participants. Someone working on a confidential project may refuse observation but agree to an anonymized interview where their rights are clear.
A research project needs stakeholder alignment before data collection starts
Kaplan says researchers should define the problem, identify all relevant stakeholders, explain the expected impact, and decide who needs to be included before starting interviews or surveys. The researcher then creates an interview guide or research plan with engineers, designers, product teams, marketing, or other groups. Engineers should give specific feedback about what they need to learn, such as what makes inferencing feel effective. Without that engagement, research tends to produce high-level findings that do not answer the team's real questions. Kaplan points to Meta, where engineers actively commented on research documents and helped shape focused questions.
UX research can cover developer communities, markets, and technical frameworks
Kaplan says UX research reaches beyond traditional product studies. In AI, researchers may examine how companies communicate about AI, what developers need, which framework capabilities matter, or how a framework compares with others. The work can also include market research that supports both engineering and marketing. Kaplan distinguishes primary research, where the researcher collects new interviews or survey responses, from secondary research, where existing sources such as community discussions are analyzed. She still expects both forms of work to follow a systematic plan tied to a clear impact.
An analysis plan makes collected data easier to interpret
When Demetrios Brinkmann asks how to connect survey data to stakeholder needs, Kaplan recommends creating an analysis plan before collecting responses. Every question should have a purpose, especially when participants have limited time. The plan can specify the information needed about respondents, their roles, challenges, and opinions, along with the patterns the researcher expects to examine. Structured questions can produce measures such as an average satisfaction score. Open-ended questions can be manually coded or analyzed with a language model. Kaplan says unexpected findings can still be useful, but a systematic plan prevents the researcher from collecting data without knowing how it will inform a decision.
Survey research has bias and new problems caused by AI-generated responses
Kaplan discusses social desirability bias, where people who choose to answer a survey may be more engaged or more positive than people who do not respond. Incentives such as gift cards can bring in additional participants, although they do not solve every sampling problem. She also mentions an unreviewed study of a commercial research platform that found 35 percent of responses involved people using AI to help fill out surveys for payment. This creates a new methodological problem. Researchers need to understand how AI-assisted questions and answers affect missing data, bias, and the interpretation of results.
Research has to connect user needs with business outcomes
For an ML engineer deploying models or building developer tools, Kaplan says a researcher can give visible evidence about users' pain points and support requests for improvements. She agrees that monetary impact can persuade organizations, especially when a large business problem has a measurable improvement. Efficiency is another useful measure, such as reducing the time engineers spend on deployment or helping them iterate faster. Kaplan also mentions product satisfaction, ease of use, pain points, and product gaps. The researcher has to find an overlap between what benefits the company and what helps the people using its products.
Phased research helps teams learn quickly and revise their questions
Kaplan describes research projects that begin with a pilot survey and then move into a deeper study of a topic such as model monitoring. The first phase can establish who is using which models or tools, while later work explores the reasons and workflows behind those choices. This approach gives teams a chance to refine questions and see what is working before committing to a larger effort. Kaplan contrasts older foundational studies, which might have received a quarter, with iterative research that may need to produce findings within weeks or days. Faster AI development makes collaboration more important because teams have less time to discover that they are asking the wrong question.
Organizational trust determines whether research findings lead to change
Kaplan says research impact depends heavily on organizational maturity, alignment, and trust. In a company with little research experience, a researcher may need to build credibility through smaller studies while also explaining the value of broader work to leadership. The researcher may have to create research operations, secure tools, and communicate findings repeatedly across the company. In higher-maturity organizations, the function and its value are already understood, which gives research more room to influence decisions. Kaplan is direct that researchers cannot fix every environment through effort alone. If people do not build trust through collaboration over time, leaving may be the right signal.
Developer experience research can guide internal engineering decisions
Kaplan applies the same methods to developer-focused tools and internal engineering teams. An engineering manager could survey the organization to learn which tools people use, where they struggle, and what support they need before changing platforms. The findings might show that a team needs help learning another tool, has a persistent data quality problem, or finds an open source component unreliable. Kaplan mentions Google's DORA work as a resource on developer experience. She says this kind of internal research can help managers decide where to spend time instead of relying on assumptions about what their teams need.
"Every engineer manager is doing this, but taking the time to survey, especially if you have big teams or you have a big engineering organization, to recognize where pain points are for the teams."Demetrios Brinkmann1:04:24
Who should watch
Engineering managers who need a practical way to learn where their teams lose time and which tooling problems deserve attention.
Teams building AI or developer products before they commit to a workflow, framework, or use case.
UX researchers entering an organization with limited research experience and needing to build trust while producing useful findings.
