Synthetic Research Part I: An Introduction
Photo by Maxim Berg on Unsplash
Synthetic research is officially here to weird up the research game. There is no question that these new methods are being widely adopted in a wide variety of companies and situations, including for market and user research. Here is a brief guide on what synthetic research is and how people are using it.
1. What is synthetic research?
Synthetic research refers to the use of generative AI tools to create simulated user profiles, behaviors, and/or responses for research purposes. These synthetic participants and their equally synthesized responses are used to stand in for human subjects in early-stage exploration, prototyping, or scenario testing.
2. Who on earth would trust synthetic research?
The idea of using wholly generated research participants is admittedly quite bizarre. The argument from proponents is that they can be useful when teams need rapid, low-cost input before investing in real participant recruitment. Other situations where it could be helpful is when exploring edge cases, rare user types, or sensitive topics. Synthetic participants can help researchers pressure-test ideas, identify gaps, or draft early insights without relying on human respondents.
Before generative AI, research professionals could engage in solo or team exercises to imagine or ideate possible answers to their lines of inquiry. Often, this included guessing at user needs, developing personas, and pretending to be a specific type of user to come up with possible answers. With generative AI, this process becomes faster, but also hijacked by whatever biases and limitations the platform’s creators have engineered into it — and these biases and limitations are continuously shifting.
3. How is synthetic research different from traditional user research?
Traditional user research relies on real people providing their input through interviews, surveys, usability tests, ethnography, or other methods. Synthetic research draws on large language models trained on broad datasets to simulate how a certain type of user might respond. According to chatGPT, it’s a speculative tool, not a substitute for genuine human data.
According to me, a human, it’s not speculative, because that would require a brain behind the wheel — but it can simulate speculation. Imagine it more like a keyboard: The keys in this case are words, and chunks of words become names, personas, clusters of characteristics, and potential pathways. It’s never going to be real, but the fidelity of the portrait of a real human being has the potential to become strikingly close to the original, just as a keyboard can sound so similar to a piano that most can’t tell the difference. (But a keyboard synth is often most fun when it’s used to create a wide range of sounds that have almost nothing in common with a piano.)
At the same time, as a researcher who is used to combing through thousands of responses, there are specific cases where I simply cannot see an AI respondent being able to mimic the real thing. As another research professional pointed out, the way a generative AI tool would answer the “Other” option in a multiple-choice question is not going to correspond to the often very quirky or disconnected responses real human beings contribute. Compared to humans, gen AI research participants are likely to care too much, be too logical, or read too carefully. In other words, generative AI will never be able to do enough mushrooms to hallucinate the real thing.
4. What are the risks or limitations?
Synthetic profiles can reproduce bias, stereotype user groups, or create a false sense of confidence in unvalidated insights. Because the responses come from AI rather than lived experience, they shouldn’t be treated as evidence of actual human behavior or needs.
5. When is synthetic research appropriate to use?
Many would say that it’s most appropriate in early concept development, hypothesis generation, or preliminary exploration—moments when teams need quick directional thinking rather than deep validation. For decisions that affect real people, synthetic insights should always be followed by research with actual users.
In my view, it also depends on the research topic and the relative frequency of the situation being researched. It is likely to be more reliable about common problems where there are established modes of problem solving and weaker in more rare or poorly understood topic areas.
6. Do I need to start using synthetic research?
You don’t have to use synthetic research if you don’t want to. No one is telling you that synthetic research is a necessary part of your process — at least, not yet. If your company starts requiring it or you decide to start exploring synthetic research methods on your own and would like a thought partner, I can provide my perspective on how it might fit into your process, but also ways to be more efficient and effective without the use of AI tools and methods. Part II of my look at Synthetic Research looks back at my experience creating a collaborative AI tool as the founder and CEO of Thicket Labs to help contextualize how synthetic research might be being deployed.
