Synthetic Research: An Explicitly Biased Introduction

Photo by Maxim Berg on Unsplash‍

Synthetic research is officially here to weird up the research game. In my opinion, as a former AI startup founder, the current generation of generative AI tools are a completely wretched take on how to make information widely available for all the ways in which people use it professionally. They are not transparent, inclusive, reliable, or accurate, making the data that comes from these platforms not strong enough to support synthetic research purposes.

That said, there is no question that they are being widely adopted in a wide variety of companies and situations, including for research. On a daily basis, I meet more people who mistrust the impact of generative AI than those who trust it. So, here is a brief guide, created for those who are likely to mistrust it, on what synthetic research is and how people are using it.

1. What is synthetic research?
Synthetic research refers to the use of generative AI tools to create simulated user profiles, behaviors, and/or responses for research purposes. These synthetic participants and their equally synthesized responses can stand in for human subjects in early-stage exploration, prototyping, or scenario testing.

2. Who on earth would trust synthetic research?
The idea of using wholly generated research participants is admittedly quite bizarre. The argument from proponents is that they can be useful when teams need rapid, low-cost input before investing in real participant recruitment. Other situations where it could be helpful is when exploring edge cases, rare user types, or sensitive topics. Synthetic participants can help researchers pressure-test ideas, identify gaps, or draft early insights without relying on human respondents.

Before generative AI, researchers, designers, engineers, and anyone else who engages in research could engage in solo or team exercises to imagine or ideate possible answers to their lines of inquiry. Often, this included guessing at user needs, developing personas, and pretending to be a specific type of user to come up with possible answers. With generative AI, this process becomes faster but also hijacked by whatever biases and limitations the platform’s creators have engineered into it — and these biases and limitations are continuously shifting.

3. How is synthetic research different from traditional user research?
Traditional research relies on real people—through interviews, surveys, usability tests, ethnography, or other methods. Synthetic research draws on large language models trained on broad datasets to simulate how a certain type of user might respond. According to chatGPT, it’s a speculative tool, not a substitute for genuine human data.

According to me, it’s not speculative, because that would require a brain behind the wheel — but it can simulate speculation. Imagine it more like a keyboard: The keys in this case are words, and chunks of words become names, personas, clusters of characteristics, and potential pathways. It’s never going to be real, but the fidelity of the portrait of a real human being has the potential to become strikingly close to the original, just as a keyboard can sound so similar to a piano that most can’t tell the difference.

At the same time, as a researcher who is used to combing through thousands of responses, there are specific cases where I simply cannot see an AI respondent being able to mimic the real thing. As another research professional pointed out, the way a generative AI tool would answer the “Other” option in a multiple-choice question is not going to correspond to the often very quirky or disconnected responses real human beings contribute. Compared to humans, gen AI research participants are likely to care too much, be too logical, or read too carefully. In other words, generative AI will never be able to do enough mushrooms to hallucinate the real thing.

4. What are the risks or limitations?
Synthetic profiles can reproduce bias, stereotype user groups, or create a false sense of confidence in unvalidated insights. Because the responses come from AI rather than lived experience, they shouldn’t be treated as evidence of actual human behavior or needs.

5. When is synthetic research appropriate to use?
Many would say that it’s most appropriate in early concept development, hypothesis generation, or preliminary exploration—moments when teams need quick directional thinking rather than deep validation. For decisions that affect real people, synthetic insights should always be followed by research with actual users.

In my view, it also depends on the research topic and the relative frequency of the situation being researched. It is likely to be more reliable about common problems where there are established modes of problem solving and weaker in more rare or poorly understood topic areas.

6. Do I need to start using synthetic research?

You don’t have to use synthetic research if you don’t want to. No one is telling you that synthetic research is a necessary part of your process — at least, not yet. Honestly, I think that if a research outfit decides to never adopt synthetic research, they will be stronger for it. For this reason, all TLC research services are scoped without the use of synthetic research methods, meaning all participants are real individuals.

If your company starts requiring it or you decide to start exploring synthetic research methods on your own and would like a thought partner, I have used it in the development of marketing campaigns and have built AI tools and platforms in the past as the founder and CEO of Thicket Labs and can discuss how it might fit into your process. I can provide my take from creating synthetic research tools in the past that were commissioned for use in client settings, and how the gen AI versions of today compare.

To discuss further, please reach out.