In June, the New York State Office of Mental Health (OMH) quietly finalized the independent evaluation of the state’s assisted outpatient treatment program, also known as AOT or Kendra’s Law.
The program, which has stirred controversy, allows adults with a history of non-compliance with mental health treatment to be court-ordered to comply with outpatient treatment. State officials say the report shows the program works as intended.
“Looking at the quantitative data, we really feel like this report shows that the assisted outpatient treatment program is having just the kind of impact it was designed to, which was to reduce hospitalizations and improve community safety,” Dr. Miriam Tepper, OMH’s chief medical officer, told BTPM News on Aug. 11.
However, researchers who authored the report claim the state is misrepresenting their findings and downplaying the harms they uncovered in their two years of research. A lead researcher on the project believes the state might be downplaying their research, in part, because the research team is largely composed of researchers with lived experience with mental health systems.
“The harms are far from insignificant or trivial. These are very, very serious harms, as people are experiencing them, and the argument then, the conclusion is, that those harms absolutely need to be taken seriously,” said Nev Jones, PhD, the evaluation co-lead and principal investigator of the University of Pittsburgh Path Lab, which co-conducted the evaluation.
The $1 million, 392-page report was required of OMH by the state legislature when the law was last renewed as part of the state budget process for 2023. The law was first passed in 1999 after the death of Kendra Webdale, a Western New York native who was pushed in front of a New York City subway car by a man with a history of mental illness.
The report compares AOT to its voluntary sister program, Assertive Community Treatment, or ACT. To enter ACT voluntarily, a referral is made, but who can make the referral and how it is made varies by county, according to OMH’s website. For AOT, it has to be court-ordered as the least-restrictive setting for an individual who has been involuntarily hospitalized at least twice in the past 36 months for not adhering to treatment, among other requirements.
According to the report and interviews with Jones and two members of her research team, their research found benefits for both groups from access to services provided by the programs. However, there were noted significant concerns found in interviews with people under involuntary orders, including coercion, fear, autonomy, understanding of the process, and inconsistencies in implementation.
Jones and two members of the research team feel the state has focused on the beneficial outcomes drawn from data but not the harms uncovered in their interviews with people under AOT.
“We knew going into that, that the risk of that is precisely that they'll just say ‘anecdote stories, we won't take it seriously,’ so we're then caught in a real structural conundrum, right? Because if a state doesn't want to take qualitative seriously, but also doesn't want to take measurement of harm seriously, how do we move past this impasse?” Jones said.
'They’re very different kinds of data': Looking at AOT two ways
The evaluation used a mixed-method approach, which combines quantitative and qualitative data. Quantitative uses numerical information — what most people think of when they hear the word "data" — like databases. Qualitative data is information not told through numbers, but through interviews.
“Quantitative research tends to get you these sort of black box findings: point A, point B. Something has changed. We don't really know the mechanisms underlying that,” Jones said. “In a realist paradigm, qualitative research comes in to ask the question of how and why did this happen? Whatever we're seeing, whatever patterns we're seeing, and to identify mechanisms.”
The state’s position — in both our interviews with Dr. Tepper and in an op-ed in the Times Union from the OMH commissioner — is largely focused on the quantitative analysis, which utilizes state data and showed significant improvements across several key outcome measures. In some measures, such as housing status, risk of harm to others, and “at least one [mental health] hospitalization,” both groups improved, but AOT was reported as better.
"What they found is that both groups improved across a range of measures, but that the people receiving assisted outpatient treatment improved more in a few key areas. For instance, they were less likely to harm others, they had greater housing stability, and they had fewer and shorter hospitalizations," Dr. Tepper told BTPM News.
“So, in those areas that I just mentioned, the people receiving assisted outpatient treatment improved more than people receiving the voluntary services, and we take really that to mean that there are things about assisted outpatient treatment, for this very small group of people who are eligible and appropriate, that really provides a degree of a unique advantage,” said Dr. Tepper.
Across the 10 outcomes evaluated by the researchers using data, the AOT group had better outcomes in four areas, while the ACT group had better outcomes in two areas, and neither group did better in the final four outcomes.
In a public letter, Jones and two of her team members argue public statements from OMH and its leaders have “mischaracterized, misrepresented or overtly 'spun' the evaluation study, its findings and their complexity.” The researchers argue statements about AOT being more successful than ACT lack nuanced or more careful disclosure of the in-depth findings of both the quantitative and qualitative data.
“Whether it's voluntary ACT or it's AOT, when people are connected with services that they want and need, certain kinds of outcomes absolutely improve,” said Jones, adding that the more in-depth analysis helps to understand what contributed to these groups improving, or not improving, compared to each other.
For the qualitative data, the research team “conducted interviews with 46 people under AOT, 25 family members, and 173 others — including providers, members of the judicial system, and AOT administrators.”
“They’re very different kinds of data. The qualitative interviews here surface some, I think, very important ideas and themes that we certainly plan to take very careful consideration of," Dr. Tepper said.
Dr. Tepper shared that while the qualitative had important themes, she doesn’t feel it can be used to generalize.
“The choice of how they gathered up the people that they interviewed was not designed to be representative of the population as a whole, and so what we can't do is extrapolate from the qualitative interviews and say, 'OK, 50% of people under AOT think this because 50% of the people in the qualitative interviews think that,'” Dr. Tepper said. “We can't generalize and apply to the whole population, necessarily, details of the qualitative interviews.”
The report, however, emphasizes in its qualitative findings issues with coercion, lack of agency, AOT recipients not understanding the how or why the orders are extended, and inconsistencies across implementation.
Jones gives the example of people under AOT feeling “disempowered by constant threats.” Individuals who do not comply with their AOT orders can face involuntary hospitalization.
“It was one of the most pervasive things we heard across this sample, with very diversified sampling across the state. So … yes, we are not trying to assign an exact percentage of population level representation, but it's not something that you can just dismiss and say isn't a real thing, right?” Jones told BTPM News.
'There was nothing that we could do': Challenges with the data and interviews
One of the challenges researchers faced, according to Jones, was the level of access to information they had. She gives the example of coercion being a known concern within mental health systems.
“There's a huge evidence base speaking to the harms of other forms of coercion. So, what we're coming in with is we want to be able to take that seriously,” Jones said. “How can we do it under constraints when the state collects no data on it, doesn't monitor it, has no system of receiving and monitoring grievances, concerns?”
The quantitative data available to the researchers came from existing databases, including Medicaid data, publicly available data, and databases maintained by OMH and the Division of Criminal Justice Services, among others.
“If the data that's collected by the state agency is related to services, it's administrative data. Then I'm going to have a higher degree of suspicion because it's not representative data,” said Scott Landes, PhD, a disabled researcher and professor of sociology at Syracuse University.
Landes spoke to BTPM News more generally about data sources used in research and his experience as the co-founder of the Disability Health Equity Research Network. DHERN supports the inclusion of disabled people in the research field and in research itself.
“It's only addressing the people who receive those services or those benefits or whatever it might be,” Landes said. “I would [also] want to know that the data source is complete, that it is inclusive of all people receiving the services.”
Landes said even when researchers have all the data they need, qualitative research is important to explain those findings.
“Even if I had the complete data set for someone receiving some particular type of service, and I can say, OK, this quantitative data represents everybody, and it shows me whether we see changes in employment status, in medication trends, whatever it might be,” Landes said. “But then I'm going to say, well, it's nice to know that. It's nice to know that the receipt of this service helped with homelessness, helped with employment. But I want to know: How did people get there?”
Landes said, as a researcher looking at a service, he ideally would be able to pull a sample size from that pool.
“My hope would be that I would be able to get access to the names and contact information for everybody who's receiving that service, and then I could do a random sample of 100 people and invite those 100 people to participate in my qualitative interview,” Landes said.
"You're never going to say for a qualitative study that is representative," Landes said. "But I'm going to be able to say the ability to participate in the interview was given to was a random draw, and so there's a better chance of my interviews representing the population in such a way that we can speak to in this group."
In response to Dr. Tepper saying “the choice of how they gathered up the people that they interviewed was not designed to be representative of the population as a whole,” Jones said they didn’t have the access they would need to pull from OMH’s clients for a random pool.
“If they didn't want to talk to us and didn't want to let us talk to their clients, there was nothing that we could do,” Jones said. “There was no mandate attached to the evaluation that guaranteed us any level of access to programs or people, so within the constraints of the evaluation, [this was] really the only way that we could start to dig into what impacts this is having.”
Ultimately, the researchers found clients to interview by distributing flyers through OMH offices, advocacy groups, and other organizations. In a follow-up email, Jones added that “theoretically a state sponsored [evaluation] can be set up with participation/cooperation mandates, but this one wasn't.”
'Concerns raised regarding alleged bias'
In a recent op-ed in the Times Union, OMH Commissioner Ann Sullivan wrote “court-ordered mental health services are a last resort for a very small group of individuals experiencing serious mental illness, and they’re not a decision reached lightly.”
Sullivan went on to affirm the improved key outcomes from the quantitative portion of the evaluation.
“The bottom line is that assisted outpatient treatment improves the lives of a small group of individuals with serious mental illness who need support on their path to recovery and aren’t getting there on their own, despite intensive efforts to engage them,” Sullivan wrote.
Jones said after two years of research, she disagrees with the notion that people are only put under involuntary order after all else fails.
“I think the reality on the ground is just so very, very different from a superficial surface reading of statutory criteria, and that's what one sees play out across the civil commitment continuum,” Jones said.
“People's own testimony is often discredited, so when they try to explain, ‘Well, it was just this thing I was protesting, not this other thing, or I got angry because I saw somebody else in the ward treated in this horrible way,'” Jones added. “That very easily, in the record, can become or be construed as: This person is violent or this person is aggressive.”
Stigma and bias against people with serious mental illnesses is an issue that impacts care even outside of mental health services.
A review in The Journal of Behavioral Health Services & Research looked at the impact of bias in clinical decision-making for people with serious mental health conditions receiving care for cardiovascular disease, diabetes, and cancer. The review found “only 21% of the findings demonstrated a clinical decision that was favorable towards people with mental illness.” The review concluded: "Healthcare provider bias against individuals with mental illness does exist and impacts clinical decisions negatively."
An article in American the American Psychological Association’s Monitor on Psychology detailed research that has shown stigma as a barrier to recovery and having a negative impact on patients.
Jones alleges that bias against people with mental health conditions might have also impacted how OMH handled the independent evaluation process with her team.
“From inception, we ran into literal charges, expressed concerns, by the Office of Mental Health in New York that people with lived experience could not or would not conduct interviews, for example, in quote unquote unbiased ways,” Jones said. “So, sort of unspoken, had we been a group of psychiatrists with no lived experience, that would somehow not have introduced bias.”
Pitt’s PathLab doesn’t shy away from its commitment to lived experience in research. Jones has been open about her experiences as a community psychologist with schizophrenia.
“OMH called me into a meeting that escalated so much that they demanded call logs with journalists,” Jones said. She alleges this specific instance happened after she was quoted in an article in Pennsylvania as a subject matter expert "articulating my concerns about AOT as a policy intervention."
“They said that they could clearly not trust the integrity of my interviews or our teams', or my teams' interviews with participants, and they wanted the full audio to determine whether I could even be trusted," Jones said.
Jones provided a letter to BTPM News she says supports her allegations. The letter, dated Nov. 21, 2025, was sent from the University of Pittsburgh Office of Research Protections to Bevin Croft of the Human Services Research Institute. HSRI was provided with the contract from OMH, with PathLab as the subcontractor. Jones was copied on the letter, as well as Lora Santilli, the Director of Operations for the New York State Office of Mental Health.
“When OMH is responsible for implementing a program and is also seeking to shape our research, it can compromise objectivity and erode trust in the findings,” wrote Bill Yates, Ph.D., the Vice Chancellor for Research Protections. “Even subtle pressure, whether the control of data, or expectations around reporting, can have a chilling effect on evaluators and undermine the integrity of the work. Our responsibility is to safeguard the autonomy of our investigators and the integrity of the research process."
“We are aware of concerns raised regarding alleged bias in the evaluation. To be clear, our investigators have confirmed they are not advocating for any outcome; they are reviewing the facts and conducting analysis consistent with academic standards and the scope of the sponsored research agreement,” the letter continues. “Interpreting data objectively and transparently is central to our institutional mission.”
Jones and two researchers interviewed by BTPM News disclosed experiences with hospitalization but not with any involuntary outpatient treatment. The two research team members interviewed emphasized they both didn’t know much about AOT before joining the research team.
“We're not a contract entity,” Jones said. “We're not trying to please government so that we can get more contracts from them to do the work. So, in my mind, we brought actually an awful lot, precisely, of independence to the questions that we were asking and the way we went about the work, and I was really rather taken aback in a pretty profound sense, honestly, by the quote-unquote monitoring and oversight of the contract.”
The New York State Office of Mental Health declined to comment on this story and referred to our interview with Dr. Tupper and Sullivan’s op-ed. We also reached out to HSRI regarding the researchers concerns the report has been misrepresented, but have not heard back.
“People want to think that if you have the characteristic being studied or being addressed that you're necessarily biased to a point that your opinion and your research should not be considered valid,” Landes said.
Landes is not connected to, or a part of, the independent evaluation. However, speaking on his own experience with DHERN and working with other disabled researchers, he said “we are consistently being told that our opinion is not valid because we're biased.”
“I think that's a very traditional and limited view of research that fails to understand that having a lived experience can actually be a place of information and does not necessarily mean that you can't be objective,” Landes said. “But yeah, that's an everyday experience for any disabled researcher, I think.”
