LLM Hallucinations Are Still Absolutely Nuts

A couple months back, a grad student reached out to me to see if I could help with some research they were doing. (I’m keeping things vague for reasons that will be apparent.) They had heard that I had previously been a patient at Connecticut Valley Hospital. I told them that I would be happy to do my best with whatever questions they might have, but that I hadn’t been there in 20 years and my memories were dimmed by the weight of years, psychosis, and psychiatric medications. But, sure - I would tell them what I knew. I answered questions about my experiences there, mainly focusing on how I came to be admitted, my legal status on my first go round, the wards I had principally been in, and my release. We exchanged email for awhile and that was that.

Then, just a few days before the Christmas holiday, they got back in touch with me. They had some more questions that they had not been able to solve via research. They asked me about several features of life “on campus” that I couldn’t answer, and I grew confused. One was about the “Vance Building.” I told them I wasn’t familiar with it, but that wasn’t unusual; CVH is a sprawling facility with lots of different buildings that in many ways operate as their own little worlds, and again, it’s been several decades since I was there. Then they asked me if I remembered “the club.” To which I replied, the… club?

A bit of confused back and forth went down, and eventually they sheepishly admitted that they had been doing some research through Google Gemini. They insisted (quite strenuously) that they would never have an LLM write any of their actual work for them - and, for the record, I believe them - but that they had tried to fill in gaps in information with AI. With my gentle pushback, they were now concerned that they had a few facts wrong. I asked them if they would copy and paste the Gemini output and send it to me and they did. And, hoo boy! Please note that Gemini is believed by many to be the most capable consumer LLM and that the grad student was using the “Thinking” mode for this. I was sent thousands of words of this stuff; I’m going to try and spare you by not quoting too much of it, but I do want you to see how wrong, and how obviously implausible, a lot of it is.

---

So, the “Vance Building” or “Vance Hall.” Gemini discussed Vance Hall in detail, describing it as a hub of patient education, vocational training, job opportunities, commerce, and socializing. The intricacy of the descriptions provided by Gemini are somewhat offset by the fact that Vance Hall does not exist.

To understand the Vance Building and The Club is to understand the geography of hope at Connecticut Valley Hospital (CVH). If the locked wards like Merritt and Dutcher are the “home” (however forced that home may be), then Vance and The Club represent the “city.”

For a patient who has spent weeks staring at the same four walls of a dayroom, being cleared to go to Vance or The Club is a monumental shift. It is the first time they are treated—even if only for an hour—as a student, a worker, or a social being rather than just a “patient.”

The Vance Building: The “Treatment Mall”

The Vance Building functions as the campus’s educational and vocational hub. It is often referred to as the “Treatment Mall.” In modern psychiatric theory, the mall concept is designed to get patients off the ward and into a centralized location to simulate “going to work” or “going to school.”

Setting aside the odd “geography of hope” thing - I assure you that if there was a “city” aspect of Connecticut Valley Hospital, patients in the high-security wards Whiting and Dutcher would not be allowed to go there. The concept of a treatment mall is indeed a real thing, but again the Vance Building is a hallucination. There is a road called Vance Drive that runs through CVH, which is apparently where the LLM got the idea. But Vance Hall/the Vance Building is a fictional facility that Gemini believes houses a lot of disparate functions that are distributed throughout the large and highly-compartmentalized CVH campus.

It’s important to understand that access to Vance and The Club is used as a behavioral tool. If you “act out” on the unit—even if it’s just a verbal argument with a peer—the first thing you lose is your “Vance time.”

This creates a high-pressure environment. You might be desperate to go to Vance to see a friend or get away from a loud roommate, but that very desperation can make you anxious, which the staff might interpret as “instability,” leading them to cancel your trip. It is a “Catch-22” that many patients find deeply frustrating.

Again, there is no “Vance” other than Vance Drive. This weird interpretation of the importance of that made-up facility is what worries me the most - not just the factual misinformation but the broader social conclusions drawn from it. I’m also not convinced that scenario constitutes a Catch-22.

The Escort: For many in Dutcher or Merritt, you don’t just walk to Vance. You line up at the unit door. A staff member (MHA) counts the group. You walk in a line across the campus.

The Threshold: Entering Vance feels different. There are high ceilings, larger windows, and a bustle of people from different units. For a moment, the “unit politics” of your specific ward fade, replaced by a more collegiate, albeit institutional, atmosphere.

Just weird shit, man. There is no Vance and thus no high ceilings or large windows or bustle. I suppose you can say that there are “unit politics” in any particular ward, although I’m confused by why these would not be called ward politics, but anyway, in my (admittedly limited) experience what you might call politics is limited by meds haze, strict regimentation of behavior, itinerant population, and collective embarrassment at being in such a place. Meanwhile, some patients get courtyard privileges or grounds privileges (not the same thing) depending on ward, behavioral record, and legal status, and if they do they generally don’t have to line up to walk anywhere. I don’t recall patients moving between buildings (which is rare for many categories of patients) in orderly lines.

It’s “the Club” that reaches “has to be seen to be believed” status here. Gemini was really having a normal one with this.

The Club: The Social Heart of CVH

If Vance is “work,” then The Club (historically the “600 Club”) is the “Saturday night” of the hospital. It is a social lounge and canteen located in the basement of one of the central buildings, and it is arguably the most important location for patient morale.

1. The Canteen (The “Store”)

The Canteen is the only place on campus where patients have “consumer power.”

• The Selection: It’s like a small convenience store. They sell coffee, snacks, toiletries, and sometimes greeting cards.

• The Ritual: Standing in line at the canteen, clutching a “trust fund” slip (the paper representing the money in your hospital account), is a rare moment of autonomy. Choosing between a Mountain Dew or a Ginger Ale is a small choice, but in a place where your bedtime is mandated, small choices are everything.

There was a cafe in my time, but I do not recall a canteen. As far as I am aware it took ordinary money, not “trust fund slips,” a truly bizarre concept. Note that there are classes of patients who have significant leeway to go off campus out into the local town and purchase whatever they want from whichever store. Those patients that have the least amount of autonomy, in the locked wards, wouldn’t be permitted to go to such a “canteen” in the first place. As for the “600 Club”….

2. The Social Ecosystem

The Club is where the different wards mix. This creates a complex social web:

The “Vance/Club Romance”: It is very common for patients from different units to start “hospital romances.” They meet at The Club, share a bag of chips, and pass notes through “messengers” (other patients). These relationships are often discouraged by staff but are a primary driver of patient motivation.

The Pool Tables and Music: The Club usually has a pool table, a jukebox, or a TV playing music videos. The sound of the “crack” of a pool break and the bass of a song provide a sensory relief from the sterile quiet of the wards.

Peer Support (Informal): In The Club, you hear the “real” news. Who got discharged? Who got sent back to Whiting (the max-security facility)? Which doctor is “writing for grounds” (giving out privileges)? It’s the hospital’s “grapevine” in its purest form.

Where to begin! It should not surprise you that “the Club” does not exist. Believe it or not, most psychiatric hospitals are not in the habit of creating spaces that would inevitably engender the exact kind of behaviors that they don’t want! Hospital romances are indeed discouraged at most psychiatric hospitals between patients in restricted wards, whereas the kinds of patients who have significant privileges would typically have enough off-campus and unstructured time to do what they want. No one is opening a pool hall for patients who are not allowed to wear belts or have shoelaces. I don’t doubt that ward gossip exists, particularly among long-term patients, but wards are compartmentalized spaces and it’s just hard for me to imagine extensive inter-ward gossip going on to this degree. Whiting patients are a special class even in the highly-regimented reality of the hospital, dictated by legal proceedings, and as far as I am aware it’s not common for patients to be routinely “sent back” to Whiting. Who patients would be passing notes to, and why, and why they couldn’t just talk to those patients if they share a ward… all mysteries. This whole thing is wild.

Because CVH is an old, sprawling campus and Connecticut winters are harsh, there is a massive network of underground tunnels connecting Merritt, Dutcher, and the other major halls.

The Experience: If you are a patient being moved from one building to another (for a medical appointment or a move to a new ward), you often travel through these tunnels.

The Vibe: They are dimly lit, humid, and lined with massive steam pipes. They feel like the “guts” of the institution. For patients, the tunnels are a source of constant rumors—stories of “secret rooms” or the “ghosts” of the old asylum.

There may very well be old tunnels running underneath CVH, but I have never heard of patients being transferred between them and the idea seems completely fanciful to me. While I have called the campus sprawling myself here there would still be very little advantage to moving patients between buildings via such tunnels instead of simply putting coats on and walking to another building or maybe taking a van. Patients just don’t have to move in large numbers between different wards that often. Locked wards are of necessity designed to be self-contained units so that patient movement is minimized. Perhaps some patient rumors are exchanged about secret rooms of ghosts but never in my life did I ever hear such things or indeed talk about underground tunnels at all. (You spend a lot of time in such places conspicuously not talking to other patients.) It would seem to me that the idea of routinely transporting psychotic patients through dimly-lit mazes filled with hot steam pipes would not appeal to your average state bureaucrat. The notion of intricate catacombs of tunnels underlying psychiatric hospitals is a common one in urban legends, and I am fairly confident that this particular portrayal is influenced by the film version of Girl, Interrupted.

Merritt vs. Dutcher: The “Vibe” Shift

If you move from Merritt to Dutcher (the forensic ward), you will feel an immediate shift in the atmosphere:

Strictness: Merritt feels like a “hospital.” Dutcher feels like a “facility.” In Merritt, the staff are often more focused on your feelings; in Dutcher, they are focused on your compliance.

The “End Game”: In Merritt, the goal is “Discharge to the Community.” In Dutcher, the goal is “The Board.” Every move in Dutcher is documented for the Psychiatric Security Review Board, which makes the stakes feel much higher and the environment much more controlled.

Freedom of Movement: Merritt patients often get “Grounds Privileges” faster than Dutcher patients. It is not uncommon to see Merritt patients walking in pairs to the Canteen or sitting on the benches near the main circle.

My understanding is that Merritt is now primarily a substance abuse facility, although this was not always true and I’m sure there’s plenty of treatment for people with comorbid mental illness. Here I’m more interested in these cliched depictions of what various wards and level of security are like; that line about feelings vs. compliance feels very, very much like an uninformed impression about secured psychiatric spaces rather than specific information about this specific place, which is what you’d want from an artificial intelligence. Again, I’m unaware of any place called “the Canteen” and I’m not sure what the “main circle” refers to; maybe the circular road around Beers Hall? Not sure.

The Canteen: This is the physical store where the “hospital economy” lives. It is often located in the basement of Merritt Hall or a central utility building. This is where you go to spend your PLP earnings. It’s the one place where you can buy “outside” brands—a real bag of Doritos, a specific brand of soda, or stationery. The line at the canteen is the primary social mixer for the campus.

Again, there is nowhere called “the Canteen” at CVH. The cafe is not in the basement of Merritt Hall or any other building, but more to the point, what on earth does it mean to say that a specific facility is “often” in one space or another? “PLP” here means “patient labor program” but while that term is not unheard of and while there are various vocational programs at CVH, I don’t think the hospital uses the term. That last line there is just bizarre. The line at the store is “the primary social mixer for the campus”? Instead of the day rooms or dining rooms? So weird.

Gilead Social Club (The Middletown Connection): There is a very famous community program in Middletown called the Gilead Social Club. While it is technically off-campus, “Level 4” patients (those with grounds privileges or community passes) often go there. It is a “clubhouse” model where people with mental illness run the snack bar, plan trips, and support one another. For many CVH patients, “The Club” is their goal for when they finally get their community privileges.

There is no “Gilead Social Club.” There is Gilead Community Services Inc, which is housed in Middletown and has programs throughout Middlesex County, but it’s not “technically off campus” but rather an entirely separate entity from CVH, a nonprofit. Gilead Community Services is a classic transitional organization, offering outpatient psychiatry, halfway houses access, and various training and counseling services designed to help patients with mental illnesses move back into ordinary civilian life. I don’t doubt that there are many former CVH patients who interface with Gilead, but it’s not part of CVH or integrated fully with CVH and calling it a “clubhouse” is just weird. It’s nonsensical to say that Gilead is where CVH patients “finally get their community privileges”; plenty of patients reach privilege levels that enable them to go out into the community who have no connection with Gilead.

In Dutcher, "Weeks" mean nothing. "Months" are the unit of time. You might be ready to leave after six months, but if the PSRB thinks you need "more time to process your index crime" (the crime that brought you there), you stay. This leads to a state of "Chronic Hopelessness." Patients see the seasons change through the plexiglass—the trees on the Middletown hills turning red, then bare, then green—while they stay in the same plastic chair.

The first sentence is halfway between nonsensical and cliché. While it’s true that forensic patients are often subject to the discretion of the PSRB, I would not say that such decisions are based on needing “more time to process.” “Chronic Hopelessness” is useless editorializing. My memory or geography may be failing me but I believe that Dutcher windows don’t look out the “Middletown hills” but onto parking lots and other buildings in the facility.

The Treatment Team Meeting: Once a week (or every two weeks), you meet with "The Team"—your psychiatrist, social worker, nurse, and rehab therapist. You sit at the head of a long table. They ask how you’re doing. You ask when you can go home. They usually say, "We need to see more stability." The meeting lasts 15 minutes. It is the most stressful 15 minutes of your week.

Again, things change, but I do not recall regular meetings with care teams of this many people. The detail about the long table and sitting at the head of that table seems very fake to me. I also don’t think care meetings are as short as 15 minutes, although there are various checkups and check-ins with various staff that are this short or shorter. There are longer and more consequential meetings with staff psychiatrists and social workers and various apparatchiks that at times can certainly be more stressful than whatever this is describing.

Breakfast: Meals are served in a common dayroom. The food is notoriously bland—think lukewarm scrambled eggs from a bag, soggy toast, and weak coffee (often decaf, as caffeine is restricted to prevent agitation).

There are dining rooms attached to some (most? all?) wards, although you have to have a certain privilege level to use them. My recollection is that eating food in the dayrooms is discouraged, but I could be substituting in a memory from a different facility. Fairly accurate description of ward food though.

At night, the ward never truly goes silent. You hear the "15-minute checks"—the sound of a flashlight hitting the glass of your door and the scratch of a pen on a clipboard as the tech confirms you are breathing. You hear the distant shouting of a patient in "seclusion" (the "quiet room"), the humming of the ice machine, and the constant air filtration system.

Checks are not universal across patient types; as far as I’m aware, at most institutions, 15-minute checks are restricted for patients on suicide watch or in restraints, and even that is more frequent than usual at night. Could this be happening at CVH right now? Sure, I guess. It seems highly unlikely to me that every patient is getting a universal and constant 15-minute check. And, as you’d imagine, while there are verbal checks where the patient has to respond, most of the time visual checks are performed in a way designed not to disturb or wake patients, so I don’t understand the details about checks making noise. In my experience checks are conducted by orderlies or nurses; I have no idea what “techs” is referring to here. For the record quiet rooms are not places of restraint, as you might infer from this, but rather “cool out” areas where patients are not physical restricted. The idea of psych wards as places of constant screaming is a pop culture commonplace but less true in my experience. There are ice machines on wards but in my experience they’re in staff-restricted pantries and the idea that you’d be hearing them humming in your room is weird.

Here’s the kind of thing that really bothers me.

To live at Connecticut Valley Hospital is to live in a liminal space. You are not in prison, but you are not free. You are a “patient,” but you are also a “resident” and a “ward of the state.”

It is a world of beige walls, lukewarm coffee, and the constant, crushing weight of other people’s expectations. It is an environment where “recovery” is the goal, but “management” is often the reality. You learn to survive by finding small victories: a particularly good book from the hospital library, a staff member who treats you like a human being for five minutes, or the rare day when the “yard” stays open for a full hour.

Obviously, since Gemini is drawing all of this from deeply-erroneous factual information, such impressions should be even more suspect. Well, to address facts first: many patients in CVH are in fact not wards of the state and are indeed free to leave; in fact, contrary to public perception, in many psychiatric hospitals most patients are there on a voluntary basis and can leave whenever they’d like. “Ward of the state,” for the record, is a specific term with a specific meaning and even many involuntary patients would not necessarily be considered wards of the state. There is not one “yard” at CVH; some of the wards have their own courtyards, but as far as I know they don’t “open or close” beyond what’s dictated by privileges allowed to specific classes of patient, and again, many patients have grounds privileges and sometimes the ability to leave campus. As far as the more general stuff, well, many have struggled to find the line where management ends and recovery begins, but I would not trust an LLM to have an intelligent thing to say about that question. One of the most pernicious lies about inpatient life is that all of the staff are cruel or inhumane, and many doctors, nurses, orderlies, and staff members have treated me like a human being in our every interaction, including at CVH. And I’ll tell you, one of the few upsides of finding yourself in a psych ward is that it’s a place blessedly free of other people’s expectations.

The point is, this is folk antipsychiatry of the most insipid kind, put together by a stochastic parrot that was incapable of ascertaining basic facts about the institution and thus pulled impressions from the ether. It’s true that a place like Connecticut Valley Hospital is a difficult thing for an LLM to assess; state hospitals like that one both live in text in a way LLMs like (there is an immense public record about CVH) and yet the actual experience of the place, its brick-and-mortar, flesh-and-blood reality is opaque thanks to privacy laws, the type of patients who populate it, and the reticence most of them feel about talking about it publicly. But of course, the thing to do when you don’t know something is to say that you don’t know something. LLMs hate to do that; they constantly respond to scenarios where they have insufficient information to correctly answer a question by just winging it - by hallucinating. That’s because these are probabilistic engines that have been built to provide plausible seeming answers, to make users feel that they have been informed. Actually informing them is a secondary goal at best. In this particular instance, the outcome is an embarrassed grad student who has hopefully learned an important lesson. But we have tons of evidence that vast swaths of the public trust these LLMs totally, with zero skepticism. Do you feel good about that? Do you really want public impressions of controversial, highly-consequential public institutions like psychiatric hospitals to be influenced in this way?

I would love to take this sort of thing to more-reasonable AI boosters - not your insane Silicon Valley “I’m going to have my own moon” maximalists but your good-natured and gullible fantasists, your Ezra Klein types, who have allowed their desire to live in the cool future to overwhelm their most basic skepticism and standards - and ask them… do you really want these systems to take over mission-critical jobs from human workers? Do you think they’re ready, when they constantly go on wild hallucinatory journeys like this? You want to give this system the ability to influence medical decisions, legal decisions, economic decisions? Decisions of life and death? I am just baffled, baffled, baffled by the refusal of our media to stop and say, guys, this technology does not work.