Every hospital leader has looked at an ED experience score and wondered whether it tells the full truth. It doesn't, and the reason is who gets counted in it.
The sample is filtered before you ever see it
Experience surveys only reach patients who complete their visit and are eligible to be asked. That matters more than it sounds. In the largest single-ED study of its kind, fewer than 2% of all patients treated in the department completed a survey. That group wasn't a random cross-section either: older patients were far more likely to respond, men were less likely, and Medicaid and self-pay patients responded at roughly a third the rate of privately insured patients.
Some patients never get the chance to respond at all. ED CAHPS, the standard experience survey, only goes to patients discharged directly home. Boarded and admitted patients, often the ones who waited longest and had the hardest visit, are excluded by design. Patients who left without being seen never generate a survey-eligible record.
The filter gets worse on crowded days
On a quiet day, almost everyone who arrives stays through discharge, so the sample looks close to the real population. On a crowded day, the patients most sensitive to long waits walk out. They still used a chair and staff time, but they drop out of the pool of patients who could ever rate the visit.
One study found patients surveyed during the highest-occupancy quarter of shifts were roughly half as likely to recommend the ED, and the authors noted that only discharged patients were eligible to answer. So the real link between crowding and dissatisfaction is likely worse than the data shows. The worst nights produce the least reliable scores, and those scores make the nights look better than they were.
What this means for the number on your desk
If your ED CAHPS or HCAHPS score looks acceptable, ask what population it actually represents. It likely underrepresents younger patients, male patients, Medicaid and self-pay patients, boarded and admitted patients, and everyone who left before being seen. In many EDs, those groups add up to a meaningful share of total volume.
None of this means your survey vendor is doing something wrong. Treat the score as a floor on how many patients are dissatisfied, and build a second, broader-reach way of listening that captures the patients the official survey was never designed to include.
This problem is solvable, and unlike the survey itself, fixing it doesn't require CMS to change anything first.