The Balance of Screening Tests

So we routinely screen people when they get to the right age - because the number of lives saved is less than the number of lives lost due to complications.

Change 'less' to 'more'

You didn't make much mention of costs, and they are a significant consideration in this analysis. And, nobody is saying a 45-year old woman can't have a mammogram. She just might have to pay for it herself, that's all.

It ends up coming down to a mathematical optimization process. You want to maximize the survival rate of the population as a whole.

It's worse than that, though, because in addition to objective costs, you also have morbidities, which are purely subjective. Like the loss of feeling and mobility in your father's leg.

Also, I would argue that even the cost of deaths are subjective -- they shouldn't be, but we are humans after all, and trying to deny that is going to just result in a bunch of unhappy humans. The perceived "cost" of a we-did-all-we-could death vs. a death caused by an undetected condition might be different. In fact, I suspect the latter is generally greater -- which helps explain the uproar over the revised guidelines.

To what extent this irrationality should be indulged is debatable, but to ignore it completely is folly...

Oh, and one minor nit: I would not say "the best recent evidence, when put into that optimization problem, has strongly suggested that the mammograms in women under 50 aren't worth the risk" (emphasis mine). There are a lot of subjective costs in the analysis that drove the recent guidelines, and some people might legitimately look at the same data and reach a different conclusion. I agree that it strongly suggests that once every two years is better than annually -- but on the 40-49 thing, the cost/benefit is not tremendously worse than in the 50-59 age group.

This doesn't change the points you made about the math behind it, but it is worth emphasizing that part of the reason for the controversy is that mammograms for 40-49 age group are a damn tough call.

But giving the mammograms will harm a different group of women - and it appears that the group harmed by the earlier mammograms is larger that the group helped by them.

But the level of harm in the group harmed by screening is going to be far, far less -- maybe a biopsy that turns out to have been unnecessary, but there's no way to know that ahead of time -- than the level of harm in the group harmed by not screening -- they're probably going to die or have much more advanced cancer by the time it is found.

spit: "But the level of harm in the group harmed by screening is going to be far, far less -- maybe a biopsy that turns out to have been unnecessary, but there's no way to know that ahead of time -- than the level of harm in the group harmed by not screening -- they're probably going to die or have much more advanced cancer by the time it is found."

Except that there is a non-zero risk of causing breast cancer with the x-rays used in mammograms. And there are risks of infections/complications with the biopsies in false positive cases as well. Each of these risks to an individual is smaller than the risk to an individual with undiagnosed breast cancer; however, these risks need to be multiplied by the entire cancer-free segment of the population to get a proper comparison.

Screening earlier presents a small harm to a large group of people, to prevent a large harm to a small group of people. The small group of people is small enough in those under 50 that the small harm to the large group is greater.

There was a similar debate recently in Australia about testing for prostate cancers:
http://www.theage.com.au/opinion/prostate-proposal-risky-20090923-g2l1…

But by doing that, you'll give the disease to some number of people who wouldn't have gotten it otherwise;

This sentence is confusing. You said the disease wasn't contagious. Do you mean that the disease will be detected in some people, but they may die earlier from other causes?

Chris@6

In Mark's scenario, the test itself has a small chance of CAUSING the disease. Therefore by testing lots of people who don't have the disease, you will inevitably give some of them the disease.

NeoDevin:

The small group of people is small enough in those under 50 that the small harm to the large group is greater.

I'm dubious about, in the real world, applying some sort of multiplication of small risk from mammograms, say, across a large population without having a way to really separate the breast cancer that was brought about by mammograms from the breast cancer that wasn't, if such a thing could really be teased out of actual data.

I don't think your conclusion is obvious to me. I don't necessarily disagree with the point you're trying to make, but I think it's fundamentally a subjective problem -- there has to be a cutoff, and people will try to make that cutoff in different places given the data that we've got.

I think the math can inform those decisions, but I don't think in this case that it can provide an obvious answer. The difference between the 1 true positive in 1300-ish total screenings (for 50 - 59 year olds) and 1 in 1900 (for 40 - 49 year olds) is not, as far as I can tell, an "obvious" line when you consider the cost to that one patient of not screening.

spit,

The increased incidence of breast cancer due to the radiation dose received from mammograms is easily quatifiable. In addition, because remaining life expectancy is lower at 50+ than 40+, there is less time for a patient to develop cancer, and therefore the risk of developing cancer due to the mammogram decreases with age. Basically older patients are more likely to die from something else before they get radiation induced breast cancer.

If cause fatal breast cancer in ~1 in 1500 patients through regular mammograms, then it's pretty clear that (neglecting other complications and quality of life arguments for the moment), with the numbers you gave, it's worth it for 50-59, but not for 40-49. I don't have the actual numbers handy, if anyone does, feel free to post them.

All of these factors, and others, are taken into account before a recommendation for screening is made. The goal of the recommendation is to maximise the life expectancy and quality of life of the people. The better you can quatify the various risks and benefits, the better recommendation you can make.

For hormonal reasons, breast tissue in pre-menopausal women is denser than in older women, which makes the mammogram a less effective screening method for those under 50. This is another reason why mass screening for younger women is not recommended.

"The increased incidence of breast cancer due to the radiation dose received from mammograms is easily quatifiable."

Easily quantified or easily extrapolated? It is hard to imagine an actual human study being allowed to take place.

It is easy to extrapolate an increased cancer rate in humans based on animals subjected to concentrations of dioxin, but I don't think it would be accurate.

If these recommendations really take into account increased cancers caused by screening mammograms, just how accurate is that number? We are talking of very low frequencies, and a 40-49 strata that is very close to the next decade.

My concern is that part of the calculus for these new recommendations was actual monetary costs, not just the 'costs' of anxiety in women patients or increased cancer rates due to mammograms. I don't think anyone is privy to the algorithm used and what weighting was given to monetary costs.

Call me cynical, but when I see anxiety listed as a cost, I'm thinking that what they mean is they can justify not spending the dollars on 40-49 year old women.

Gingerbaker, does it matter if monetary costs are included? Unnecessary tests are part of the larger problem of healthcare in the US (increased costs for the insurance companies lead to increased costs for the rest of us, even when we do not participate in the original cost-increasing tests). If the data shows convincingly that more women are hurt by screening under 50 than are helped by it, and there are adequate provisions for women with BRCA genes to still be screened regularly, then why shouldn't this equation factor in the monetary cost of screening women under 50 as well? I guess I just don't see how that's a bad thing.

Gingerbaker,
"Easily quantified or easily extrapolated? It is hard to imagine an actual human study being allowed to take place."

Easily quantifiable by comparing rates of incidence in people who have had the screening vs. those who have not.

"My concern is that part of the calculus for these new recommendations was actual monetary costs, not just the 'costs' of anxiety in women patients or increased cancer rates due to mammograms. I don't think anyone is privy to the algorithm used and what weighting was given to monetary costs."

Anxiety caused by false positives needs to be taken into account. Given that being told "Your mammogram was positive", or even "You need to come back for additional tests" causes a huge amount of stress.

Monetary costs need to be taken into account as well. Every dollar spent on mammograms is one dollar less that's available for another procedure. This also has to be weighed into the cost-benefit analysis. If the money to screen women 40-49 for breast cancer comes at the expense of, say, screening men 50-59 for colon cancer, there will be a trade-off. There is not unlimited money to go around, and financial concerns need to be taken into account.

@14: It's not as simple as comparing the incidence rate in those screened v. those not screened. Because we know a lot about other factors for breast cancer than 'subject has breasts'.

Those with genetic predispositon will be screened more often. Some with extreme predisposition for extremely agressive cancers will have profylactic radical mastectomy. Thus, never screened, never positively tested. It's far more complex than you suggest.

What about doing a baseline mammogram at 40, then another at 45? Why does it have to be yearly or nothing?

As a woman diagnosed with aggressive BC in her 30s, the only solution I see is a cheap, safe screening method. I had never had a mammogram because I was too young. By the time my tumor was discovered it was basically the whole breast. The guidelines wouldn't change my situation, but what if I had been slightly older when the cancer started? A mammo at 40 might have saved my life.

One other thing not in the discussion: rates of under 45 breast cancer is skyrocketing. Delaying diagnosis will lead to an increase in BC deaths. Mark my words. Plus, these new guidelines will make it easy for insurance companies to deny mammograms.

The whole situation is very troubling.

What I wonder is if the board didn't take the "lead time bias" argument into account. There is a great essay by Plotkin arguing that we aren't curing many early BCs, just finding them earlier and therefore making it look like women's survival is longer. His argument is that the "cured" diseases would never have killed anyone anyway - they would have remained hidden forever. And the aggressive cancers will kill regardless of how early they are found.

Pretty chilling stuff for a 41 year old with a huge grade III tumor.

And no, I didn't have any family history, was healthy, slim and a vegetarian.

So, even though YOU at home probably think you're safe and that cancer only hits the fat, old, lazy or genetically predisposed, you're wrong. It can happen to you too.

@12:

Monetary costs *are* a real and important factor to consider in things like this.

For many people (in the US, for most people), the amount of money that they can spend on medical care is limited. There's always a balance between giving people every possible bit of care that they could possibly ever want, and limiting it to what's necessary.

Why not do mammograms twice a year? You'll catch some cancers much earlier than you would have if you did it yearly. Because the cost of it would be huge (both in terms of health, and in terms of money), and the benefit of it would be miniscule.

If every time one of my kids got sick, we ran a genome extraction on the infectious agent to identify it precisely, and then picked a medication which was best for that specific infection - that would be wonderful. We'd be able to treat them better, they'd get better faster, and we'd have a whole lot more information about how and where antibiotic resistance is developing.

We don't do that, because the cost of sequencing the genome of a bacteria is just too expensive, and most of the time, the kid will get over the infection by themselves without help.

Even being less extreme than that - last spring, I caught the swine flu. Or at least, I *think* I did. I wasn't tested for it. The area that I live in had a huge infection rate, and the school system knew that several children had been sent home and tested positive for H1N1. Anyone who became ill with flu symptoms during the normal incubation period after exposure was assumed to have swine flu. Would it have been better to know for sure? Maybe. But it would have cost a huge amount of money to do the tests on the thousands of people in the county who were infected, and it wouldn't have really helped anything.

In the case of cancer, if you spend more money on screening, you're leaving less money available for research and treatment. The numbers are fairly interesting to consider: the number that I recall is that for each 1800 mammograms of a woman below 50, 1 cancer is detected that wouldn't have been detected otherwise.

A quick web search says that a mammogram costs roughly $100. So the screening costs
$180,000 to detect a single cancer.

If that detection saves a life, that's wonderful. But a fair number of the people detected have cancers that would have been detected anyway; and an additional number have cancers that won't respond well to treatment - their lifespan won't be improved by the early detection. (See Orac's description of lead time bias.)

It works out to something on the order of $250,000 per treatable case of cancer.

Now, what you need to consider is: what else could have been done with that $250,000 that was spent to detect a cancer at an early stage? Could it have been used to provide better treatments for women with breast cancer?

That's a real question that needs to be balanced. It's possible that you'll save more younger women with breast cancer by putting the money into treatment rather than early detection.

Another possibility is to use other screening methods to identify people at risk. There are lots of women out there with the mutations that make breast cancer more likely, but who don't know it. What if we spent the screening money on identifying the women at risk?

There's a similar tradeoff being debated right now about cervical cancer. Cervical cancer is a terrible disease which kills lots of women. Right now, starting at age 21, most women get screening tests yearly to detect cervical cancer. But the vast majority of cervical cancer cases are caused by a virus - HPV. We can spend money on pap smears, or we can spend money on a rather expensive vaccine. Which one should we do? The balance appears to run in favor of the vaccine.

Talking about these balances sounds cold. But it's a real thing that needs to be done. You can't just spend infinite amounts of money on medical screening. So you need to find out the balance, where the cost justifies the benefit.

@12:

Monetary costs *are* a real and important factor to consider in things like this.

Mark, just ignore it. Gingerbaker, a commenter who is usually somewhat sensible, made a big stink over at Orac's blog with the same ludicrous argument. At one point, he/she actually asserted that the only reason "[r]esources are limited [is] because of collusion and lack of social justice." Seriously.

Normally a very sane commenter, but not on this issue.

Let me preface this by saying these are just questions and first impressions. I don't actually know much of anything about insurance, health care, or economics.

How's the change of money work? So, right now, my insurance pays the $100 for my wife to get the mamogram because that's a service in our commodity based system.

How's that money get diverted away from that detection effort and toward the cure for my wife's cancer?

More likely, I would think, our insurance premiums go down by some amount. Then either (a) the insurance companies profit goes up and perhaps they contribute that increased profit to cures? or (b) the gov't imposes some kind of tax which is used to fund cure research efforts. But often, the government uses money for other purposes... unless legislation is laser precise.

From a monetary standpoint and in view of the way the system is constructed now: is it a necessary implication that if you don't spend the money on detection efforts, then the same money is spent on efforts to develop a cure?

If not, then are we sure that the money is being put to better use? Or is it used to build a bridge, or fund a school, or build some insurance executives house? Does the comittee take the way that the system really works into account?

All arguments about the detection effort causing what you are trying to detect are perfectly reasonable and valid. But the monetary costs vs. benefits... I don't know that its our best interests at heart... like whoever is spending the money will take that and do something "better" with it. I don't know about that since its really us who the money comes from for a service provided directly to us. I'd like to see the accountable mechanism that proves that the money will now be spent on cure research.

So I see Gingerbaker's point. Maybe someone can educate me about that concern.

Just as a quick follow up:

I'd rather spend $100 a year on my wife's mamogram than on a new tv or electrontrics or to fix a pot hole or whatever. She means more to me than any of that. But I'd also not mind spending $100 to find a cure or do something for her that will benefit her life more than that.

And really, I'm just saying my wife as a placeholder for any female figure I care about.

So where's the money really going?

Nelson: I see these as two separate problems. Rarely when we save money in one area is it clear "where the money goes" -- that might not even be a reasonable question in most circumstances. It's like saying, if I scoop a bucket of water out of the ocean, from which river is the water coming to replenish it?

To me, there is the localized problem of, "What is the best bang for the buck on cancer screening?" and there is the global problem of, "How can we identify and eliminate gross abuses of resource allocation?" If we conflate these two separate problems, it can lead to all sorts of absurd conclusions.

For example, I might argue that we don't want to take away the money to "build some insurance executive's house", because that money will just end up going to finance illegal CIA interrogations. At least when the executive was getting it, the outcome was innocuous, and it maybe provided a handful of jobs, right? Of course this is ridiculous, but you see my point... there is often not a clear point A-to-point B connection of "where the money saved is going", and if you worry that money saved will be spent on something even more worthless, you can never save any money on anything.

I have a bad habit when my wife and I are grocery shopping, that if she decides to buy some super high-end candy bar (her vice), I say, "Well, if we're spending seven bucks on a candy bar, I can surely spend nine bucks and get this 6-pack of microbrewed beer!" (my vice) But the fact that she probably shouldn't have indulged in the candy bar doesn't mean that I should indulge in the beer. Or should I just get my beer anyway, because if I don't, the money will just be squandered on candy bars? :D

In the specific case of privatized insurance companies, this argument has a touch of legitimacy. The percentage of any given healthcare savings that ultimately just go to increasing the companies' bottom lines is most likely appalling. (Though it's hard to make a direct A-to-B connection as I described above) But I still argue that this is a separate problem. The fact that for-profit health insurance is a retarded idea does not give us carte blanche to spend money any way we want to, any more than my wife's purchase of a seven-dollar candy bar gives me carte blanche to buy as much beer as I want. It's still necessary to weigh costs and benefits -- even monetary costs -- or else you are going to end up with all sorts of pathological conclusions.

I do think, though, that as long as we're stuck with this broke-ass healthcare system, that maybe slightly deprioritizes monetary costs... but again, isn't that exactly the problem we are having right now, that are causing healthcare costs to spiral out of control?

As I argued to Gingerbaker over in Orac's blog, the problem that healthcare reform is trying to solve is that resources are being allocated inefficiently. To argue that this means we should disregard efficiency altogether until reform is achieved is just silly.

How would it change the numbers if there was a less dangerous version of a mammogram, like maybe an ultrasound based one? (Not that I actually have any idea whether that's currently practical.) And MRIs are less dangerous but I think also more expensive.

I do like the comment about finding cancers which would have probably been "mostly harmless" (or at least not fatal), and then treating them so aggressively as to cause more harm than the cancer would have.

@Chuk: I for one don't know the answers to those questions, but one problem with MRI as a first-line screening method (in addition to the cost) is that it's so sensitive that you end up treating even more of the "mostly harmless" tumors, i.e. ones that would never have become symptomatic during the patient's lifetime.

It's really tough, because when you find a tiny tumor, there is no way of knowing if it is slow-growing and will never become a problem, or if it's super-aggressive and you just got lucky in finding it early (really aggressive tumors can go from "undetectable" to "you ain't gonna survive" in less than a year, i.e. between mammograms :/ ). Ideally, someday in the future you will be able to tell from a biopsy which type it is, and then make a decision between "aggressively treat" vs. "watch and wait". But until that day, we've got this weird paradox where sometimes it's better to not know you have a tumor... :(

@22

I don't think that we should disregard efficiency, but we should remember how the system really works.

With respect to the candy bar analogy: In this country you can buy an expensive candy bar or microbrew beer any time you want. It's bad in that you can get obese, but its good in that you can.

I remember coming home from serving overseas, and I got really bad food poisoning. I went to a local small town hospital after five days of not being able to keep anything down. The doctors there gave me two CAT scans. They probably did it to bill the insurance company since this hospital was having issues paying their own bills. But did I care then? Hell no, I wanted to get fixed.

Do I care now? Maybe that money could have been spent more wisely on somebody else in the hospital or on the hospital itself. Maybe instead of two CAT scans I was only justified one. Maybe none. Maybe it was inefficient. Then again, maybe not. Doctors made that call, and even if it was inefficient you can be and do so in America.

An engineer friend of mine once said: "Be efficient 99% of the time, but when you have to be inefficent don't worry about efficiency."

So the question that this problem really raises is how you keep that freedom, while at the same time making sure that everyone has the ability to obtain affordable health care. Maybe you can't do both at the same time... maybe you can. I guess we'll find out in a couple months.

The reason most people take affront to the 40-49 recommendation is that it is not coupled with anything else in these news articles. Mark and other people like him have to spell it out.

Maybe the recommendation is good, but it (or more precisely, the news that tracks it) lacks the follow up of what other screening processes or pertinent risk factors justify a screening. Most people see it and think that it will give insurance companies justification to drop coverage for mammograms. Probably an irrational argument, but that's what people see and think.

I plan on reading some of the recommendation fully before I come to any decisions. I don't know enough to support or refute it. I would recommend others do so as well.

http://www.ahrq.gov/CLINIC/uspstf/uspsbrca.htm

@23

There is a breakdown of some of the considerations made with regard to the 40-49 argument and the treatment methodologies at the link below.

http://www.ahrq.gov/clinic/uspstf09/breastcancer/brcanrs.htm#clinical

Interesting post. Just for closure to the story, who ended up being right about your dad's second surgery? Was it scar tissue or cancer?

@27:

Sorry, I thought I said. It was not cancer. He went through the surgery, the horrible recovery, and the 20 years of trouble with circulation in the leg, for nothing. Complications from that are what led to his death - he died of an antibiotic-resistant staph infection acquired in the hospital where he was surgery because of a blood clot in that leg.

Thanks. I guess you did say it was an example of how a false positive can do harm now that I think about it. In any case, that's really a shame.

Hey guys, for more information about all of this, I would suggest reading Gigerenzer, Gaissmaier, Kuz-Milcke, Schwartz, and Woloshin's 2008 paper "Helping Doctors and Patients Make Sense of Health Statistics." It is located in the journal Psychological Sciences in the Public Interest. It is a very well-written paper (an easy read), and it gives a good lit review of much of the current research in the area of statistical illiteracy and the potential health problems that it can pose. Have fun reading, and keep up the pursuit of knowledge guys!

While another great analysis, good luck in fighting the good fight. The reason I say this is because of conversations I've had with people in a much, much, much simpler example of probability in a real world problem. When should you guess on an SAT question? This was back in the early 90s when I took it. 1 Point for a correct answer. -1/4 for an incorrect answer. Five choices. Guess if you can eliminate one response as dead wrong. Some people could not realize this was not a penalty for guessing. The Princeton Review made a small fortune pointing this out to people.

Between 1: A lot of people not getting it and

2: How big a deal it was when The Princeton Review pointed this out to people, good luck seeing a much more sophisticated analysis enter into the popular conscience.

The "where the money goes" argument doesn't have a lot of applicability when it comes to something that almost everybody gets, like breast cancer screening. You can shift the costs around a bit so that men shoulder some of the costs for women, or the more well-to-do carry some of the costs for the poor, but basically the cost of the extra screening, plus overhead, just gets tacked onto your insurance premiums (or taken out of your salary if your employer pays, or--if we were to go to a more socialist health care system--tacked onto your taxes).

The only real virtue of having routine screening covered by insurance is that it gets people to do things that are good for them, because they're paying for them anyway, instead of being tempted to spend the money on something else that provides more immediate gratification. So it is worth considering the purely economic issue (even if, for the sake of the argument, you could save a few more people by spending a lot more money). How much would you spend to reduce your risk of dying of cancer by say, 0.1%? $100? $1,000? Would you sell your house? There is a cut-off point at which almost everybody would draw the line. And remember, it's not like you aren't allowed to sell your house to pay for the test, if it is really worth that much to you--you just aren't going to be forced to do so. The problem with covering such things by taxes, or insurance, is that it is easy to loose sight of the fact that under any plausible economic system, unless you are at an extreme end of the economic scale, it's going to be mostly your money paying for your tests. So people slip into grousing, "Those rich SOB's would rather spend money on big houses and fancy cars than spend a nickel to save my life" (which is likely true, but largely irrelevant)

While another great analysis, good luck in fighting the good fight. The reason I say this is because of conversations I've had with people in a much, much, much simpler example of probability in a real world problem. When should you guess on an SAT question? This was back in the early 90s when I took it. 1 Point for a correct answer. -1/4 for an incorrect answer. Five choices. Guess if you can eliminate one response as dead wrong. Some people could not realize this was not a penalty for guessing. The Princeton Review made a small fortune pointing this out to people.

I'd put it even stronger: always guess if you don't know the answer. You may know more than you think you do. And if you don't, you come out even on the average, anyway. Yet to this day, I still hear people saying, "Don't guess, because they take off points fro wrong answers." My nephew was given the "don't guess" advice in a test prep course just a few years ago.

Regarding the 'always guess' attitude - you aren't answering an infinite amount of questions, and while the expectation value of your result will stay the same (or rise, if you had [b]any[/b] information about the answer), the standard deviation will rise - giving you a better chance of getting a grade well below what you 'deserve'. (and a better chance of getting one much better than you deserve, but I fear being underrated more than I desire being overrated :))

Sorry for derailing the discussion. Interesting post, MCC.

"@12:

Monetary costs *are* a real and important factor to consider in things like this.

Mark, just ignore it. Gingerbaker, a commenter who is usually somewhat sensible, made a big stink over at Orac's blog with the same ludicrous argument. At one point, he/she actually asserted that the only reason "[r]esources are limited [is] because of collusion and lack of social justice." Seriously.

Normally a very sane commenter, but not on this issue."

-- Jamse Sweet

In case anyone happens to come back to this thread, as I just did, let me just say that according to you I caused a "big stink" and I am "not sane" on this issue? Hmmm... I think I made two posts there?

Seriously, James, go fuck yourself.