When popularity outpaces science in surgery

In science- and evidence-based medicine, the evaluation of surgical procedures represents a unique challenge that is qualitatively different from the challenges in medical specialties. Perhaps the most daunting of these challenges is that it is often either logistically impossible or unethical to do the gold-standard clinical trial, a double-blind, randomized placebo trial, to test the efficacy of an operation. After all, the "placebo" in a surgical trial involves exposing patients to anaesthesia, making an incision or incisions like the ones used for the operation under study, and then intentionally not doing the actual operation. Even leaving the ethics aside, it's impossible to blind the surgeons and operative team involved to which treatment, real surgery or placebo, the patient is receiving without having a different surgeon do the surgery from the one overseeing the postoperative care of the patient, with the operative surgeon barred from communicating to the postoperative surgeon what happened in the operating room and from participating in the postoperative care of the patient upon whom he operated. This sort of restriction, besides being also highly dubious on an ethical basis, goes against the grain of surgical culture, in which a surgeon is expected to provide the postoperative care for his patients as a matter of surgical honor. A final problem that complicates any surgical trial is that surgeons of differing technical operating skill will necessarily be involved, and surgical skill is indeed very important in determining outcome. Although there have been examples of double-blinded trials with sham surgery as placebo, for example, in injecting dopamine-producing cells into the brain to treat Parkinson's disease, difficulties doing such studies tend to force us as surgeons in many cases either to rely on retrospective data, prospective non-randomized data, or, when we're lucky, a prospective randomized (but not double-blinded) trial of one surgical procedure versus another.

Despite these difficulties, there have nonetheless been some great successes in applying rigorous science to the evaluation of surgical procedures. One of these examples comes from my own primary specialty: breast surgery. As recently as 30-40 years ago, most women with breast cancer were still subjected to radical mastectomies as originally pioneered by William Halsted in 1882. The radical mastectomy is a disfiguring operation that involves removing the underlying pectoralis major muscle along with the breast and axillary lymph nodes (the lymph nodes under the arm). The radical mastectomy is, as its name implies, a very radical procedure. It also results in considerable morbidity, which led to the search for less radical alternatives. The first of these was the "modified radical" mastectomy, which was essentially the same operation without the removal of the pectoralis major muscle. Then the question of whether it was necessary to remove the entire breast was examined, and there followed the concept of "lumpectomy" or "partial mastectomy" for small tumors. Consequently, in 1976, a rigorous trial was undertaken by the National Surgical Adjuvant Breast and Bowel Project (NSABP) to look at this question. The study, NSABP-B06 was designed to determine in patients with or without clinical axillary node involvement who may be amenable to segmental mastectomy (SM or lumpectomy) whether:

1. SM and axillary dissection with or without radiation of the breast is equivalent to total mastectomy plus axillary dissection,
2. cosmetically acceptable preservation of the breast in a subset of patients with primary cancer can be achieved without unfavorably influencing treatment failure and mortality rates as well as morbidity,
3. and to obtain evidence to indicate the clinical significance of microscopic multifocal tumor in the breast.

Let's go back in time some 30 years and imagine, if you will, trying to explain this trial to a woman with breast cancer and persuade her to be randomized to have either a modified radical mastectomy (total mastectomy with axillary lymph node dissection), a lumpectomy plus axillary dissection, or a lumpectomy with axillary dissection plus radiation. Remember, when a patient signs up for a trial like this, she agrees to accept whatever treatment to which she is randomized. In this case, that would mean not knowing whether the operation would be a lumpectomy or removal of the entire breast. Truly, women today with breast cancer owe a huge debt of gratitude to these women who agreed to such an arrangement because this trial showed conclusively that lumpectomy plus radiation therapy produced equivalent survival rates as modified radical mastectomy. Moreover, just the thought experiment of thinking how you might persuade a woman with a relatively small cancer to agree to be randomized to such a trial is a good way of driving home the difficulties inherent in even a fairly straightforward clinical trial of a surgical intervention. NSABP-B06 was an excellent example showing that rigorous science can indeed be applied to surgical procedures. Indeed, it still is, as the data from this trial are still being followed and analyzed, with periodic updates being published in the peer-reviewed medical literature.

Given the inherent difficulties in applying rigorous testing to new surgical procedures leads to two aspects of surgery that are exaggerated compared to medical specialties. The first aspect (the good one) is that there tends to be a remarkable degree of innovation and creativity among surgeons trying to develop new operations or more effective variations on old operations. These innovations can range from coming up with a new way to do an incision, position a patient, or do a stitch to clever new devices, surgical instruments, and even whole operations. The downside of this is that this combination of the natural tendency of surgeons to experiment coupled with the difficulty in rigorously testing tends to promote a "bandwagon" effect among surgeons when a new surgical procedure is developed, sometimes aided and abetted by one or a few charismatic or influential surgeons. An example of this latter phenomenon is Dr. R. J. Heald's tireless prosletyzing for total mesorectal excision (TME) for rectal cancer, in which he even went so far as to label proper surgical plane entered during the operation as the "holy plane." (I really, really hated that term, as you might imagine.) Good quality studies later supported the superiority of TME as a procedure, but this bandwagon effect nonetheless all too often leads to the widespread adoption of a procedure before studies show it to be safe or more efficacious than the procedure it is designed to supplant.

Perhaps the most striking example of this phenomenon occurred when I was doing my general surgery residency. In 1990, just before I entered the laboratory to work on my PhD, I was a second year surgical resident. Open cholecystectomy (gall bladder removal) was the standard of care for symptomatic cholelithiasis or cholecystitis. So many of these operations were done that they were considered a case for a second or third year surgical resident, and I had done quite a few of them before I entered the lab. At the time, I had heard of the new procedure that had been recently developed, the laparoscopic cholecystectomy, which was done through small incisions using a laparoscope, but I had never seen one and, as far as I knew, no one had done one yet at our institution. When I left the lab three years later to return to clinical surgery residency, laparoscopic cholecystectomy was the procedure of choice for symptomatic cholelithiasis. Suddenly, I was thrown into the O.R. as a third year resident never having handled a laparoscope and looking quite clumsy and ignorant compared to the crop of third year residents whom I had joined, all of whom had gone straight through residency without a stint in the lab and thus had learned how to handle the laparoscope during their first and second years.

What had happened? One thing I can tell you is that the reason laparoscopic cholecystectomy so rapidly supplanted open cholecystectomy in the early 1990s was not because rigorous studies showed it to be superior. What had happened is that surgeons and hospitals had seen a marketing opportunity and run with it. Ads for laparoscopic cholecystectomy proliferated. Older surgeons, who had not been trained in laparoscopic cholecystectomy, rushed to sign up for courses to learn how to do this procedure and had to be proctored through the "learning curve" of the case, which for experienced surgeons was estimated to be 25 cases. They had to. Cholecystectomy was (and is) "bread and butter" surgery for general surgeons, a large component of their case mix. Patients needing gallbladder surgery were going elsewhere, to surgeons who knew how to do the latest and (presumably) best operation.

While it was true that one almost didn't need a study to see that patients undergoing laparoscopic cholecystectomy suffered less postoperative pain and went back to work or activities of daily living much faster. What wasn't known was the complication rate. One of the most feared complications of any gallbladder or biliary tract surgery is an inadvertent injury to the common bile duct, the tube into which the gallbladder and the biliary tree from the liver empties, which leads to the duodenum, into which bile made by the liver is deposited. Such injuries didn't always show up right away and typically manifest themselves as a stricture causing obstruction to bile flow that, if not noticed in a timely fashion and allowed to become chronic, could lead to liver damage and cirrhosis. What has become apparent in the 15 years since then is that, although laparoscopic cholecystectomy has many advantages over conventional open cholecystectomy, it does has one downside. Since its popularization, the rate of common bile duct injury has risen from around 0.1%-0.3% to 0.4%-0.6%, with some old data reporting a 1%-3% rate with laparoscopic cholecystectomy. True, more recent studies suggest that the rate may now similar between open and laparoscopic cholecystectomy, but most surgeons agree that there is still at least a somewhat greater chance of common bile duct injury with laparoscopic surgery. Even so, laparoscopic cholecystectomy is now firmly entrenched as the standard of care for most gallbladder disease requiring surgery because it has in essence been accepted that the benefits of less pain and faster return to activity outweigh the increased risk of common bile duct injury by a large margin and because patients don't want the old procedure if they can possibly avoid it.

A more recent example of this phenomenon came across my desk last month in the form of a systematic review article in the May issue of the journal Colorectal Disease about the procedure known as transanal endoscopic microsurgery used for rectal cancer. The article, coming from the Academic Surgical Unit, Castle Hill Hospital, Castle Road, Cottingham, East Yorkshire, UK, was entitled Transanal endoscopic microsurgery in early rectal cancer: time for a trial? I can tell you one thing after having been "in the biz" for a while: It's a sure sign that a surgical procedure's usage has outpaced science and clinical trials to support it when you start seeing calls for trials in the peer-reviewed surgical literature.

So what is transanal endoscopic microsurgery (TEMS)? To understand its utility, it's necessary to understand a bit about rectal cancer. Larger rectal cancers definitely require one of two operations, a low anterior resection, which involves resecting the portion of involved rectum and putting the ends back together again, sometimes quite low and close to the anus, which makes this a particularly challenging operation for low-lying cancers. However, when the tumor is either too big or too low (or both), the other operation is known as an abdominoperineal resection (APR). This operation involves removing the entire rectum and anus and basically sewing the hole shut, necessitating the placement of a permanent colostomy. However, for small, early rectal cancers or for rectal adenomas that are too large to be snared using sigmoidoscopy or colonoscopy, traditionally, an acceptable substitute in selected patients has been to do what is known as a transanal excision. What this involves is dilating the anus and excising the tumor through the anus. It's a much less invasive operation, but it can be very difficult to perform, particularly if the tumor is too high (and thus out of easy reach of instruments placed through the anus) or too large. TEMS seeks to increase the range and ease of transanal excision thusly:

Transanal endoscopic microsurgery (TEM), pioneered by Professor Gerhard Buess, has extended the boundaries of transanal surgery by virtue of its longer reach and enhanced visibility. The device facilitating TEM is an endosurgical unit that regulates four different functions simultaneously -- namely, carbon dioxide in sufflation, water irrigation, suction, and monitoring of intrarectal pressure. The TEM scope itself is closed and airtight. When carbon dioxide is insufflated, the rectum distends and maintains itself in an 'open' state as long as there is not a leak within the system. This rectal distention greatly facilitates visualization, excision of the lesion, and precise closure of the wound. Water irrigation is used to cleanse the lens periodically, while suction re moves blood, smoke, and particulate matter. The rectal pressure is set at approximately 15 cm H₂O; the unit automatically adjusts the suction and rate of insufflation to maintain this level.

The rigid operating scopes are available in lengths of either 12 or 25 cm and have a diameter of 4 cm. The distal end of the scope is beveled and should face down at the lesion. This determines patient position. For a posterior lesion, the patient is positioned in the lithotomy position; and for an anterior lesion, the patient is positioned in the prone position. Colonoscopy should be performed before surgery to ensure no synchronous lesions are present. If TEM is to be used to remove a cancer, endorectal ultrasound is performed to determine the depth of penetration and hence the suitability for TEM. Patients undergo bowel preparation. The operation may be performed under either regional or general anesthesia. Most patients are treated as outpatients unless urinary retention or medication-related nausea requires admission.

While it's fairly clear that TEMS is an acceptable alternative procedure for nonmalignant adenomas that would otherwise be amenable to transanal excision, what is not clear is whether it is acceptable for invasive cancers or malignant polyps of the rectum. However, that has not stopped the procedure from being marketed as a better alternative to radical surgery for small cancers of the rectum when it is not clear yet whether it is. Indeed, the authors of this systematic review could only locate five trials comparing TEMS with more traditional operations for rectal cancer, and only two of them were randomized. The rest were retrospective studies, with all the capacity for selection bias and other biases that are inherent in the analysis of retrospective data. They topped this off by looking at 28 case series, which had no control group and didn't compare TEMS with anything. Case series are primarily useful for getting an idea of safety and possibly efficacy of a new procedure, but, because they have no control group and must therefore be weighed against historical controls, case series are generally considered a fairly weak form of clinical evidence.

As far as the prospective studies, one of them, published in 1996, examined 52 patients with small rectal cancers, of which 26 underwent TEMS and 26 underwent traditional LAR. TEMS was associated with shorter operative time (103 vs. 149 minutes), less blood loss (143 ml vs. 745 ml), and shorter hospitalization (5.7 vs. 15.4 days). This latter observations is remarkable, given that most patients undergoing LAR today stay in the hospital less than a week, making me wonder about the applicability of this study to today's practice. Unfortunately, the study was not adequately powered to detect differences in local recurrence and/or survival; yet this remained, as of 2007, the best study of TEMS versus LAR. The second study looked at 40 highly selected patients with larger tumors who underwent either TEMS or laparoscopic LAR, all of whom received neoadjuvant chemoradiation therapy (therapy before surgery designed to shrink the tumor and make its removal easier). This study also favored TEMS, but also was not powered to detect differences in overall outcomes. The study also showed a high incidence of side effects due to the neoadjuvant therapy, which could have confounded the results and is not really applicable to small tumors (known as T1), for which chemoradiation therapy is unnecessary. I will note that a more recent randomized trial was published, after the publication of this systematic review, and this study found comparable survival rates between groups of patients undergoing TEMS and LAR, although there was a trend towards higher local recurrence rates in the TEMS group.

Not surprisingly, the retrospective series are very difficult to interpret together, as are the case series. There was significant patient and tumor heterogeneity, different rates of full thickness excision (which is important for complete removal of the tumor), and different surgical indications used. Moreover, there was likely to be strong selection bias, in which patients with favorable tumors were more likely to be selected for TEMS, as would "high risk" patients with more medical problems who, it might be judged, would be unsuitable for a large operation (and who would be more likely to die of other causes before their tumors would recur). Complications associated with TEMS, though, included transient anorectal dysfunction and fall in anal sphincter tone resulting in impaired continence, but for most patients this was not a big problem. The overall "feel" of the studies was that TEMS probably offers fewer problems with continence and gastrointestinal function than more radical surgery.

Also, one problem with TEMS is cost. The instrumentation required is specialized and expensive. Given that, hospitals are unlikely to invest in this equipment unless it will be used. A lot. Certainly it can be argued that if sufficient evidence for the efficacy of TEMS can be produced then overall it might save money by decreasing morbidity and mortality, as well as length of stay and time off work. Even more certainly, having this equipment could be a nice marketing strategy for some hospitals. As was the case for laparoscopic cholecystectomy some 15-20 years ago, it would be all about marketing TEMS as an alternative to a big operation, namely APR or LAR. The problem is that we just don't have the evidence yet to make that decision or to justify that marketing, at least when it comes to rectal cancer rather than rectal polyps, which is why the authors of this review article conclude:

Results of current individual TEM series' are variable but comparative studies show that TEM can be used in selected patients. There is sufficient evidence to justify a prospective randomized trial comparing TEM +/- neo-adjuvant therapy against RR in selected patients and tumour types. Unlike previous trials, the outcome should not just emphasis cancer recurrence rates but should also include immediate and long-term morbidity, gastrointestinal function and quality of life. This trial will aid in deciding the optimal surgical procedure. No further information can be obtained by re-analysing previous studies or commencing further prospective individual case series. The time for a trial is now.

I agree. Even though there was a more recent trial since this systematic review, it only examined 70 patients. A single trial, particularly with so few patients, is rarely enough to settle a question. This is exactly the sort of issue that cries out for a large trial, and, indeed, it's rather disappointed that there have been so few rigorous trials since the 1990s.

I don't know that such a trial will ever be done, however. For one thing, it would be the equivalent of doing the trial of mastectomy versus lumpectomy 30 years ago in that it would be a comparison of a big operation (LAR or APR) with a smaller, less morbid, operation (TEMS). Getting patients to enroll would be a hard sell these days. I base my assessment on my experience enrolling patients for the NSABP-B32 trial. That trial occurred in the days, not so long ago, when the sentinel lymph node (SLN) procedure for breast cancer was rapidly supplanting axillary lymph node dissection (removal of all or most of the lymph nodes under the arm). This transition occurred in the late 1990s and early 2000s almost as fast as did the the transition from open to laparoscopic cholecystectomy and with almost as little rigorous data to support its superiority. However, surgeons wanted to avoid the complications of axillary dissection, particularly lymphedema, as did patients. The SLN procedure allowed that, being a procedure in which blue dye and/or radioactive dye is injected near the tumor, taken up in the lymphatics, and, conceptually, goes to the first draining lymph node a tumor cell that broke off from the main tumor into the lymphatics would encounter. This lymph node, the "sentinel" node would then be biopsied and the presence or absence of tumor in it used as a surrogate for the presence or absence of tumor in the axillary lymph nodes. If the SLN is negative, the axilla is considered to be negative. Axillary disseciton is only carried out in the case of a positive SLN. In any case, the B-32 trial randomized women to no treatment or full axillary dissection after SLN biopsy in order to determine the accuracy of the procedure for predicting additional positive nodes as well as the false negative rate (the SLN finds no tumor but there is tumor in other lymph nodes). Trying to persuade patients to enroll in this trial was not easy, because few of them were willing to be randomized to the arms that might require them to undergo an axillary dissection. Indeed, our NSABP coordinator took a hard line and said that we would not do SLN biopsy at our institution except under the auspices of B-32 and that patients who wanted SLN and were not willing to enroll in B-32 could go elsewhere. That restriction was not lifted until B-32 closed, and I lost more a few patients because of it. Private surgeons in the community were more than happy to accommodate them.

Fortunately, B-32 and other rigorous trials supported SLN as the new standard of care allowing most women with breast cancer to avoid axillary dissection today, with its complications of lymphedema, decreased shoulder range of motion, numbness under the arm and parasthesias along the upper arm, and possibility of nerve damage. I sometimes wonder, however, what would have happened if B-32 had shown that SLN was a highly inaccurate and inadequate tool to identify tumor in the axillary lymph nodes. By the time the trial closed, much less by the time the data were analyzed, SLN had almost completely supplanted axillary dissection in actual clinical practice for women with early stage breast cancer. I don't know if that could have been reversed in the event of clinical trial data showing marked inferiority of SLN. Fortunately, there was no need, as SLN is quite accurate for a surgical test and survival thus far appears to be equivalent between women undergoing axillary dissection and women undergoing SLN biopsy.

Surgery is unique as a specialty in that no other specialty depends so much on the technical skill of its practitioners and in that no other specialty is it as hard to do a true double-blind randomized trial. Indeed, the issue can be so complex that it has even been proposed that we distinguish between "tinkering" (a minor modification of a surgical procedure that does not produce a reasonable expectation of increased risk to the patient, which is the sort of thing surgeons do all the time), "innovation" (change in therapy designed to benefit an individual), and "research" (studies according to protocol in which the goal is to gain knowledge, not necessarily to benefit the individual being treated). These distinctions are, of course artificial, and the line between them is not always clear, but considering them can be a useful exercise in determining why surgical research is so difficult.

The bottom line, though, is that we as academic surgeons have to resist the temptation to jump on the bandwagon for seemingly "sexy" new procedures before strong evidence supports either their superiority over or equivalence to previous procedures. Surgery has had an amazing history of innovation and improvement, be it in the realm of cancer surgery, transplant, or cardiovascular disease, but sometimes innovation runs way ahead of science. Academic surgeons have to try to insist, as much as is possible, that our treatments be science- and evidence-based before widespread adoption, balancing that against holding on to old operations longer than we should have, as was probably the case with the radical mastectomy. Unfortunately, in our present medical system, with its attention to the bottom line and the competition among hospitals and surgeons for an edge, such devotion to scientific rigor can be very difficult to maintain, but we abandon it at the risk of losing the right to call surgery science-based.

REFERENCE:

Suppiah, A., Maslekar, S., Alabi, A., Hartley, J.E., Monson, J.R. (2008). Transanal endoscopic microsurgery in early rectal cancer: time for a trial?. Colorectal Disease, 10(4), 314-327. DOI: 10.1111/j.1463-1318.2007.01448.x