"A Catastrophic Failure" - More on the Ketek Trial Debacle

We previously posted (also here, here and here) about the ill-fated clinical trial, study 3014, of the antibiotic telithromycin (Ketek) made by Sanofi-Aventis, run by Pharmaceutical Product Development Inc. (PPD). Problems with the trial included fabrication of data at one clinical site, and allegations of manipulation of data at another. The physician in charge of the first site was convicted of mail fraud, and the physician in charge of the second had his license suspended. Although the results of this trial were never published, it still crept into the clinical literature: it was cited in a review article in the New England Journal of Medicine.

The case is now the subject of a US congressional investigation. As reported by the Associated Press (via the International Herald Tribune),

Democrats in the U.S. House of Representatives are threatening to hold a member of President George W. Bush's Cabinet in contempt for not turning over documents in a probe of a Sanofi-Aventis antibiotic.

Dingell, who chairs the House Energy and Commerce Committee, said he would support holding Health and Human Services Secretary Michael Leavitt in contempt for refusing to turn over FDA briefing documents subpoenaed by the committee.

The documents were used to prepare FDA Commissioner Andrew von Eschenbach for his appearance before lawmakers last year. Von Eschenbach testified the FDA did not use the flawed safety study to approve Ketek. Dingell and other Democrats say that statement may be untrue.

Robert West, an FDA agent who first investigated Ketek, said he tried to get permission in 2002 to look into whether Aventis was aware of fraudulent data when it submitted the study. West said his request was blocked by senior FDA officials, although he said he did not know which ones.

"A catastrophic failure" was how another FDA investigator, Douglas Loveland, described the company's handling of the study.

"The decision-making process Aventis used to investigate these problems was illogical and ineffective and it could have led them to come to the wrong conclusion," Loveland told House lawmakers.

The investigator stopped short of saying Aventis, Sanofi-Aventis' corporate predecessor, knew the results were false when they submitted them but said it "should have known." Loveland said the results contained all the hallmarks of sham data, including forged signatures and crossed out information.

Sanofi President for Research and Development Paul Chew testified that the company submitted the study in "good faith."

The company was not able to spot the fake patient results....

It seems clear that this case involved a very sloppy clinical trial sponsored and run by a pharmaceutical company. Such sloppy science breaks promises made to human subjects of clinical research that their participation would go to advance science and improve health care. Sloppy trials that produce dubious data will do neither. Furthermore, sloppy science that produces dubious results provides patients and doctors pseudoevidence rather than the best possible evidence relevant to decisions about health care interventions, and hence may also lead to bad decisions that harm patients.

It is bad enough that clinical trials sponsored by pharmaceutical, biotechnology, and device companies are often sloppy, and designed to show their products in the most favorable light, rather than provide valid and unbiased data testing hypotheses most important for patients' and physicians' decisions. This case now raises the more disturbing possibility that high government officials entrusted with protecting the public's health and safety may have hid the sloppiness and unreliability of the Ketek trial. Time and further investigation will tell if this possibility is more than speculation. But the latest events in the Ketek saga clearly show the need for more open and transparent clinical research on drugs and devices, and more open and transparent regulation of such companies by the government.

New Black-Box Warnings and Congressional Hearings on Ketek: "Whose Interest is Being Protected?"

It has not been a happy few days for the pharmaceutical industry.

We previously posted (also here) about the ill-fated clinical trial, study 3014, of the antibiotic telithromycin (Ketek) made by Sanofi-Aventis, run by Pharmaceutical Product Development Inc. (PPD). Problems with the trial included fabrication of data at one clinical site, and allegations of manipulation of data at another. The physician in charge of the first site was convicted of mail fraud, and the physician in charge of the second had his license suspended. Although the results of this trial were never published, it still crept into the clinical literature: it was cited in a review article in the New England Journal of Medicine. Nonetheless, the US Food and Drug Administration (FDA) approved Ketek for use in even relatively mild infections. Back in June, 2006, US Senator Charles Grassley (R-Iowa) announced his intention to figure out why.

On Monday, the US Food and Drug Administration announced that Ketek would no longer be approved for sinusitis and bronchitis, relatively mild infections for which many other antibiotics are available. It also announced more "black-box" warnings for Ketek. (See this report by Bloomberg News, for example.) In preparation for a sub-committee hearing in the US House of Representatives about FDA oversight in this and other cases, Representative Edward Markey (D- Massachusetts) called these warnings "overdue," and suggested there was "no coincidence" about the timing of the FDA announcement prior to the committee hearing. The Associated Press (here via the Washington Times) quoted Markey, "it appears that a healthy dose of congressional oversight has reminded them to do their job and ensure the safety of drugs on the market." The AP also quoted Senator Grassley, "the FDA's action today shows what transparency can do. When a spotlight was turned on the questionable way in which Ketek got approved by the FDA for certain sinus and lung infections, the FDA was held accountable."

Today, reports on the hearings first appeared. Per Reuters,

U.S. lawmakers criticized the Food and Drug Administration's monitoring of the risks of the antibiotic Ketek (Sanofi-Aventis) and some other prescription drugs on Tuesday.

At a hearing on the FDA's drug safety oversight, lawmakers said the agency stifled dissent about the risks of drugs when top managers disagreed. They cited problems with Ketek, Vioxx (Merck & Co. Inc.'s withdrawn arthritis drug) and antidepressants made by several companies.

'With each of these drugs, it appears that the FDA is not seriously questioning whether the risks outweigh the benefits of the new drug,' said Rep. Bart Stupak, a Michigan Democrat who chairs a subcommittee of the House of Representatives Energy and Commerce.

'One must ask, if the FDA is not protecting its client, the American people, whose interest is being protected?' Stupak said.

Iowa Republican Sen. Charles Grassley, a vocal FDA critic, told the committee that 'scientific dissent is discouraged, quashed and sometimes muzzled' inside the agency and 'there's widespread fear of retaliation for speaking up about problems.'

And Anna Wilde Matthews, again in the Wall Street Journal, reported,

A former employee of the Sanofi-Aventis SA contractor that oversaw a clinical trial of the antibiotic Ketek told Congress the drug maker knew there was evidence of a problem with the study before regulators discovered fraud.

The testimony of Ann Marie Cisneros, a former employee of contractor Pharmaceutical Product Development Inc., came as the House Energy and Commerce Committee opened hearings into how the Food and Drug Administration handles drug-safety issues.

Committee Chairman John Dingell, a Democrat from Michigan, called the FDA 'badly broken,' and Rep. Bart Stupak, another Michigan Democrat and chairman of its investigations subcommittee, said the FDA's interactions with the drug industry had become 'incestuous.'

[For balance,] In a statement, Sanofi-Aventis said it was 'unaware at the time' that what appeared to be deviations at the site 'in fact reflected fraud in the conduct of the study.' The company said Aventis 'acted in good faith' in conducting the study and has cooperated with investigations.

The FDA, which had none of its leaders testifying, said in a statement that it 'shares Congress' commitment to drug safety issues' and the agency has announced a number of recent drug-safety initiatives. The agency said it followed 'standard procedures in identifying and communicating risks associated with Ketek.'

Pharmaceutical Product Development said it complied with FDA regulations and its contract with Aventis and reported all issues in the trial to Aventis.

In summary, it appears now that the common wisdom about Ketek was based on "pseudoevidence." (See our relevant post on "pseudoevidence-based medicine.") Unfortunately, these sorts of stories suggest that patients and physicians need to be very skeptical about the integrity of the evidence used to support use of the latest, greatest, and most expensive pharmaceuticals. If the drug industry wants to improve its sagging reputation, it will have to show that it can clean up how it does clinical research. Or perhaps we ought to think about getting drug (and biotechnology and device) companies out of the business of sponsoring clinical research on products in which they have vested interests? Of course, such a change will threaten quite a few peoples' sources of incomes, so we can expect plenty of resistance to it.

More Stories About the Fall of the FDA

Related, disturbing stories about how far the US Food and Drug Administration (FDA) has fallen have just come out.

Ketek Approval

We previously posted about the ill-fated clinical trial, study 3014, of the antibiotic telithromycin (Ketek) made by Sanofi-Aventis, run by Pharmaceutical Product Development Inc. (PPD). Problems with the trial included fabrication of data at one clinical site, and allegations of manipulation of data at another. The physician in charge of the first site was convicted of mail fraud, and the physician in charge of the second had his license suspended. Although the results of this trial were never published, it still crept into the clinical literature: it was cited in a review article in the New England Journal of Medicine. We also posted about how US Senator Charles Grassley, (Republican - Iowa), chair of the Senate Finance Committee, alleged that the FDA was covering up the process it used to approve Ketek.

Now the New York Times reported that Dr David Graham, now known as an internal FDA whistle-blower, and other FDA scientists challenged the initial approval of Ketek, to no avail. Graham wrote:

It’s as if every principle governing the review and approval of new drugs was abandoned or suspended where telithromycin is concerned.

The Times reported that four other agency officials, including Dr Charles Cooper, Dr David Ross, and Dr Rosemary Johann-Liang, "expressed serious reservations about Ketek."

Senator Grassley commented:

It’s no surprise to learn that the F.D.A. didn’t listen to Dr. Graham on the dangers of Ketek.

The F.D.A. has made it their business to discredit Dr. Graham and others who aren’t willing to cater to the drug companies.

FDA spokesperson Susan Bro countered, "Every issue or question raised during the Ketek review process and subsequently since approval has been rigorously reviewed by the nation’s best physicians, statisticians and epidemiologists both internal and external to the F.D.A."

Vioxx Aftermath

The Associated Press reported (here in USA Today) that Senator Grassley also "asked the inspector general at the Health and Human Services Department to probe whether the Food and Drug Administration and Merck acted in concert to call into question the safety findings made by Dr. David Graham, an FDA drug safety official." The report continued,

In a letter Wednesday, Grassley cited handwritten notes made by the Merck employee documenting an Oct. 13, 2004, conversation with the FDA official that suggests the two collaborated.
The FDA official mentioned an 'opportunity to get (the) message out' on Graham, a longtime employee of the agency, and provide journalists with a company critique of him, according to notes quoted in the letter.

'It is no secret that Dr. Graham was and is a critic of the FDA. However, that does not mean the FDA should scheme with drug sponsors to discredit its own employees,' Grassley said in the letter to Inspector General Daniel Levinson. The FDA, Grassley said, must maintain a 'clear, bright line between the regulated and the regulator.'

This time, FDA spokesperson Bro "had no comment."

Further details provided by AP were:

FDA e-mails seen by The Associated Press indicate that the agency shared in advance with Merck details about a presentation that Graham was to make in France in August 2004 about the dangers of Vioxx. The e-mails suggested that such a practice was commonplace.

Merck then issued a statement saying it stood by the safety of Vioxx. An FDA spokeswoman at the time said removing the drug was 'not on the table.'

The notes excerpted by Grassley indicate the FDA later went even further in helping Merck rebut Graham's work.

The FDA's Dr. Brian Harvey suggested to Merck's Dr. Ned Braunstein 'an official rebuttal on Graham,' according to the notes, which were admitted as evidence in a federal Vioxx trial.

Graham said he was 'quite shocked' to learn about Braunstein's notes.

FDA Employee Survey

The Baltimore Sun reported on a survey by the Union of Concerned Scientists of FDA scientists. The survey was sent to 5918 people, of whom 997 responded. The Sun reported that key findings were:

• "Fifteen percent of the 997 FDA scientists who answered the questionnaire said they were asked to keep information out of agency documents or alter their conclusions for nonscientific reasons."
• "Nearly one in three said the FDA doesn't routinely provide complete and accurate information to the public."
• "37 percent said the agency's leadership wasn't as committed to product safety as to approving products for sale."
• "Fifty-two percent said their job satisfaction had fallen the last few years, and 70 percent said the agency lacked the resources to carry out its mission."
• "two out of five scientists saying superiors didn't consistently stand behind staff whose "scientifically defensible positions" might have been politically controversial."
• "Eighty-one percent of respondents said the agency needed to strengthen its oversight of drugs after they go on sale."

Responses were bi-partisan. Senator Barbara A Mikulsi (Democrat - Maryland) responded, "This agency has been politicized and degraded. Many FDA employees don't feel the FDA is doing enough to protect the public's health and are afraid to speak candidly about it." Senator Grassley responded that the FDA needs "major overhaul and a culture change at the highest levels." He also said it "needs to re-establish its relationship with its own scientists and distance itself from the drug industry. The FDA needs to get rid of its mindset that it's a facilitator for the drug industry and become regulator once again. The FDA's focus should be only on science and the public good."

Spokesperson Bro responded by criticizing the "scientific rigor" of the survey, and charging "This is a counterproductive exercise based on leading questions and innuendo. For centuries, science has depended on rigorous and disciplined processes to distill truth from exploration and debate -these principles above all others guide our daily work at the FDA on behalf of the American public health."

By the way, regarding technical issues about the survey: I agree that a response rate of less than 20% does raise concern about the generalizability of the results of the survey. However, given the content of the responses (and given the content of some written comments made on the sruvey), these results are still very disturbing. And I'm not sure on what Ms Bro's charge that the survey questions were "leading" was based.
Unfortunately, these stories make it even more imperative for doctors and patients to be extremely skeptical about what they are being told about pharmaceutical products, even by the formerly respected FDA.
These stories added together suggests that Senator Grassley's call for a complete overhaul of the FDA is on target. Unfortunately, the FDA, once regarded as a paragon of integrity (albeit one that may have been ponderous, bureaucratic, and sometimes inefficient), seems to have fallen into the same muck that too many other health care organizations, from hospitals to to managed care organizations to pharma and device manufacturers now occupy. Thus now doctors and patients must be extremely skeptical about nearly anything they hear about health care when it is said by someone who could possibly have vested interests other than improving patient care or the integrity of science.

We have a lot of overhauling and much-raking to do. Meanwhile, we have to work in a health care system in which doubt, ambiguity, and skepticism rule.

NOTE: See the takes on this by Pharma Gossip here, here, and here. See a related story by Dr Aubrey Blumsohn on the Scientific Misconduct blog here.

Senator "Smells a Coverup" About How FDA Approved Ketek

We previously posted about the ill-fated clinical trial, study 3014, of the antibiotic telithromycin (Ketek) made by Sanofi-Aventis, run by Pharmaceutical Product Development Inc. (PPD). Problems with the trial included fabrication of data at one clinical site, and allegations of manipulation of data at another. The physician in charge of the first site was convicted of mail fraud, and the physician in charge of the second had his license suspended. Although the results of this trial were never published, it still crept into the clinical literature: it was cited in a review article in the New England Journal of Medicine.

The Washington Post just reported on the failed efforts of US Senator Charles Grassley, (R-Iowa), Chair of the Senate Finance Committee, to get more information about how the US Food and Drug Administration (FDA) approved Ketek. The story is certainly colorful:

The agency acknowledges that there were major improprieties in a 25,000-patient clinical trial done for the drug's developer, Aventis. One person has gone to prison for fraud in connection with that trial, data from several other trial sites have been discarded because basic standards were not met, and the agency told the company it could not say anything about the trial results on its product label.

Nonetheless, the FDA allowed Aventis to use much of the data to support the company's contention that the drug is safe. It also allowed Aventis to present the data to an FDA advisory panel without telling its members of the widespread fraud allegations clouding the trial. Documents show that some FDA employees argued that the trial should have been discarded because of the company's inadequate oversight.

Grassley has pressed to speak with an agent in the FDA's Division of Scientific Investigations who, he said, 'is key to understanding what the FDA did when it became clear that the safety study required by the FDA in order to approve the drug was fraudulent and faulty. Did FDA managers turn a blind eye and let the drug maker off too easy, or did the FDA do the right thing?'

In reply, FDA spokeswoman Susan Bro said: 'We would like to do whatever we can to help [Grassley] fulfill his constitutional duty as well as ours to the American people. However, we will not compromise an ongoing investigation.'

After months of trying to get firsthand information from a government official familiar with a controversial new antibiotic, Sen. Charles E. Grassley (R-Iowa) marched into the Department of Health and Human Services headquarters yesterday asserting his congressional right to receive the data.

After a brief meeting with senior HHS and Food and Drug Administration officials, Grassley departed empty-handed and angry.

'This is extraordinary for me,' the senior Republican said outside the headquarters. 'I haven't had to go to an agency like this since 1983 to get information I requested.'

'I smell a coverup.'

'I'm tired of the runaround,' he said after leaving the HHS headquarters. He said that he told the agencies' officials that 'I know it's probably not your intent to protect companies, to cover for companies.'

'But I said that every time that you stand in the way of information getting out that ought to be public, that's the impression you give the American people.'

It is noteworthy how this story has evolved. Originally, it seemed to be about sloppily supervised research whose unpublished results still had influence. Now the story seems to be morphing into one that includes allegations of government cover-ups of information about how questionable commercially sponsored clinical research may be. The moral is that patients and physicians need to be extremely skeptical about the results of commercially sponsored clinical research.

As we have said before, doctors and patients ought to base choices of tests or treatments on honestly done, honestly reported clinical research. Depending on poorly executed or misleadingly reported research will lead to bad decisions and bad outcomes. Furthermore, failure to honestly report the results of clinical research betrays the trust of the human volunteers who willingly participated in the research partly to help add to scientific knowledge and improve clinical care.

The Saga of Study 3014 and the Safety of Telithromycin

An article in the Wall Street Journal by Anna Wilde Matthews (subscription not required to access article) raises questions about the integrity of the clinical research evidence about the antibiotic telithromyci (Ketek) made by Sanofi-Aventis.

Background

Telithromycin is a ketolide, a molecular modification of erythromycin with some similarity to macrolide antibiotics. In the US, it is approved by the Food and Drug Administration (FDA) for treatment of outpatient upper respiratory infections and pneumonia.

A brisk review of the drug is found in this editorial in the Annals of Internal Medicine: Turner M, Corey GR, Abrutyn E. Telithromycin. Ann Intern Med, Mar 2006; 144: 447 - 448. This editorial notes concerns about the liver toxicity of the drug, based in part on a case report of three patients in the same issue ( Clay KD , Hanson JS, Pope SD, et al. Severe Hepatotoxicity of Telithromycin: Three Case Reports and Literature Review. Ann Intern Med, Mar 2006; 144: 415 - 420.). One patient required a liver transplant, and one died. The case report also noted that Sanofi-Aventis reported seven cases of hepatitis or hepatocellular damage in patients taking telithromycin in data from Phase III trials. Also, the FDA received ten post-marketing surveillance reports of serious liver problems in patients taking the antibiotic, and often, other drugs.

The Wall Street Journal Article

The article focused on a large randomized controlled trial, study 3014, initiated by Aventis (to become Sanofi-Aventis) in 2001. According to the WSJ article, "Aventis first sought permission to sell Ketek in the U.S. in March 2000. Fifteen months later the FDA refused to approve it." So, "Aventis originally undertook study 3014 in 2001 at the request of the FDA, which was worried about liver damage, blurry vision and other possible side effects from Ketek after reviewing the company's earlier trials." Then, "Aventis hired a contractor called Pharmaceutical Product Development Inc. (PPD), which specializes in coordinating clinical trials." The study was designed to enroll patients with respiratory infection seen in the offices of 1824 primary care physicians. Patients were randomized to telithromycin or amoxicillin/clavulanate potassium (Augmentin).

Problems were soon discovered at study sites that enrolled the most patients.

• The study eventually enrolled 407 patients from the office of Dr Maria "Anne" Kirkman Campbell. Her practice "attracted patients by advertising weight-control treatments." By January, 2002 she was enrolling "30 new people a day." Minutes from a PPD study management meeting stated that someone was a "little uncomfortable" with the site, which required "additional monitoring." In February, 2002, Nadine Grethe, "an Aventis manager overseeing the study," got an email from PPD warning of problems at the site with lack of "proper diagnosis of an appropriate medical condition" for study patients, and that medical charts were "very limited," and laboratory test results "suspiciously similar." A statistical analysis by Aventis failed to indicate problems with the data. "When Aventis turned in the results of study 3014 to the FDA on July 24, 2002, they included 407 patients from Dr. Campbell. At this point, 'Aventis did not alert the Agency to any problems....'" Yet when an FDA inspector examined Dr Campbell's office in the fall of 2002, chosen simply because of the volume of patients enrolled there, problems found included "patients [who] said they hadn't gotten any medication," patients "who were allegedly being treated fro weight loss, and not respiratory infections," and some who "were family members and friends of Dr. Campbell." Later, "Sanofi-Aventis says it was only after the government investigation that it discovered Dr. Campbell was fabricating data." Dr. Campbell eventually plead "guilty of one count of mail fraud in March, 2004 and was sentenced to four years and nine months in federal prision."
• The site with the third greatest number of patients, 214, was that od Dr. Egisto Salerno. His "medical license was on probation during the study." "Aventis told the FDA in December 2002 that it didn't know Dr. Salerno was on probation." An FDA inspection found "use of white-out on some study documents." Seven weeks after ending study enrollment, "police found Dr. Salerno with cocaine in his underwear and a loaded handgun," and he eventually surrendered his medical license, and plead guilty to a misdemeanor, which was later expunged after community service and drug counseling.

However, "when a committee of outside adviers to the FDA met early in 2003 to weigh a recommendation on Ketek, agency officials didn't mention the problems turned up by its inspections. The FDA's Dr. [Janice] Soreth and Dr. Jenkins say revealing the suspicions might have biased the decision and impaired the investigation. The committee voted to recommend Ketek's approval. Two weeks later the FDA rejected the recommendations. It asked Aventis for more documents on study 3014 and potential side effects overseas. Aventis complied. But the FDA ultimately decided the study was so flawed that the data couldn't be trusted." After considerable internal debate, "the FDA formally approved Ketek on April 1, 2004...." "FDA officials said they believed the original Aventis data submitted in 2000, plus the data from smaller studies and the drug's safety record oversea, justified approval."

The WSJ article noted that study 3014 was cited in an "article in the New England Journal that suggested Ketek is as safe as other antibiotics. Five of the six authors of that article disclosed that they received consulting fees from Sanofi-Aventis, and the sixth was an Aventis employee at the time of the study." The article mentioned in the WSJ appears to be this report of a randomized controlled trial of talithromycin versus placebo for patients with acute asthma ( Johnston SL, Blasi F, Black PN, Martin RJ, Farrell DJ, Nieman RB, the TELICAST Investigators. The Effect of Telithromycin in Acute Exacerbations of Asthma. N Engl J Med 2006; 354:1589-1600.) In apparent reference to safety information derived from study 3014, Johnston et al concluded, "among patients with normal liver aminotransferase levels at entry, the incidence of elevations of at least three times the upper limit of normal after treatment was similar among patients receiving telithromycin and drugs used for comparison."

The WSJ quoted US Senator Charles Grassley (R-Iowa, and Chair of the Senate Finance Committee), "the Ketek allegations appear to be as serious as anything I've seen so far." Additionally, US Representatives Edward Markey (D-Massachusetts), and Henry Waxman (D-California) are separately investigating. Rep. Waxman said he is "deeply disturbed," and that Aventis "failed to disclose to FDA grave flaws in a key safety study."

Summary

At best, this appears to be yet another story about a drug trial that was poorly executed by a contract research organization, and not rigorously supervised by a pharmaceutical company. We have previously discussed apparently sloppy work by other contract research organizations (see post here). Furthermore, results from this poorly executed trial seem to have been used to suggest that telithromycin is relatively safe, even though the integrity of the data provided by this trial is questionable.

The story of study 3014 appears to be yet another cautionary tale about how the clinical evidence that physicians and patients rely upon to make clinical decisions must be regarded with skepticism. Our skepticism about particular research studies needs to be extended not just to the usual considerations of study design, the nature of the data collected, and the appropriateness of the statistical analysis. We must now be skeptical about the details of the study execution, particularly when the study has been done by a contract research organization, and worry about problems that may go beyond just honest mistakes.