| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Editorials |
Center for Applied Proteomics and Molecular Medicine, George Mason University, Manassas, VA
aAddress correspondence to this author at:, 10900 University Blvd, Room 181A Discovery Hall, Manassas, VA, 20155, Fax 703-402-4288, E-mail lliotta{at}gmu.edu
The discipline of mass spectrometry (MS)-based serum biomarker profiling is a relatively young field that was launched in a flourish of scientific hope and the clinical promise of a better cancer test (1)(2)(3)(4)(5). Nevertheless, the field has already undergone a rollercoaster cycle of optimism and disappointment and renewed enthusiasm during its brief 7-year history (1)(2)(3)(4)(5)(6)(7)(8). Before 1999 very little effort was made to directly use MS (MALDI-TOF or ES) as a means to discover new blood biomarkers. Many scientists thought that serum was too complex for MS analysis, and dominated by contaminants, rendering these samples unacceptable for direct introduction into expensive and sophisticated MS research instruments. All of these arguments seemed to be overturned in 1998 when a new class of MALDI-TOF, surface-enhanced laser desorption and ionization (SELDI) was commercialized.
Scientists with no formal training in MS saw SELDI as an opportunity to explore the application of MALDI MS to biomarker discovery. Although this method had relatively low resolution, it appeared to provide a fresh, one-step approach to the search for ion signatures of hundreds of candidate biomarkers. Importantly, SELDI-TOF offered a means to support a critical new hypothesis that was emerging in the protein biomarker field. This hypothesis abandoned the assumption that a single specific tumor cell–derived cancer biomarker existed. Instead, cascades of biomarkers were generated from the tumor tissue microenvironment through interactions between the tumor cells and the host cells (e.g., endothelial cells, stroma cells, and immune cells) (1)(5). The tissue-microenvironment hypothesis predicted that a panel of biomarkers could achieve a sensitivity and specificity superior to previous failed cancer biomarker searches (1)(5)(6)(7)(8). In 2002 SELDI provided a means for discovering the ion signatures of these putative biomarkers, although their identity was unknown (1)(2).
During the past 5 years, a large number of scientists were able to identify candidate protein disease biomarker profiles using patient research study sets and to achieve high diagnostic sensitivity and specificity in blinded test sets (1)(2)(5)(6)(7)(8). Nevertheless, translating these research findings to useful and reliable clinical tests has been the difficult part. Clinical translation of promising ion fingerprints has been hampered by sample collection bias, interfering substances, biomarker perishability, laboratory-to-laboratory instrument variability, SELDI chip discontinuance and surface lot changes, and the stringent dependence of the ion signature on the subtleties of the reagent composition and incubation protocols. These difficulties are exemplified by the 2 reports published in this issue by McLerran and colleagues (9)(10). Both papers describe the unsuccessful attempt to validate a specific SELDI-based biomarker ion fingerprint, and algorithm, for the diagnosis of prostate cancer. The original SELDI-TOF method and set of ion peak mass/charge values in question was described in 2005 (11). The validation process that generated these 2 reports was conducted under a rigorous system developed by the National Cancer Institute Early Detection Research Network (EDRN). These 2 reports are models of biomarker validation and demonstrate a tremendous return on the NIH investment supporting the EDRN process. Too often we focus our attention only on the successful experimental studies. In the highly competitive biomedical field, a manuscript describing an unsuccessful validation trial may not be considered to have impact warranting publication. It is commendable that the Clinical Chemistry peer review process recognizes the value of this information for the biomarker community.
McLerran et al. studied the sources of variability and bias during the use of the SELDI-TOF prostate cancer (PCa) algorithm (10). They report a significant sample bias that may have been the source of algorithm failure. The investigators noted that PCa serum cases collected before 1996 (64%) exhibited considerably different spectral profiles than cases collected in 1996 or later. Those PCa cases collected after 1996 contained spectral profiles more similar to normal control samples, and all but one of the control samples were collected after 1996. The authors conclude that differences associated with sample age may result from serum degradation related to storage time or the number of freeze-thaw cycles. In a companion paper (11), McLerran et al. report the results of a retrospective study that was designed to minimize the confounders (such as sample storage) described in the 1st paper. The authors conclude that the SELDI-TOF serum profiling algorithm for PCa described in 2005 does not hold up in a new retrospective cohort of 400 patients. The authors conclude "bias in serum specimens of earlier studies, differences in study design, and limitations of proteins detected by SELDI-TOF MS applied to unfractionated serum may explain the inability of this validation study to identify men with PCa." The authors also state that "One should not conclude from our studies that a particular method does not work or that previous studies were wrong." The authors emphasize that the technique can be made reproducible across multiple laboratories, "but that the failing is specific to failure of the targeted ‘peaks’ to discriminate." Finally, the authors recommend that all subsequent biomarker candidates should be subjected to equally rigorous validation. Moreover, the authors express concern about the difficult and time-consuming process required (2 years in this case) to collect enough "ideal" unbiased samples to accommodate the discovery process. One overall lesson to be learned is that inadequate attention, up front, to potential sources of bias in the training samples increases the probability of false discovery.
Indeed the real conclusion of these papers is not that the algorithm or the platform failed, but that the specifically selected ions, specific to the chip surface employed and the binding protocol conditions, were not robust and did not transcend sample variability. However, other investigators using different MALDI platforms, capture surfaces, and sample fractionization have recently described specific ion fingerprints that appear to contain diagnostic information and that hold up over time and over independent blinded sample sets (6)(7)(8)(12)(13). For example, Belluco et al. recently reported an ion classifier for detection of early-stage breast cancer that was robust across both blinded independent validation and independent prospective sample sets run 14 months later using the original ion classifier (12). Thus the utility of MS fingerprinting from body fluids for disease classification remains a very viable and attractive approach.
The outcomes of these 2 validation studies highlight technological obsolescence and the rapid pace of changing MS technology as a serious issues for the field of biomarker science. This point is not adequately covered in the articles. For proper clinical validation, the diagnostic protocol and the instrument must be fixed and standardized. By the time adequate clinical samples are accrued for a prospective validation (3 years in this case), the technology has moved forward several generations. Thus the validation is constrained to use a technology that is not even the current state of the art. Commercial MS technology is highly competitive, and new instruments and technologies are emerging several times a year. Consequently, a diagnostic ion fingerprint profile that is strictly dependent on a particular MS technology, specific capture-surface chemistry, and a precise sample handling and processing protocol will be obsolete in a few years. Moreover, the requirement for a highly specific platform and processing protocol will hinder independent laboratory confirmation. The solution to this problem is to generate a diagnostic biomarker readout that is independent of the measurement platform. In this case, sequencing and identification of the proteins or peptides underpinning the diagnostic peaks renders the output independent of the measurement platform. Once the proteins or protein isoforms are identified, then they can be measured by any suitable immunoassay or analytical system now or in the future. Instead of a pattern of unknown ions, the diagnostic test is based on a panel of known molecules. This approach is now the major one used for biomarker discovery. The output of an MS biomarker discovery workflow has become a list of specific identified proteins, or protein isoforms, that are differentially abundant between cases and controls (13)(14). Currently, with properly validated antibodies, the identified proteins can be validated by any suitable current immunoassay platform. This does not mean that mass spectrometry based profiling will become obsolete. A significant impediment to immunoassay-based diagnostics is the requirement of a well-performing specific antibody (4)(5). MS technologies such as multiple reaction monitoring, immuno-MS, and high-resolution MS profiling may not require a well-performing antibody to confidently read out the analyte identity in the clinical arena (5)(14). Indeed, as MS technology, analyte labeling, and amplification technologies advance, it is not unreasonable to envision a future in which MS can play a dominant role in the clinical diagnostic laboratory. Of course, any clinical assay, regardless of the platform or technology, must meet the strict CLIA requirements and are expected to meet basic aspects of performance and validation as laid out in expected elements of method validations (15). The transfer of marker assays to technologies such as immunoassay or multiple reaction monitoring–based MS are in part a move toward more robust quantitative methods that can meet the validation, calibration, and QC requirements of a clinical laboratory (15). The guiding principle for investigators pursuing biomarker discovery is to ensure that the biology, and the biomarkers themselves, remain independent from the changing technology.
Acknowledgments
Grant/funding Support: Aspects of our ongoing biomarker work are funded by the ISS-GMU/US-ITALY Cancer Proteomics Program, and the funding support of George Mason University.
Financial Disclosures: The authors are inventors on US government and university-assigned patents and patent applications that cover aspects of the technologies discussed. As inventors, they are entitled to receive royalties as provided by US Law and George Mason University policy.
References
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
