RNA Reference Materials for Gene Expression Studies
While walking up several flights of stairs in an elevator-deprived building recently, my colleague, between some gasps of breath, turned to me and exclaimed that he had to start his New Year’s resolution and begin exercising again. At that moment this seemed to me to be a very sensible thought, given his fish-like gasping and chameleon-like color changes. However, I also recalled that my colleague had a history of serious asthma and a travel schedule that rivaled most airline pilots. I asked him how he would be managing these challenges while seeking a more healthy lifestyle. He smiled at me and said that he would deal with these issues later. “Look”, he said, “at least getting some exercise is a start”.
The first steps in transitioning microarray- and quantitative reverse transcription-PCR (QRT-PCR)-based human-genome-wide RNA expression profiling from the current, primarily research, applications to the more exacting medical diagnostic and drug development arenas were made recently. The NIST organized a meeting in March 2003 (1) that focused singularly on establishing the types and properties of universal RNA reference materials to be used for expression-profiling assays. A summary of the proceedings and conclusions of the meeting is reported by Cronin et al. (2) in this issue. The principal conclusion of the workshop was that two types of RNA reference materials were needed: an Assay Process Control and an Array-Specific Hybridization reference material. The reported consensus of the meeting attendees was that such RNA reference materials would provide a measure of the accuracy, dynamic range, sensitivity, specificity, and reproducibility for the multiple types of currently available RNA expression technologies (arrays and QRT-PCR). Although acknowledging that there were several major challenges related to utilization of the proposed RNA reference materials and interpretation of results derived from their use, the authors of this report sounded a familiar refrain in their justification of focusing only on the goal of identifying and characterizing universal RNA reference materials. This was an important first step. Tackling associated implementation and interpretation issues was labeled as “beyond the scope of the March conference”. Memories of my colleague’s statement and the smile on his face came immediately to mind as the phrase “beyond the scope” was repeated several times in the report.
The goal of establishing measurable parameters to evaluate the performance of expression-profiling assays and instrumentation is indeed essential and almost self-evident. As described by Cronin et al. (2), approvals for new drugs or diagnostic tests using RNA profiling data as part of the data-submission process will be hampered unless acceptable and comparable reference materials are established for array- and QRT-PCR-derived data sets. As part of any performance evaluation effort, it is essential to identify and characterize control RNA reference materials that will be used by all RNA-profiling assay systems. The characteristics of both types of RNA reference materials were well described, and their utility was justified by the meeting attendees. However, the goal of the workshop to focus only on the RNA reference materials is troubling and leaves one with the same kind of unrealistic sense that filled me on the stairs with my colleague. Although the report clearly indicates that there are operational and contextual issues in the use of these reference materials, these challenges are viewed as separable from the purpose and the description of these reagents. Acknowledgments by the authors that there are troubling issues associated with the application of the proposed RNA reference materials does not help in achieving the goal of identifying measurable parameters by which expression-profiling assay systems can be evaluated. In fact, it may well be that these efforts will need to be revisited when some of these issues are eventually addressed.
Deferring the acknowledged issues catalogued by the authors jeopardizes the meeting’s efforts at identifying and using the characterized universal RNA reference materials, and this strategy seems to miss the integrated circumstances in which these materials will be used. Consider, for example, the combination of two deferred issues: not defining the biological background in which the standard RNA samples are to be tested and the desire to correlate results derived from cDNA- and oligonucleotide-based arrays. Testing of each the 96 and 12 suggested RNA transcripts comprising the Assay Process and Array Hybridization reference materials, respectively, is recommended to occur initially in a low-complexity hybridization background. Results from such an approach may well provide a misleading conclusion concerning the performance of any array system. Because the overwhelming majority of mRNAs to be monitored in a biological context are present at low copy numbers per cell (3), the capability of measuring the performance of an array system in the context of the competing related background sequences is integral.
An oligonucleotide array system tested in a low-complexity hybridization context with the RNA transcripts chosen for the Assay Process reference materials may be evaluated as being highly specific but less sensitive than a cDNA-based array system. However, in the context of a complex hybridization setting, lower specificity in a cDNA array system may lead to similar overall sensitivities for both the cDNA and oligonucleotide array systems. Thus, if initial testing and evaluation of expression-profiling systems is carried out in low-complexity hybridization conditions, performance results and impressions may well be misleading. Altering these impressions, if they are shown to be different in complex hybridization conditions, could be difficult and time-consuming. The identifications of specific RNA transcripts chosen as part of the Assay Process reference materials may themselves need to be reevaluated when used in a complex hybridization setting. Certain RNA transcripts may not be suitable to serve as low-copy-number reference standards because of the competitive-sequence-related transcripts present in a complex hybridization reaction. As one example indicating that membership of the RNA reference materials should be considered in the context of a complex hybridization setting is the observation that the number of pseudogenes in the human genome is now estimated to be nearly 20 000 (4).
The effect of deferring a solution to this single issue of hybridization complexity in which the RNA reference materials are to be used extends into other issues that will influence how the performance of any array- or QRT-RCR-based technology will be evaluated. The authors report in detail several issues related to the application of the recommended RNA reference materials (2). Among these is the issue of common algorithms or platform-modified algorithms used to calculate the detection and quantification of the proposed RNA reference materials. For the probes in both cDNA and oligonucleotide arrays, there is a quantitative relationship for each probe between true signal and concentration of the target. However, probe-sequence-specific behavior often clouds this relationship. The goal of all algorithms used by expression-profiling systems is to determine which probes are detecting the intended target in the sample mixture as opposed to sequence-related background and, in doing so, to provide the relative amount of detected target in one sample compared with another. Such algorithms may be dependent on estimating the probe-specific behavior. This estimation is based on the signal generated by the probe when there is no target in the sample and also when a range of target concentrations are present. In this latter case, the linearity of the relationship between the signal and the target concentration can be determined for each probe or collection of probes. In the absence of complex hybridization conditions, the value of RNA reference materials to help guide these estimations is dramatically reduced.
These issues have not gone entirely unattended. Recently, a collection of representatives from 20 different international academic, commercial, and governmental institutions, including NIST, have begun addressing this complex topic. This External RNA Control Consortium met in December 2003 to begin to explore the use of RNA controls in the context of the full complexity of expression-profiling experiments (5).
The interrelationships and dependencies of the issues summarized by Cronin et al. (2) for the proposed RNA reference materials are considerable. Focusing only on the characteristics of the RNA reference materials gives the impression that this is a way to begin to compartmentalize and simplify the standardization of elements to be used in the technologies used to measure expression profiling. Whitehead et al. (6) once said, “Seek simplicity and distrust it”. Deferring the challenges that are so closely interconnected with the use of the proposed universal RNA reference materials seems an ill-considered start and may create the illusion of progress on this important and complex topic.
