| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Articles |
a Author for correspondence. Fax 716-453-4402; e-mail: flasky{at}cldx.com
 |
Abstract |
|---|
 Top Abstract Introductiondesign inputdesign and development planningdesign verificationdesign outputmanagement responsibilityafter product launchconclusionReferences |
|---|
Key Words: indexing terms: product development • expert systems • quality control
|
Introduction |
|---|
|
TopAbstractIntroductiondesign inputdesign and development planningdesign verificationdesign outputmanagement responsibilityafter product launchconclusionReferences |
|---|
In a recent guidance published by the FDA, two models of design
processes were offered (Fig. 1
) (2). The Waterfall Model is a sequential process
that begins with understanding user needs, which are translated into
clear specifications, followed by prototype component and system
testing. Finally, the complete system is evaluated in its working
environment. At every stage verification takes place to assess the
ability of both the components and the system to meet the proposed
specifications and functional requirements. Frequent design reviews by
management and the design team are used to either accept performance or
recommend changes.
|
FDA appreciated that design processes seldom occur sequentially, and
therefore it offered another example, called the Upstream Model (Fig. 1B
). Here, continual interaction occurs to clearly define user
requirements, determine product manufacturing processes, and plan for
adequate service support after market introduction. This model suggests
the highly interactive processes often used to continually and
simultaneously improve the device design and those needed to deliver a
quality medical device to the marketplace.
The real process is typically a combination of the event sequences in these models. It is noteworthy that this guidance received some criticism because of industry concerns that guidances are often treated as surrogate regulation (3), and as illustrative as these models might appear, they do not adequately represent the dynamics of design control.
Quality by design involves a process that is outlined in the ISO 9000 and FDA Quality System standards (4) (and regulation (5)). We provide some examples of tools and processes that can and have been applied to in vitro diagnostic (IVD) products, although each manufacturer chooses which are appropriate for the product being proposed and the capability of each manufacturer's own organization. Then, when a device is introduced into the market, quality assessment does not end there. Quality practices [and good manufacturing practices (GMP) regulations] require vigilance by manufacturers, with help from users, to maintain a product's fitness for use over its useful life.
|
design input |
|---|
|
TopAbstractIntroductiondesign inputdesign and development planningdesign verificationdesign outputmanagement responsibilityafter product launchconclusionReferences |
|---|
Product definition.
Planning, marketing research, and
marketing benchmark the competition and define the changes,
differences, or improvements that will appeal to customers and set the
new product apart in the market. Performance characteristics,
throughput, size, cost, and even color must be defined.
Environmental conditions.
Where and how a product will be sold
and used helps define the environmental settings in which it must
operate safely and effectively. Marketing professionals provide input
for where and how the IVD will be used, while design engineers use
their expertise to establish tolerance limits of temperature, humidity,
or other variables within which reagents, components, or the system as
a whole is expected to perform. Expectations and capabilities must
match.
IVD product users.
Whether an IVD product is expected to be
used in the home, at the bedside, in a physician's office or clinic,
or in a moderate or high complexity testing clinical laboratory, the
users must be considered during the design input phase. This will
affect decisions for instructions (how they are written and presented),
the difficulty and number of operational steps, product storage and
stability, and ease of use. Marketing, marketing research, regulatory
affairs, technical publications, and human factors engineering can
contribute to defining these elements.
Duration of use.
Whether a device is intended for single-use
testing or uses nondestructive measurement techniques affects
assessment of the anticipated costs of device manufacturing and
subsequently the profitability. Additionally, because design and
capability of electronic components change rapidly, their impact on
obsolescence must be considered to arrive at reasonable estimates of
product life cycle. Design and service engineers can provide input.
Service and repair.
Requirements for frequency and ease of
repair, reliability goals, component obsolescence, and cost of
component replacements should be set. Additionally, serviceability and
support requirements affect the extent of the service network that must
be put in place and provide guidance for product design. Design
engineers, service engineers, and customer support personnel can define
requirements, whereas marketing personnel can provide details of
customer expectations.
Product requirements.
Regulatory, clinical, and technical
requirements must be met. Many regulatory requirements or minimum
functional standards might not be known to the user. For example, many
countries established limits for maximum instrument electromagnetic
interference and (or) required UL listing. Design engineers and
regulatory professionals can provide this input.
Environmental regulations are becoming more substantial. Germany has a recycling requirement (the "Green Dot") that must be met to sell effectively in that country. When chlorofluorocarbon use was restricted, it not only affected refrigeration, but also resulted in changes in some plastic manufacturing processes. Environmental engineers and regulatory professionals have access to these requirements and standards.
Product regulatory requirements often vary by country. Many countries, such as the US, France, and Japan, require that some medical devices, including some IVDs, be subjected to performance reviews before they are permitted to be imported or sold. Submissions must contain the information that each country's government agency requires. If a product is designed for home use, the FDA, and soon, the European Union, has special requirements for instructions for use. Also, FDA product classification (6) can help determine the appropriate amount of risk analysis that might be used during design. Clinical laboratory regulations and accreditation standards must be met if an IVD test or instrument is to be successful in the US. CLIA complexity categorization, quality-control recommendations, and other requirements will influence product accessibility. Regulatory affairs and technical support specialists can interpret regulations and provide input and recommendations.
Clinical performance standards for IVD tests have not been generally available; few examples are published in the peer-reviewed literature. Two possible exceptions include plasma lipid concentrations and blood glucose monitoring. The National Cholesterol Education Program (7), familiar to most clinical scientists, recommends bias and precision limits for lipid testing that are meant to fulfill the needs of the clinician.
Clarke et al. (8) published limits for self-monitoring
whole-blood glucose, shown in Fig. 2
, that attempt to correlate analytical error to clinical impact.
Areas A and B are regions in which results compare sufficiently well
with reference values so as not to affect patient treatment, or
treatment is benign. Errors in areas C would result in the patient
overcorrecting acceptable blood glucose concentrations (between 3.9 and
10.0 mmol/L). Areas marked D represent zones in which the patient is at
risk because of a failure to detect and subsequently to treat his
condition. Areas marked E are zones of erroneous treatment, where a
reported result is in a clinical region opposite to the reference
value. Although this analysis does not enjoy consensus, it does provide
a basis for transforming clinical acceptability criteria into
quantifiable limits. Unfortunately, similar analyses are lacking for
other analytes and other patient populations.
|
Analytical performance criteria are easier for manufacturers to estimate. Interlaboratory performance data from proficiency testing or external quality-assurance schemes, such as the College of American Pathologists or New York State, are readily available. These data have been used in processes that translate total error estimates into error components (within-run imprecision, bias, and lack of specificity) (9)(10), which can then be converted into process release specifications. Consultant clinicians, pathologists, clinical scientists, and statisticians often must elucidate needs and requirements.
All these inputs must be clearly defined, detailed, and understood by all members of the design team. When agreement and sign off are reached, an appropriate project plan can be developed.
|
design and development planning |
|---|
|
TopAbstractIntroductiondesign inputdesign and development planningdesign verificationdesign outputmanagement responsibilityafter product launchconclusionReferences |
|---|
Accessible and interpretable documentation of the whole plan provides for orderly changes and modifications to the product design or to the plan itself, tracks what is often a complex process, and places management in a better position to agree to changes or reassign priorities.
Many planning tools and techniques such as the Six Sigma Technique, developed at Motorola (11), and Program Evaluation and Review Technique (PERT) charts are available for design and development planning.
The Six Sigma objective is to have processes in place that produce
yields of 99.99966% (a failure rate of <3.4 per million). However,
this goal will not be achieved even if the upper and lower limits of
customer acceptance (quality boundaries) are greater than ±6
(SD)
of the process variability (12). Other actions are needed;
for example, suppliers of raw materials and components, as well as
consultants and service providers, should be brought into planning
development as early as possible. Supplier specifications should be
clearly defined and, if possible, quantified to allow routine
verification of the suppliers' ability to meet them. Minimizing the
number of production parts and suppliers reduces the possibility of
component failures or nonconformances.
Six Sigma can be achieved in six steps:
1. Identify characteristics needed to satisfy the customer; listen to "the voice of the customer," as described above.
2. Identify critical characteristics so specifications are established on all the important components, not just on those that are measured easily.
3. Determine whether important operational steps and components can be controlled by product or process design. Here is the opportunity to improve reliability and reduce mistakes that can occur on the production line or by the end-user.
4. Decide maximum tolerance limits for the critical characteristics that have been identified. Tolerances must satisfy the needs of the user and, simultaneously, be within limits that can be economically manufactured—not a trivial task.
5. Measure process variation to provide assurance that processes are under control and within specification (and make modifications before a process variation can result in unacceptable products).
6. Change product design or reduce process variability if needed to achieve performance and economic goals.
A PERT chart (Fig. 3
) contains a diagram of task sequences required to carry the
plan to success. Each task box contains the name of the department that
fulfills the task, the number of days for completion, the completion
date, and the percent of task completed to date (13).
Putting a line through the box shows completion. Often, the tasks
requiring the longest time are outlined in another color. This helps
determine when a task is to be initiated and provides information about
the critical path toward schedule completion. Where the arrows
converge, a management or design review might take place. Reviews
enable the team to assess progress and decide whether resources (people
and money) should be reallocated. If tasks are incomplete at review or
a finding is unacceptable, management can decide whether the program
should proceed. Tasks can be completed later, the plan can be modified,
or an aspect of the plan can be dropped. (Such decisions are common,
further illustrating why the Waterfall Model is overly simplistic.) All
decisions should be documented to keep management apprised of progress
and aid in assigning resources as needed to maintain progress.
|
In reality, a PERT chart can consist of hundreds of tasks for each department, and all departments must interact and coordinate time lines. Thus, the chart is a multidimensional road map providing a reliable means of tracking complex projects.
|
design verification |
|---|
|
TopAbstractIntroductiondesign inputdesign and development planningdesign verificationdesign outputmanagement responsibilityafter product launchconclusionReferences |
|---|
The need to establish clear, precise, and quantifiable specifications should now be apparent. Each critical characteristic will be verified against its specification. Component specifications are derived from budgeted allotments of the user requirements. For example, method precision might be dependent on sample metering, incubation time and temperature, and calibration. Acceptable precision will be achieved if the sum of all variables is within the total limit for customer acceptability. If one component is found to be a minor contributor to variability, greater tolerance can be "awarded" to another component to still meet system requirements.
Cost analysis should not be overlooked. An elegant design may not be cost-effective or easily serviceable. Also, although some design advantages are currently available that would provide operational and cost benefits to the laboratory and the patient, regulations such as CLIA prohibit their use, e.g., inflexible quality-control requirements (15).
Failure analysis might be undertaken during verification, if deemed appropriate by the degree of risk associated with the device. Two such procedures are Fault Tree Analysis and Failure Mode and Effects Analysis (FMEA).
Fault Tree Analysis is a deductive, top-down approach (16). It identifies potential adverse consequences to a patient or an operator, e.g., death, injury, or a missed diagnosis. Malfunctioning components or system conditions that can lead to each type of serious event are then listed. Opportunities to improve device design then can be implemented to eliminate, reduce, or accept the conditions that can cause undesirable consequences.
FMEA is an inductive, bottoms-up process (16). The
consequence of a potential component failure is assessed, e.g., what
would happen to a result if a pump fails or if an operator picks a
collection tube from the wrong patient? Both the frequency (e.g.,
frequent vs occasional vs remote) and the severity (minimal vs moderate
vs severe) of the failure are estimated. A matrix can then be
developed, as shown in Fig. 4
, to identify risk and tolerance. This analysis can help
identify which failures must, should, or can be minimized or
eliminated. Options can include changes in design or inclusion of
appropriate labeling and notification. Decisions include whether the
consequences or the likelihood of a failure incur an acceptable risk,
or if a "fix" can be incorporated cost-effectively.
|
The desired outcome is to minimize device hazards, perhaps by incorporating mistake-proof mechanisms. Some hazards are inherent in healthcare and, perhaps, cannot be risk-free. Phlebotomists and nurses are at substantial risk when they are exposed to sharp needles during blood collection. Appropriate needles, syringes, and training minimize the hazard of infection.
Some successful design initiatives inherent in Vitros clinical chemistry analyzers (Johnson & Johnson Clinical Diagnostics, Inc., Rochester, NY) are often unnoticed by the operator. Short sampling of patient specimen is avoided by automatically monitoring the air pressure profile at the pump. Using infrared spectrometry for wetness detection ensures that the sample was dispensed onto a reagent slide. Ion-selective electrode slides are automatically monitored for integrity by use of an impedance check. Instrument reliability was enhanced in one model of analyzer by a 50% reduction in the number of circuit boards. In that same model, the barcode identification on the test-specific reagent slide provided assurance that the test reported and the test performed were identical. Design features such as operator-alert communication and increased system reliability minimize the likelihood of a mistaken result being reported.
|
design output |
|---|
|
TopAbstractIntroductiondesign inputdesign and development planningdesign verificationdesign outputmanagement responsibilityafter product launchconclusionReferences |
|---|
Product documentation and labeling, including instructions for the user (method sheets and manuals), are prepared and shown to be intelligible and user-friendly. Descriptions of procedures that enable laboratory operators to comply with their own regulatory and accreditation requirements also need to be confirmed. CLIA and College of American Pathologists requirements for quality control, calibration, and maintenance protocols are familiar to clinical laboratories operating in the US. Manuals and instructions for servicing, installing, and troubleshooting equipment are also included, as appropriate.
The final or near-final product configuration is used for validation, defined as "confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use can be consistently fulfilled" (see 21CFR820.3(z) (5)). This requires that the acceptability of the IVD product be assessed as it is intended to be used by the operator, and the manufacturer must have processes that lend confidence to its ability to produce a device of acceptable quality consistently.
Therefore, a key element of the design output is the preparation and transfer of all documentation required to manufacture the product with consistent quality. Documents usually include evidence of manufacturing equipment qualification, process flow diagrams, and a complete set of manufacturing procedures, specifications, and instructions. The task of assuring transferability of the necessary documentation and demonstrating that the documentation is adequate to manufacture products successfully is complex and arduous. These documents are contained in the Device Master Record (DMR), which provides the basis for GMP audits by the FDA and other quality service organizations, such as ISO auditors. Procedures contained in the DMR must be followed for a manufacturer to be in conformance with its own quality standards.
|
management responsibility |
|---|
|
TopAbstractIntroductiondesign inputdesign and development planningdesign verificationdesign outputmanagement responsibilityafter product launchconclusionReferences |
|---|
Management is also empowered to revise the design plan, as appropriate. Flexibility in the plan is essential to bring a product to market within cost and on schedule that can satisfy its intended use. Change should be systematic and under control, with appropriate documentation that explains changes and the reasons for them. Any change made during the design process can have profound effects on the output. The later a change is made, the greater the potential of negative impact on the cost of production, tooling, promotions, training, and schedule.
|
after product launch |
|---|
|
TopAbstractIntroductiondesign inputdesign and development planningdesign verificationdesign outputmanagement responsibilityafter product launchconclusionReferences |
|---|
|
conclusion |
|---|
|
TopAbstractIntroductiondesign inputdesign and development planningdesign verificationdesign outputmanagement responsibilityafter product launchconclusionReferences |
|---|
Just as each manufacturer decides which of the tools and processes are appropriate for its unique product, laboratory directors must be permitted to decide how to best monitor performance and reliability of test results (not just reagents and instruments) in his or her own environment. Control frequency should depend on the capabilities of the systems in use, the criticality of the test result, and the operation of the laboratory (such as throughput, specimen collection and transport, and how results are stored and reported back to the provider). Additional quality-control protocols should be considered if an assignable change in the process is expected, e.g., calibration, a new reagent shipment, or major equipment maintenance. Concentrate on the identification of sources of mistakes and reduce or eliminate them.
Historical performance should be used to advantage. Less stringent control procedures, e.g., reduced sample rate, might be "earned" if collected data support it. Conversely, one might accept that processes can change and that "falling back" to more frequent monitoring might be needed when an unplanned loss of control occurs. Regulations must permit flexibility in quality-control protocols and procedures.
According to Sir William Osler, "Medicine is a science of uncertainty and an art of probability," but when quality systems are used effectively by IVD device manufacturers and clinical laboratories, together we help reduce the uncertainty in the clinical science so that physicians can more effectively practice their art.
|
Footnotes |
|---|
1 Nonstandard abbreviations: FDA, Food and Drug Administration; ISO, International Organization for Standardization; IVD, in vitro diagnostic; GMP, good manufacturing practices; PERT, Program Evaluation and Review Technique; DHF, Design History File; FMEA, failure mode and effects analysis; CFR, Code of Federal Regulations; DMR, Device Master Record. 
|
References |
|---|
|
TopAbstractIntroductiondesign inputdesign and development planningdesign verificationdesign outputmanagement responsibilityafter product launchconclusionReferences |
|---|
The following articles in journals at HighWire Press have cited this article:
|
|
 |
|
  R. W. Jenny and K. Y. Jackson-Tarentino Causes of Unsatisfactory Performance in Proficiency Testing Clin. Chem., January 1, 2000; 46(1): 89 - 99. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
