Log in Article Discussion Edit History Go to the site toolbox

HTM ComDoc 7

From HTMcommunityDB.org

Contents

Creating a community database of maintenance findings from PM and repair records (under revision)

(This document was last revised on 10-1-18)


7.1 Introduction

As explained in some detail in Sections 4.3-4.6 of HTM ComDoc 4., the documented findings that we intend to compile in this database will constitute the all-important evidence needed to support the Healthcare Technology Management Community’s contention that current levels of reliability and safety of the medical equipment in use in the nation’s hospitals should be considered acceptable.

We describe in HTM ComDoc 3. how we have used an RCM-based risk analysis to separate the various kinds of medical devices (of which there are approximately 750 different types) into seventy-one (71) device types that are theoretically capable of causing physical harm to a patient if they either fail completely, or if they develop a significant performance or safety degradation. Table 4 separates these 71 different device types into six tiers of criticality (A-F) according to the potential severity of the worst-case outcomes of device failure. Seven (7) are placed in the top tier (Category A) because they are deemed critical at the “life-threatening” level (LOS 3) with respect to both reliability and safety; and another eight (8) are placed in the second tier (Category B) because they are deemed critical at the “life-threatening” level and also critical at the “possible patient injury” level (LOS 2) with respect to reliability and safety. Another five (5) are judged to have at least one life-threatening failure modeand we suggest that these "top twenty" device types be considered to be the most critical of the theoretically critical device types. It is our immediate goal to compile reasonably current data on all of the various manufacturer-model versions of these "top twenty" critical device types.

In addition to this, we believe that it is quite likely that the data collected here will prove that at least the great majority of the various versions of the remaining 51 device types will prove to be acceptably-safe with respect to their possible failures.

And all of the remaining (approximately 675) device types are considered to be, de facto, non-critical and therefore inherently safe with respect to their possible failures, without the need for any additional supporting evidence.

7.2 The data aggregation process

With the initiation of the AAMI-supported RCM project and subsequent promulgation of this website we are appealing to every member of the Healthcare Technology Management Community to provide the Maintenance Practices Task Force with summaries of the documented findings of their on-going maintenance on all of the device types listed in Table 4, but particularly the "top twenty" device types. We are providing immediately below (in paragraph 7.3) guidelines on how the maintenance, testing and reporting must be performed if it is to be included in this project.

To streamline the reporting part of the process we are also asking certain organizations to volunteer to act as intermediaries and data aggregators. Organizations that are candidates for this data aggregator role include independent service organizations (ISOs), national or regional hospital systems with their own in-house maintenance services, and computerized maintenance management system (CMMS) vendors. We are also providing below (in paragraph 7.5) guidelines on how the data aggregators should compile the data into organized batches suitable for submission.

7.3 Guidelines for maintenance, testing and reporting

In order to make these maintenance findings such that they can be legitimately aggregated, the maintenance, testing and reporting must be performed in a consistent manner. This is very important.

  1. The maintenance entity must use a PM procedure that has, as a minimum, all of the device restoration and safety testing steps listed in the relevant HTMC PM procedure for each manufacturer-model version identified in Table 5 This is to ensure that all of the non-durable parts identified by the device manufacturer in their recommended PM procedure are similarly addressed by each maintenance entity, and that all of the hidden failure modes that are considered potentially critical are similarly addressed. Note that Table 5 is still incomplete at this time. Currently there are procedures for only some of the "top twenty" critical device types listed, and these are provided as initial models that can be used to develop similar procedures for the rest of the devices that will eventually appear in the complete table. Note also that there may be several variants of the generic procedure for one particular device type. For example; the code for the generic procedure for an apnea monitor is listed in Table 4 as APN.M, but since there are apnea monitors that operate on at least two different principles, there will probably be several variations on the generic procedures for the different manufacturer-model versions listed in Table 5 The codes for these variants will appear as APN.M-01, APN.M-02, etc.
  2. For our purpose here it is not necessary for the maintenance entity to perform the maintenance and testing at the same interval as that recommended in the manufacturer’s procedure. Indeed, in the absence of any regulatory mandates, some diversity is welcome since it is one of our goals to compare the reliability and safety achieved at various maintenance intervals. We address this issue in more detail in HTM ComDoc 6.
  3. The maintenance entity must use some form of coding for repair calls that allows for a separate count of the failures that are attributable to inadequate PM (similar to the MR 1 described in HTM ComDoc ?). Because of its value in maximizing total equipment safety, we also recommend a coding of at least the three basic causes of total failure described in HTM ComDoc 1- namely IRFs or inherent reliability-related failures; MRFs or maintenance-related failures; and PRFs or process-related failures. Adopting the full 15 category classification and coding method described in HTM ComDoc 1 and HTM ComDoc 8. is highly desirable because of its value in diagnosing possible non-maintenance remedial actions.

7.4 The Summary Proof Tables

The primary part of the website-based database will consist of a set of tables (Summary Proof Tables) summarizing the key findings from submitted sets of data for each of the PM-critical device types. The the format is shown in this illustration - Table 5. Critical care ventilators (illustrative hypothetical example). Like the Detailed Proof Table (Critical care ventilator/ Carefashion-Velour/ 6 months) this table contains hypothetical data and is intended only to illustrate the kind of useful information that this project will eventually make available to the entire community.

- Column C1 contains two descriptive data strings indicating the device type, the manufacture-model and the PM interval.
- Columns C2 and C3 show the size and acceptability of the experience base. These are the bottom line totals from columns 6 and 7 of the Detailed Proof Table (Critical care ventilator/ Carefashion-Velour/ 6 months).
- Columns C4, C5 and C6 show the MTBF representing the level of reliability/ safety of this specific type of device with respect to any DR-related failure modes; the Level of Severity of the DR-related failure modes; and the acceptability of the MTBF shown in C4. These are the bottom line totals from columns 8-13 of the Detailed Proof Table (Critical care ventilator/ Carefashion-Velour/ 6 months).
- Column C7 shows the PM Priority level associated with the MTBF shown in C4.
- Similarly, columns C8 – C11 show the same data reported for the ST-related failures. These are the bottom line totals from columns 14-17 of the Detailed Proof Table (Critical care ventilator/ Carefashion-Velour/ 6 months).

Those devices that have acceptable levels of reliability (acceptable MTBFs) for both the device restoration-related failure modes and the safety testing-related failure modes (as indicated in columns C7 and C11 of the table should be considered to be either low-risk or very low-risk devices - because the failures occur relatively infrequently. In other words the MTBF s representing the reliability of their DR-related failure modes and their ST-related failure modes are both greater than the relevant threshold of acceptability. Table 5.3 shows the three different levels for this threshold of acceptable risk (25, 50 or 75 years) for the three different Levels of Severity of the adverse outcomes of those failures.

According to the Commentary on the illustrative findings shown in the Summary Proof Table (Table 5. Critical care ventilators (illustrative hypothetical example)), it appears that the Velour model of the Carefashion company’s critical care ventilators (whose test results are shown in rows R1 and R2) is demonstrating an unacceptable level of device restoration-related failures when the devices are maintained at a 12 month interval – showing an MTBF of only 13 years. This level of poor reliability is not seen when the same devices are maintained at the recommended 6-month interval. In contrast to this, the Coyote company’s Model 300 ventilator (test results shown in rows R3 and R4) still shows an acceptable level of reliability and safety when it is maintained at the longer 12 month interval.

A different and more concerning pattern is seen with the Matelot company’s Model 500 ventilator. It demonstrates (the test results shown in rows R7 and R8) an unacceptable level of reliability and safety at both maintenance intervals - apparently related to the poor reliability of the device’s non-durable parts. Based on this evidence this particular model should be characterized as a high-risk (PM Priority 1) device when maintained at either 6 or 12-month intervals.

Yet another pattern is seen with the Uno model of the Herman company’s ventilator. While it shows acceptable reliability and safety when it is maintained at a 6-month interval, it appears to demonstrate an unacceptable level of performance or safety problems when it is maintained at the longer 12-month interval. Based on this empirical evidence, this particular model should also be characterized as a high-risk (PM Priority 1) device when it is maintained at the 12-month interval. It seems that these particular PM-critical devices, all of which have potentially life-threatening outcomes, show a tendency to become unreliable if the maintenance interval is not carefully controlled.

7.5 Guidelines for compiling the data into organized batches

  1. Each separate batch of data should contain only data on maintenance and testing performed on one particular manufacturer-model, at one particular interval, by one particular maintenance entity.
  2. The following data should be assembled in a spreadsheet-like format. The format can be seen by clicking on the link in the first table shown in Table 5. - which is copied here - Critical care ventilator/ Carefashion-Velour/ 6 months - shows how the compiled data will be displayed in the database's Detailed Proof Table (Critical care ventilator/ Carefashion-Velour/ 6 months).
- The first three columns contain three different code numbers that will be associated with each of the submitted batches of data
- Column 4 shows the size of the sample, the number of devices (same manufacture-same model) in the batch (N)
- Column 5 shows the time period over which the testing was done (T months)
- Column 6 shows the resulting size of the experience base (E), in device-years, for each batch, where E = NxT/12
- Column 7 shows whether or not the size of the experience base meets the criteria for acceptability (see Table 5.2)
- Column 8 shows the number of MR 1 failures reported (M). These failures were the result of inadequate device restoration.
- Column 9 shows the number of PM Code 9 failures reported (W). These are failures that were detected just before they could become MR 1 type failures.
- Column 10 shows the total number of device restoration (DR)–related failures reported(R) - where R = M + W
- Column 11 shows the MTBF associated with the DR-related reliability, in years, which is calculated as E/R
- Column 12 shows the Level of Severity that was associated with the DR-related failure modes of the device (from the criticality analysis described in HTM ComDoc 3.)
- Column 13 shows whether or not the level of reliability/ safety of the MTBF associated with the DR-related failure modes (shown in column 11) meets the criteria for acceptability (see Table 5.3)
- Column 14 shows the PM Priority level associated with this combination of the severity level of the potential DR-related failure and the reported probability (MTBF) that the DR-related failure will actually occur (see Table 5.4)
- Column 15 shows the number of safety testing (ST)–related failures (hidden failures) reported (S)
- Column 16 shows the MTBF associated with the ST-related reliability, in years, which is calculated as E/S
- Column 17 shows the Level of Severity that was associated with the ST–related failure modes of the device (from the analysis described in HTM ComDoc 3.)
- Column 18 shows whether or not the level of reliability/ safety of the MTBF associated with the ST-related failure modes (shown in column 16) meets the criteria for acceptability (see Table 5.3)
- Column 19 shows the PM Priority level associated with this combination of the severity level of the potential ST-related failure and the reported probability (MTBF) that the ST-related failure will actually occur (see Table 5.4)

7.6 Periodic reviews of the findings


It is intended that the Task Force will periodically review the aggregated maintenance findings and post their collective informed judgments on:

  1. The adequacy of the sample size and experience base for each manufacturer-model version at each maintenance interval, and
  2. Whether or not the indicated level of reliability and/ or safety should be deemed acceptable. (See illustrative hypothetical data in the supporting detail Critical care ventilator/ Carefashion-Velour/ 6 months included as a part of Table 5)

7.7 Tentative thresholds for acceptability


The initial values selected for the two thresholds for acceptability (for the amount of data in the experience base, and for the minimum acceptable level of PM-related device reliability and safety) are shown below. They are considered to be tentative at this time.

Tentative characterizations of different amounts of data in the experience base (copied from Table 11.)


(This table was last updated on 1-23-16)


Experience base thresholds
(tentative)
Amount of data
(device-yrs)
Inadequate <50
Good 50-200
Very good 200-500
Substantial >500


Tentative definitions of what should be considered the minimum acceptable levels of PM-related device reliability and safety (copied from Table 12)

The level of risk associated with a device failure caused by not completing the device's PM in a timely manner (which increases the risk that the device will fail) is determined by both the level of severity of the potential adverse outcome if the device fails (LOS 1, LOS 2, or LOS 3) and the device's demonstrated PM-related reliability (expressed as an MTBF* or mean time between failures).

So, devices that have at least one high severity failure mode (LOS 3) combined with a demonstrated poor PM-related reliability (MTBF less than 75 yrs) should be considered to represent the highest risk and labeled PM Priority 1 devices.

Devices with better levels of reliability (represented by longer MTBFs) and/or lower severity outcomes should be considered PM Priority 2, 3, 4 or 5 devices. The full hierarchy of different combinations of the worst-case severity of the result of the device failing and the device's demonstrated reliability is shown in the table below.

(This table was last updated on 5-18-16)






Level
of
potential risk
Outcome Severity Level
A, B, C, or D

Devices with
at least one
failure mode
with severity level
LOS 3
(Life-threatening
injury)
Outcome Severity Level
E, F, or G

Devices with
at least one
failure mode
with severity level
LOS 2
(Possible patient
injury)
Outcome Severity Level
H

Devices with
at least one
failure mode
with severity level
LOS 1
(Disruption of
patient care)
"PM Priority 1 device"
(high risk)
Poor
reliability

(MTBF <75 yrs)
"PM Priority 2 device"
(moderate-high risk)
Good
reliability

(MTBF 75-150 yrs)
Poor
reliability

(MTBF <50 yrs)
"PM Priority 3 device"
(moderate risk)
Very good
reliability

(MTBF >150 yrs)
Good
reliability

(MTBF 50-100 yrs)
Poor
reliability

(MTBF <25 yrs)
"PM Priority 4 device"
(low risk)
Very good
reliability

(MTBF >100 yrs)
Good
reliability

(MTBF 25-50 yrs)
"PM Priority 5 device"
(very low risk)
Very good
reliability

(MTBF >50 yrs)
  • For definitions of Outcome Severity Levels A thru H - see column 5 of Table 4.
  • MTBF (mean time between failures) is a measure of the device's demonstrated level of reliability. It is the inverse of the device's average failure rate. A higher MTBF indicates that the device is more reliable.

14.15 Proposed format for documenting the PM Findings data on every PM Report


A section is added at the end of each PM procedure asking the service person to indicate by circling one of three letters (A, B or F) whether or not the performance and safety testing of the device revealed any significant degradations or hidden failures.

A = nominal. The letter A should be circled when the results of all of the PVST tests were in compliance with the relevant specifications, and any other functions tested were within expectations.
B = minor OOS condition(s) found. The letter B should be circled when one or more conditions were found that were slightly out-of-spec (OOS) or slightly outside expectations. The purpose of this B rating is to create a watch list to monitor for future adverse trends in particular performance or safety features, even though the discrepancy is not considered to be significant at this time. An example of this would be an electrical leakage reading of 310 microamps which is within 5% of the 300 microamp limit. A B rating should be considered a passing grade.
F = serious OOS condition(s) found. The letter F should be circled when one or more performance or safety features is found to be significantly out-of-spec. (OOS). This is a failing grade and, if it is a high-risk device, it should be removed from service immediately.

The service person is also asked to indicate by circling one of four numbers (1, 5, 9 or 0) the physical condition in which the device parts that were rejuvenated by the traditional PM tasks were found. The numerical ratings should be circled to indicate one of the following findings.

1 = better than expected. There was very little or no deterioration; i.e. the physical condition of the restored part was found to be still good.
5 = nominal. There was some minor deterioration but no apparent adverse effect on the device’s function; i.e. the physical condition of the restored part was found to be about as expected.
9 = serious physical deterioration. The restored part was already worn out and probably having an adverse effect on device function; i.e. the physical condition was found to be considerably worse than expected.
0 = no physical restoration required. The device has no parts needing any kind of physical restoration.

If the PM findings are systematically documented each time a PM is performed, then aggregated into a PM Findings database, it will be possible to:

  • get an indication of the mean time between failures (MTBFs) of any hidden failures, and
  • get an indication of how well the PM interval matches the optimum - which would be when the part being restored has deteriorated - but only to the point just before the deterioration begins affecting the functioning of the device.
  • If the interval is too short, this would be indicated by a preponderance of PM Code 1 findings; and
  • if the interval is too long, it would be indicated by a preponderance of PM Code 9 findings.
  • PM findings of PM Code 5 indicate that the PM interval is just about right.


Site Toolbox:

Personal tools
This page was last modified 19:48, 6 October 2018. - This page has been accessed 442 times. - Disclaimers - About HTMcommunityDB.org