Navigation

Built In Medicine Cabinets

Navigating paths for effective plant-wide standard compliance.

There's an approach to integrated process control that challenges engineers and project managers but also gives the plant more flexibility. Plants built on this model will require fewer personnel to maintain them. This model also successfully meets all 21CFR11 requirements regarding electronic records.

The approach uses terminal server-based user interface, network-based authentication, centralized setpoint and recipe management via databases, and centralized Webbased reporting and data mining. This results in a standard mechanism for capturing all critical operator actions with a 21CFR11 compliant audit trail. When coupled with enhanced procedural controls, these solutions can provide a mechanism for 21CFR11 compliance that lowers downstream support costs.

Most 21CFR11 compliance schemas are equipment specific and result in plants with a series of isolated mechanisms for 21CFR11 compliance. This non-integrated approach is difficult to design, implement, and validate and is cumbersome to maintain and defend to regulatory authorities. The proposed scheme presents mechanisms to capture and filter all operator actions, such as setpoint and recipe changes, manual overrides, alarm acknowledgement, and starts/stops/aborts with full audit-trail support that includes date, time, and operator ID. Core portions of this scheme capitalize on the use of server, application, and network redundancy as well as terminal server-based, operator interface, and Web-based reporting tools. Other benefits include lower support and change management costs.

Applicability to process control

The audit trail requirement of 21CFR11 says you need to capture date, time, and operator ID for all electronic records. While the source code is an electronic record you must maintain, the running control system doesn't produce electronic records, and therefore meets the electronic records requirements of 21CFR11. The compliance issue is industry uses supervisory control and data acquisition (SCADA) systems to allow operators to alter the state of the systems via the various forms of human machine interfaces (HMIs).

SCADA systems often act as a repository for instrument data, recipes, and setpoints. The issue is not how to construct a 21CFRll-compliant PLC; the issue is how to construct a 21CFR11-compliant SCADA. An important distinction about 21CFR11 compliance is it is not an off-the-shelf product you can purchase. It is a methodology each company needs to define, design, implement, and validate according to their own definitions. SCADA systems need to capture audit trails for all critical operator actions. Some actions, such as cycle start/stop/abort, alarm acknowledgements, setpoint changes, recipe selection, recipe edits, PID loop tuning, and manual overrides, affect the state of the system. These are classified as electronic records, and they require audit trails. Other actions, such as screen navigation, report generation, and data viewing, do not affect the state of the system. They are not classified as electronic records, and they do not require audit trails. The important difference between the two types of actions is, in most cases, the critical process actions are usually associated with writes to memory addresses in process control systems.

Traditional process control models

The most traditional process control model involves a series of PLC/DCS systems that are stand-alone. In cases where you need to move data critical to process control among them, this information travels via individual analog and digital I/O. HMI hardwiring occurs via dedicated embedded panels (not computers) to the control systems. SCADA systems may communicate with some or all control systems via a highway type communication pathway (Ethernet, Control Net, DH485). Reporting from the SCADA happens through dedicated reporting stations. (See figure at left.)

A core problem with this scheme is almost no embedded panel-based HMI support two-part logins (operator ID and a password). Some types of panels require no login to access the process. In other cases, you do need a login, but it is a short numeric string. For some panels, when an operator logs in, the panel displays the password on the screen as the operator types it. With these types of systems, as operators affect changes to the process, there is no straightforward mechanism for transmitting operator ID from the panel into a SCADA-based audit trail. For these reasons, embedded panels are not acceptable for applications where 21CFR11 compliance is required. This scheme also has support and maintenance issues. These panels frequently are a source of a single point of failure. If the panel fails, operators can no longer control the process. From an IT management standpoint, panels are costly to support because each panel in a plant has its own access control list. If there are 10 panels in a plant, there are 10 lists to maintain.

A slight variance on the traditional model involves PCbased HMI (industrial computers for hazardous or cleanroom environments). As in the embedded panels, you usually hardwire these into the control systems. SCADA systems may communicate with some or all control systems via a highway type communication pathway (Ethernet, Control Net, DH-485). Reporting from the SCADA occurs through dedicated reporting stations. (see figure at right.)

The Windows-based model can solve many regulatory compliance issues. It supports a two-part login that in many cases you can tie in with operating system authentication. An additional feature is quite a few PC-based HMI platforms support the export of operator activity logs directly into databases. This simplifies the creation of the audit trail. You can use this architectural model to meet all existing 21 CFRIl requirements.

The main detriment to this type of scheme is it has a high ownership cost. If you use more than one HMI platform brand within a plant, you'll need to develop different database schemas to collect, filter, and report the audit trail. Another issue is reliability. At the heart of the stand-alone, PC-based HMI is an operating system for home or office use.

Web-based reporting

This model implements the HMI software on a terminal server. The terminal server houses a centralized code source you can view at the thin client HMI stations on the plant floor. As in the previous model, you'll want to design the servers with redundancy to mitigate the risk of network or operating system failure. The operator event logs export into the facility database. Reports generate through a Web report engine that points to the database. One reporting server may serve multiple plants, and it allows reports to run from operators' desktop PCs.

Terminal server based HMI works by internally generating the control panel screens and painting them onto the thin clients. Since all screens in a plant draw from a single code source, the code deploys and validates only once. Since any screen can appear on any client, this enhances the plant's flexibility. While there are scalability issues in that the number of clients served is dependent on the computing power of the terminal server, modern servers are extremely powerful and scaleable. If you use Ethernet networks extensively, you can implement full network redundancy from all servers to all process control cabinets and to all HMI for extremely low costs.

This combination of server, panel, and network redundancy mitigates the inherent fragility of operating systems. Since developers have engineered redundancy into the system design and tested it as part of validation, you can dramatically reduce the paperwork required when a network, server, or HMI component fails. Also, the redundant design allows for routine maintenance on these components without the need to schedule plantwide shutdown.

Since there are no codes or applications on the thin clients, process control is no longer tied to the local panel operating system. You can patch operating systems and install or modify ancillary applications as required. The only validation is to ensure the change does not affect the running of the terminal services client. This testing can be run offline and referenced across an enterprise. Any system that can run the terminal services client can become a control panel. This allows you to use low-cost Windows CE-based panels instead of costly industrial computers. You can use wireless PDA systems or tablet PCs to allow a single person to bring the HMI into the field when calibrating field devices. A remote station with the correctly configured terminal services client can allow for control systems software support personnel to remotely troubleshoot a plant (using appropriate network security and safety precautions). Also, any PC in the Enterprise can access the reporting Web server to run reports, which eliminates dedicated reporting stations and printers.

Integrated plant architectures

An integrated plant model with centralized HMI servers and databases supporting all plant control systems (including BAS/BMS and utility systems) has even more advantages. It's straightforward to construct database drive systems tables to automatically capture batch start/stop times to feed web driven reporting. You can store and manage recipes with the site database with excellent version control. This data mining ability can reduce investigation cycle time and facilitate periodic inter-batch trending. You can use data linkages to link CIP, SIP, and production with other activities, such as autoclave or vial washer runs, to pull together the full plant picture to higher level application, such as MRP or electronic batch record systems. When the SCADA is collecting all plant alarms from all control systems, use database alarm lookup tables to route alarms to cell phones and pagers via e-mail. This gives you flexibility to define alarm distribution schedule and e-mail group destination on an alarm-by-alarm basis.

With good network security design, you can make read-only HMI applications available to any desktop PC in the enterprise, including dialup or VPN users, without compromising plant safety. Together, the alarming and remote read-only HMI functions can allow on-call personnel to check the nature of an issue and quickly triage it to the correct personnel from any location.

Terminal server model challenges

Without good training on how the terminal server model operates, non-information technology groups will not understand tnese architectures can reduce the change control, validation, and deviation report burden as well as minimize downtime. For this type of implementation to be successful and realize its full potential, IT, engineering, regulatory, validation, and quality groups all need to work together.

Since terminal and Web-server technologies haven't traditionally applied to process control, some of the best engineering and integration firms are hesitant to move to these models. A related challenge is scope delineation. In a plant construction project using this model, groups need to work together at a much higher level. Under traditional models, systems had clear responsibilities with few extra-departmental dependencies. Under the new model, almost every system will have overlapping responsibilities with dependencies on a series of other departments.

This can make engineering and integration firms uneasy about scope. They might be unclear about where to draw the dotted line on the facility map. Standard equipment from OEM vendors will not be able to participate in this model without significant HMI customization, which can lead to significant vendor surcharges. Alternate!}', they could accomplish the work by an integration firm between factory acceptance testing (FAT) and site acceptance testing (SAT). To take this path, the OEM vendor must agree to execute the SAT on the modified code.

As this architecture relies heavily on networks to provide connectivity as well as system security, a good network design is essential. Multiple subnet architectures with PLC/DCS systems in one network, SCADA/HMI servers in another network, and terminal clients in multiple other networks are essential. Network filtering across network boundaries is a powerful security tool, but only if you properly implement, configure, and monitor it.

Behind the byline

Jack E. Greene is manager, Laboratory and Process Automation Systems at Alkermes, Inc. in Cambridge, Mass.

Copyright Instrument Society of America Apr 2005
Provided by ProQuest Information and Learning Company. All rights Reserved