How SCADA Acquires and Processes Data

As stated above, data acquisition is the process of SCADA acquiring data about the devices in the monitored system. SCADA gets data from a variety of sources, including RTUs, remote SCADA sites, operators, and even custom applications.

“Data processing” involves the conversion of raw data (data acquired from RTUs) into an internal form useful for display and calculations, along with the storage of that converted data in the SCADAMOM database.

Sources of Data

Each piece of data that SCADA acquires has a source from which it is usually acquired. The normal source for much data is the RTUs. However, SCADA can also get data from another SCADA site, a calculation, an operator entry, the State Estimator process, or another process external to SCADA.

Whether a point’s value normally comes from telemetry or some other source, the point can be removed from service by the operator. Removing a point from service prevents SCADA from updating the value from the normal source.

Although SCADA does not update not-in-service points with values from normal sources, it can use a value from a secondary source. For example, consider a transformer measurement that is normally acquired from an RTU. If the analog point for the transformer is removed from service, the operator may want to enter a new value for the point. The manually entered value is a secondary source of data, or a replaced value, for that point.

SCADA processes and stores three different types of data: analog, status, and count data.

Analog Data

Analogs are numeric values representing the state of variable-state devices, such as power lines, transformers, and pumps. Analog measurements are stored in ANALOG records. (Throughout this guide, the terms “analog measurements” and “analog points” are used interchangeably.)

In a monitored system, a physical variable is usually measured by a transducer, and the output of the transducer is passed through an analog-to-digital (A/D) converter in the RTU. The A/D converter produces a number that the host computer can process. All analog values are converted to host computer floating point numbers and adjusted by the SCADA Front End to represent the physical measurement in engineering units — that is, megawatts, or pounds per square inch. In this form, the values are stored in the database and usable for display and calculations.

Each variable-state device in the monitored system is associated with an ANALOG record in the SCADAMOM database. The purpose of the ANALOG record is to identify a single analog data point. Other records subordinate to the Analog record describe characteristics of the analog point, such as limits.

“Limits” are defined by two values that specify a range. For example, a power line may have one pair of limits that define a high limit and a low limit for the line’s normal operational state. Outside those limits, SCADA considers the line to be in an abnormal state and it issues an alarm.

The list below describes the different types of limits that can be associated with an analog point:

Normal limits: Define a range of values within which the device is considered to be operating normally. Normal limits are defined by Limit records.

Reasonability limits: Define a range of values that SCADA uses to determine whether a value retrieved for the analog point is reasonable. If, for example, a value is above the high or below the low reasonability limit, SCADA considers the value to be unreasonable and ignores it; instead, SCADA uses the last retrieved value for the point.

Forbidden range limits: Define a range of values that SCADA considers violated when the analog point falls within the range. Forbidden range limits are used if there is a range of values within the normal range of limits where operation of the device should be minimized or prevented.

Deadband limits: Apply to the return of an analog value from an abnormal to a normal state. For example, for a pair of normal limits, if the low limit is violated, the value must rise above the low limit by at least the deadband amount before SCADA considers the analog to be back within normal limits.

Rate-of-change limits: Define an acceptable rate at which the analog measurement can change between scans. This information is contained in RATLIM records.

Upon receiving a value for an analog point, SCADA checks the validity of this value against the IEEE standard to make sure it can safely use this value. If it cannot, SCADA generates an alarm to the operator and ignores the value. For a very small value, SCADA rounds the received value to 0.0.

As SCADA receives valid data for an analog point, it checks the limits associated with that point. If the analog’s measurement violates any of the limits, SCADA can issue an alarm.

The SCADA operator can inhibit individual limit pairs on an analog point. This prevents SCADA from considering every single violation of the inhibited limits to be abnormal or alarmable.

SCADA has the capability of automatically switching between different sets of limits for a given analog, based on such external factors as temperature, season, etc. This function, called “limit replacement”, can also be invoked manually by the operator.

Status Data

Status values represent the state of discrete-state devices, such as tap changers and valves. SCADA can accept status inputs representing a simple on/off or open/closed input, or a combination of inputs from a three-state device. (An example of a three-state status point is one that has “ON” and “OFF” states and a transition state in between ON and OFF.) All status point values are stored in POINT records in the SCADAMOM database.

With MCD, SCADA can detect multiple changes that occur between scans. This flag is set by the eterracontrol Front End and processed by SCADA.

Some RTUs can time-stamp status changes so that the time at which a device changed state can be precisely determined. Such information, called “sequence of events (SOE) data”, is accumulated by the SCADA Front End, and it can be viewed on a special display.

Count Data

A count measurement is a value from a pulse accumulator. Pulse accumulators are often used to measure the total amount of energy, liquid, or gas that has passed by a specific location in the monitored system. The detection device alternately opens and closes a contact each time a unit of the measured matter passes by it. The pulse accumulator counts the contact changes and passes the count to SCADA, which stores the total accumulation of counts since the last time the count was reset.

Like analog values, count values are stored as floating point numbers and adjusted to represent the physical measurement in engineering units, such as megawatt hours (MWH) or cubic feet. In this form, the values can be used for display and calculations. Each pulse accumulator or counter being monitored is associated with a Count record in SCADAMOM, which stores the count value.

Some RTUs allow pulse accumulators to be “frozen”. This means that, at a particular point in time, a “freeze” command can be issued to the RTUs to simultaneously capture current running accumulations. The values are stored by the RTU until the various accumulator points throughout the system are scanned; at that point, the values are sent to the SCADA Front End.

When accumulator value data is received by SCADA, its validity against IEEE coding is checked the same way the analog value is checked.

SCADA's Scanning Capabilities

The SCADA Front End gets data about the monitored system by scanning RTUs that are wired to receive analog, status, and count measurements. The data retrieved in a scan depends on which RTU scan groups (also called “addresses”) are being scanned.

A “scan group” is a set of data points retrieved in a single command to the RTU. Usually, the most important status points are defined as one scan group and retrieved every two seconds. Less-important measurements can be defined as a different scan group and retrieved less often — say, every 10 seconds.

Data Quality

“Data quality” is a catch-all term relating to information that the SCADA operator may want about the values SCADA is displaying for monitored devices. For example, the operator may ask the following questions about a particular value:

Where did the data come from? For example, was it telemetered or manually entered?

How accurate is the data? Does it reflect the actual state of the monitored device?

Is the monitored device in an acceptable or unacceptable operating state?

How is State Estimator using the data?

To provide the operator with answers to these and other questions, SCADA supports five groups of data quality indicators that can be displayed on SCADA one-line and tabular displays. (Data quality indicators are also referred to in this guide as “data quality flags”.)

Not all of these groups of data quality indicators are likely to be used in a single SCADA system, but all five groups of flags available for use on one-lines and tabulars are described here.

Data source flags: These indicate the source of a value — for example, whether it comes from an external program such as the State Estimator or a remote site. Data source flags are usually seen on tabular displays.

Detailed data quality flags: These indicate the reliability of data — for example, whether the data is old or bad. Detailed flags are commonly used on one-lines.

Composite data quality flags: These also indicate reliability, but in a more-general way than detailed data quality flags do. Composite flags are commonly used on one-lines.

State Estimator flags: These relate to the State Estimator’s use of SCADA data.

Data attribute flags: These indicate whether a device in the monitored system is in an acceptable or unacceptable operating state. For example, if an analog measurement exceeds its limits, it can be flagged as unacceptable.

The specific flags and their meanings are explained in the following sections.

Data Source Flags

Data source flags indicate where a measurement came from — for example, whether it was acquired from an SCADA Front End, entered manually by an operator, or supplied by the State Estimator software. Data source flags are usually used on tabular displays rather than one-lines. The flags are as follows:

NREMOTE: The value is normally reported to SCADA by a remote site, by an SCADA Front End, or by eterracomm. These kinds of measurements are called telemetered.

NCALC: The value is normally calculated by SCADA.

NMANUAL: The value is normally entered manually by the operator at the host SCADA site.

NESTIM: The value is normally supplied by State Estimator.

NEXTERN: POINT, ANALOG, and COUNT data is supplied by a process external to SCADA’s scanning function. For example, data can be supplied by an application in a different part of the Energy Management System (EMS). These are not telemetered and the tiers program typically uses SCADA API to provide the value by opposition of the telemetered measurement, which receives its value by the Inter Site Data protocol.

Only one data source flag appears for a given data point. If the value shown for a data point was not acquired from the normal source, then a Replacement flag appears next to the value, indicating the secondary source of the value (such as operator entry or State Estimator).

Data Quality Flags

There are two sets of data quality flags: detailed and composite. Both indicate something about the reliability of data. Detailed flags provide more insight into the quality of data than composite flags do.

Whether they are detailed or composite, data quality flags are usually displayed adjacent to measurements on one-lines. For example, an “O” may appear next to a monitored point to indicate that the displayed value for the point is “old”. The absence of a data quality flag next to a measurement indicates that the measurement was retrieved successfully in the last scan and is reliable and accurate.

Detailed Data Quality Flags

The detailed data quality flags are as follows:

Un-initialized (U): This means that no data has been retrieved and stored for the measurement point. Operators should consider measurements with “U” flags as meaningless. When SCADA is started up, a “U” flag usually appears for all measurements. As scanning resumes and new data is retrieved and stored, “U” flags start disappearing. While the new data is coming in, no alarms are issued for the changing data since the un-initialized values are not considered meaningful by SCADA.

If a measurement is supposed to be used in a calculation, then the calculation is generally not done when a “U” flag is present.

Note that the measurement is set to uninitialized in two cases:

After a cold start

After retrieving a new scada savecase without using the selcop option

Otherwise, SCADA will start by setting all measurements suspect if they received at least one value after the cold start.

Old (O): Data is marked “O” if it could not be retrieved in the last scan. It is the last valid data received from the RTU. Data may be considered old by SCADA for these reasons:

In a “warm” system startup, where a new SCADAMOM database is merged with the most recent SCADA data and SCADA is restarted, all measurements acquired by SCADA Front Ends or remote sites are marked “O” until scanning brings in new data.

A measurement failed a reasonability check, which means it is not stored in the database. When this is the case, the Old flag is accompanied by an Unreas flag. The Unreas flag indicates a value that is old because a routine has attempted to store an unreasonable value.

For analog data, UNREAS indicates that the value violates a reasonability limit for the device.

For status data, UNREAS indicates that the monitored device is in an illegal state. The Unreas flag does not apply to count data.

The data point has been placed out of service by an operator to prevent new values from being stored and displayed (in other words, the normal source of the data has been temporarily blocked).

An unrecoverable communications error occurred while the data was being retrieved from an RTU or while it was being sent from a remote SCADA site. SCADA considers the data old until the data can be successfully retrieved from the RTU or remote site. Communication errors can affect more than one value coming from an RTU; they can affect one scan group, all the scan groups handled by an RTU, or all data being shipped on a particular communications line.

The measurement has been set “Not In Service” (see NIS).

Bad (B): Bad data is unreliable. It can be caused by a calculation that produces unexpected results or by bad circuitry in the RTU.

NIS (NIS): The data point is not in service. Data labeled “NIS” represents a measurement point that has been removed from service. When in the NIS state, SCADA does not process and store new values provided either by calculation or telemetry for the measurement point.

Composite Data Quality Flags

Like the detailed data quality flags described previously, composite flags indicate a measurement’s reliability. However, composite flags provide a quicker way for the operator to judge reliability.

There are four composite flags – GARBAGE, SUSPECT, REPLACED, and GOOD – that are defined below:

GARBAGE: Data is unreliable. This flag appears when the measurement point is un-initialized.

SUSPECT: Data is labeled with one or more of these flags: OLD, BAD, OVER, or REMSUSP (defined below), TFAIL (telemetry failure), UNREAS, or STALE.

REPLACED: Data is labeled with one or more of these flags: MANREP, ESTREP, REMREPL (defined below) or GENREP.

GOOD: Data is not labeled as GARBAGE, SUSPECT, or REPLACED.

Remote Site Scanning Flags

For data received from an SCADA Front End or from a remote site, two data quality flags are significant:

REMSUSP: Remote site value is suspect. Measurements labeled REMSUSP have been marked OLD, BAD, or OVER at the remote site or SCADA Front End.

SCADA can be configuredto reject suspect values from remote sites by selecting the BLOKSUSP option on the SITE record.

REMREPL: Remote site value is replaced. Measurements labeled REMREPL have been replaced by the operator or State Estimator at the remote site, and so are marked either MANREP or ESTREP.

Hierarchical Flags

The hierarchical flags indicate the quality of the measurement inherited from the bay or the substation quality they belong to (measurements modeled into this bay/station).

Starting with eterrascada 2.6, the following hierarchical qualities (HIERFLAGS) were added for analogs, status points, and counts:

HIERNIS: The substation/bay has been NISed so that the measurement is hierarchically NISed.

HIERINHIBIT: The alarms for the substation/bay have been inhibited so that the measurements’ alarms are inhibited too. Counts do not have this flag.

HIERTST: The measurement is in hierarchical test mode state (from substation or bay).

The hierarchical quality has the same effect on the measurement as its self quality, with the sources being different. The main difference is the fact that these measurements will not appear in the list; only the parent will.

Starting with eterrascada 3.0, substation’s quality also impacts the quality of its children the same way bays do. To impact the bays as well, DEVTYP records also have NIS, Inhibit and Test hierarchical flags, though stored in DFLAGS rather than in a separate HIERFLAGS bitfield.

Effective Flags

When there must be a differentiation between the hierarchical quality of a measurement and its self quality, the effective quality has to be computed. eterrascada 2.6 uses the old quality flags (DFLAGS) for the effective quality for NIS and INHIBIT. SCADA proposes two new flags for the hierarchical quality (see above) and the self quality (SELFFLAGS).

Self Flags

The “SELF” flags indicate which quality flags are set at the measurement level of the measurement object (i.e., analog, count, or point).

Starting with eterrascada 2.6, the following quality flags (SELFFLAGS) are available:

SELFNIS: Measurement objects are set “Not in Service” individually. This flag is managed by the user.

SELFINH: Measurement objects are set “Inhibited” individually. This flag is managed by the user. Counts do not have this flag.

SELFTST: Measurements are in a self test mode state.

Starting with eterrascada 3.0, SUBSTN and DEVTYP records also have self NIS, Inhibit and Test flags, though stored in DFLAGS rather than in a separate SELFFLAGS bitfield.

State Estimator Flags

There are two flags related to the State Estimator, which uses SCADA data in its estimation of the system state:

ANOMAL: Anomalous value — State Estimator considers the data anomalous. ANOMAL is not a data quality flag per se; if State Estimator detects that the values for a particular measurement point do not fit well into its system state solution, then it considers the measurement point anomalous. When the incoming data improves, State Estimator removes the ANOMAL flag.

ESTREP: State Estimator–replaced value — the value for a measurement point is overridden, or replaced, by the State Estimator. The replacement may be by operator request, or automatically if the ANALOG/POINT is configured for automatic State Estimator replacement and certain conditions are met.

The ANALOG/POINT can be configured to be automatically replaced with good-quality State Estimator data if telemetry from its normal source is unavailable or of suspect quality. The measurement can also be configured to automatically use the State Estimator value if the telemetered value and the State Estimated value vary by more than a programmer-defined amount.

Data Attribute Flags

Data attribute flags indicate whether measurements reflect an acceptable or unacceptable operating state for a monitored device. These flags are generated as a result of data checking, which compares a point’s value against database-defined limits and normal states.

Data checking is done for all data: telemetered, calculated, manually entered, and program-entered. Depending on the results of the checks, the following data attribute flags may appear for a point:

CMD: Indicates a commanded state for both analog and status points.

ABNORMAL: Indicates an abnormal state for status points.

LIM1, LIM2, LIM3, LIM4, LIM5: Indicates an abnormal condition for an analog point. Five abnormal conditions can be defined in the SCADAMOM database; each condition has its own flag.

NOLIM: Indicates that an analog point is not violating any of its limits (that is, the analog point is not abnormal).

TFAIL: The value cannot be polled from any remote source.

Replace flag:

MANREP: The normal value has been replaced by the operator

ESTREP: See section 1.5.6.3 State Estimator Flags.

GENREP: The normal value has been replaced by the generation

AUTOREP: The normal value has been automatically replaced by the current source.

BYPASSED: The current value has been replaced by the operator without using the NIS state from an un-initialized value. The BAD quality has been bypassed.

State Attribute Flags

These flags represent the current state of the measurement

INHIBIT (not COUNT): The measurement is alarm inhibited. Depending on the current inhibition flags, SCADA will not log any event, and will not consider the measurement abnormal or unacknowledged.

NIS: The measurement is Not In Service. SCADA ignores data coming from the source and the operator is able to replace the value with any manual or estimator value.

TESTMODE: The measurement is in test mode. This specific mode changes the Area of Responsibility of the measurement; the alarms are logged separately from the normal alarms, and the measurement can be manipulated by an operator that usually does not have access to this measurement.

STALE (ANALOG only): The value from the normal source has not changed in the configured time frame.

UNACK: The current measurement is unacknowledged. This means an alarm has been generated and needs to be acknowledged by the operator.

DELAYED: The alarm has been delayed. If the measurement becomes normal again, no alarm will be generated.

Initial Values of Flags

Data points are assigned either Uninit (un-initialized) or Old flags, depending on whether SCADA undergoes a “cold start” or a “warm start”.

In a cold start, SCADA is started with a new database and no existing values for monitored data points. With this startup, all telemetered, calculated, and extern data points are marked UNINIT.

In a warm start, SCADA is started with a new database and existing values (usually the most recently retrieved values). In a warm start, measurements coming from their normal sources are marked OLD. This flag remains until scanning brings in up-to-date values.

Oscillation Detection and Temporary Inhibition on a Point Measurement

SCADA can monitor unexpected changes to a point measurement during a defined time window and set the measurement in a temporary inhibition state for a defined period.

This SCADA feature can be enabled for each point measurement by configuring the specific oscillation detection parameters and temporary inhibition settings. Note that this feature needs to be enabled for the entire application before being allowed on a POINT measurement basis.

The following parameters are modeled and available on the SCADA Online Editor display.

Oscillation Detection frame (time window in minutes or seconds): The value must be positive to enable processing.

Oscillation Detection maximum number of changes: The value must be positive to enable processing.

Temporary inhibition frame (time window in minutes or seconds): A zero value indicates permanent inhibition.

Once the maximum number of changes is exceeded, the point is placed in (temporary) inhibition and the user can manage the temporary inhibition state on this point. For more information, see section 2.2.5 Temporarily Inhibiting Alarms from a Tabular.

Stale Detection on Analog

SCADA also provides a feature to detect if an ANALOG value is not stalled. This may hide a communication failure or physical device problem, since it is very rare to have a real value not changing.

This feature has to be enabled for the entire SCADA application and then at the ANALOG level (as the oscillation detection feature).