Before you build a Predictive Maintenance model, check the data it will rely on

Sebastian Dudzik
Manufacturing

Even the best model will not compensate for errors in the input data

Predictive Maintenance is meant to detect deteriorating machine condition early enough to assess the risk of failure and plan the right response. The effectiveness of this approach depends not only on the algorithms used, but above all on the quality of the data those algorithms are built on.

In practice, historical data on failures, maintenance work, operating cycles and measurement signals is rarely ready for analysis straight away. Missing measurements, inconsistent part identifiers, an incomplete event history or signals containing a double-digit percentage of missing values can easily go unnoticed during a quick review of the data. At the same time, each of these issues can significantly affect the results of later modelling.

This is the classic garbage in, garbage out problem. A model may be technically well trained, yet its results will be based on an incomplete or inconsistent picture of how the machine actually operates. Such an error often does not surface during training. It becomes visible only once the predictions stop matching the real behaviour of the equipment.

The Predictive Maintenance Validator was developed to detect data problems before modelling begins, rather than after the solution has been deployed.

The tool can be used both during a Predictive Maintenance pilot and in the day-to-day operation of a deployed solution. In a pilot, machine data is usually delivered as files, before the target integrations with source systems have been built. Being able to verify its quality at that stage matters a great deal, because preparing a full set of integrations is time-consuming and costly, and its value depends on whether the available data genuinely supports a reliable model. Early validation therefore makes it possible to decide on the scope of the work before that investment is made. After deployment, the same tool is used for recurring checks of the data feeding the models.

What the Predictive Maintenance validator checks

The Predictive Maintenance Validator is not a general-purpose tool for checking Excel and CSV files. It reflects the structure of the data used in Predictive Maintenance projects, including:

  • the history of events, failures and maintenance work
  • data on the operating cycles of parts and components
  • machine parameters
  • process and measurement signals from sensors

Validation is carried out in three consecutive phases.

Phase 1 – completeness of the file set

In the first step, the Predictive Maintenance Validator checks whether the basic data set has been delivered and whether the files can be correctly assigned to individual machines.

The checks include, among others:

  • the presence of the main file with the list of machines
  • the presence of an event history and parameters for every machine
  • consistency between the machine identifier and the names of the files assigned to it
  • the availability of files containing process signals

Missing files required for further processing produce an error. Missing sensor data, by contrast, may be recorded as a warning, because its significance depends on the planned analytical scope.

If critical errors are found in this phase, the remaining validation stages are not performed. The user receives a report indicating which elements of the data set need to be completed or corrected.

Phase 2 – structural validation

The second phase checks whether the individual files are correctly constructed.

The Predictive Maintenance Validator verifies, among others:

  • the presence of the required columns
  • consistency of machine identifiers
  • data types
  • date and timestamp formats
  • positive integer values in the fields where they are required
  • values drawn from predefined dictionaries
  • the format and maximum length of part names
  • the correctness of numeric values in files containing measurement data

For example, the event type may be checked against a dictionary containing the values FAILURE, PLANNED_MAINTENANCE and UNPLANNED_MAINTENANCE. This prevents the same type of event from being recorded in several different forms that would be treated as separate categories during modelling.

Phase 3 – semantic validation and cross-file checks

A correct structure in every file does not yet mean that the data set as a whole is consistent. In the third phase, therefore, the Predictive Maintenance Validator analyses the relationships between the files assigned to the same machine.

It checks, among others:

  • whether the minimum number of operating cycles is available for each part
  • whether the minimum number of events required for further analysis has been recorded
  • whether the share of missing values in the measurement data stays below the configured threshold
  • whether the required number of process variables is available for the machine
  • whether the parts appearing in the event history are also defined in the machine parameters

These thresholds are not a universal guarantee that a reliable model can be built on the data. They define the minimum requirements agreed for a specific project and make it possible to assess whether analytical work can begin and which data limitations need to be taken into account.

What validation looks like in practice

The Predictive Maintenance Validator is launched with a single command. The tool analyses the data in the input directory and then generates a validation report as an Excel file containing a sheet that summarises the validation result. Depending on the issues detected, the following sheets are also created:

  • error – issues that block further processing or the start of the analysis
  • warning – issues that need not block the work, but may limit the quality or scope of the future model
  • info – additional diagnostic information

The basic information in this report includes:

  • the rule code
  • the severity level
  • the file name
  • the machine identifier
  • the name of the part or column
  • a description of the detected inconsistency
  • the number of cases violating the rule
  • the recommended corrective action
rule_codestatusfiledescription
CSV_FLOAT_VALUEERRORM1_temp.csvInvalid numeric value in the value column
MAX_MISSING_RATIOWARNINGM1_temp.csvThe share of missing values is 15%, against a configured limit of 10%

The first of the issues shown in the example above requires the data to be corrected before further processing. The second points to a data quality limitation that must either be removed or consciously factored in when defining the feasible analytical scope.

As a result, the data owner receives a concrete list of items that need to be corrected, instead of a general message stating that data quality is insufficient.

Configuration tailored to the project

Data requirements are not the same for every machine and process. A model for equipment running continuously will require a different history than the analysis of a machine that is started periodically and operates in several modes.

For this reason, the rules and thresholds used by the Predictive Maintenance Validator are defined in the config.yaml file. The configuration may cover, among others:

  • the required date and time format
  • the maximum length of a part name
  • the minimum number of cycles
  • the minimum number of events
  • the minimum number of process variables
  • the maximum acceptable share of missing values
  • dictionaries of event types
  • dictionaries of part replacement reasons

Individual rules can be enabled and disabled without modifying the source code. This makes it possible to adapt validation to the scope of the pilot project, the characteristics of the equipment and the data available on the client side.

This does not mean abandoning quality standards. On the contrary – the requirements are stated explicitly, applied repeatably, and can be agreed before modelling begins.

Verified operating logic

The Predictive Maintenance Validator is covered by an extensive set of automated tests: more than 120 unit tests and 17 integration scenarios.

The tests cover, among others:

  • a correct data set with no errors
  • missing required files
  • invalid names and identifiers
  • structural errors in Excel and CSV files
  • an insufficient number of cycles or events
  • exceeding the acceptable share of missing values
  • too few process variables
  • cases sitting exactly at the configured thresholds
  • multiple errors occurring at the same time
  • data sets covering several machines of differing data quality

The unit tests verify the behaviour of individual components and rules. The integration scenarios, in turn, run the full validation process – from loading the data to generating the result.

This approach to testing improves the repeatability of validation and reduces the risk that changing one rule will affect other parts of the process in an uncontrolled way.

The first quality gate in a Predictive Maintenance project

The Predictive Maintenance Validator acts as the first quality gate in Predictive Maintenance projects and pilots delivered with Smart RDM.

Before the analytics team starts preparing indicators, risk models, decision thresholds and forecasts, the following has to be established:

  • whether the data is complete and correctly structured
  • whether it can be unambiguously assigned to machines and components
  • whether the event history is consistent with the operating data
  • whether the available signals provide sufficient coverage
  • which analytical results can realistically be achieved
  • which data needs to be completed before modelling begins

The Predictive Maintenance Validator does not assess the effectiveness of the future model and does not replace the analysis carried out by data experts and maintenance specialists. What it does is answer an earlier and more fundamental question:

does the data meet the agreed conditions for starting analytical work, what are its limitations, and what specifically needs to be corrected?

Detecting problems at the beginning of a pilot avoids the situation in which, after many weeks of modelling, the team discovers that some events were never recorded, that component identifiers are inconsistent, or that key signals contain too many gaps.

In Predictive Maintenance, credible analysis begins not with the choice of algorithm, but with a check of the data that algorithm is meant to rely on.

Light mode