Pick up any trade journal, watch the news, or sit in on discussions at technical conferences these days. Before long, you’re thinking that all problems in a plant can be solved by way of complex data analytics and such-and-such high-tech device or solution. Not so fast.
Industrial -user issues regarding Artificial/Augmented Intelligence (AI), Machine Learning (ML) and the Internet of Things (IoT) aren’t necessarily settled business. Two questions at the end of the recent 29th Annual Conference of the Society for Maintenance and Reliability Professionals (SMRP, smrp.org) in St. Louis, MO (Oct. 25-28, 2021), were a case in point.
Posed to the featured Industry 4.0 panelists during the event’s closing session, those questions were reflective of several earlier discussions among SMRP members. I’ve heard variations of them asked before (many times) in the past few years. Concrete answers have been few, if any. Moreover, the most vocal respondents often seem to be individuals advocating for specific technologies or vendors. I’ve paraphrased the two questions here:
1. What standards are out there, and where to we find them?
2. Many AI, ML, and IoT options claim to provide exact expectations,
such as time to failure or remaining life. How is this possible?
In St. Louis, the first question was asked by reliability manager from a major corporation, the second one came from a well-known subject matter expert for predictive technologies and applications within industry. Instead of focusing on applications, reliability, or security, these questions were more general in scope. To learn more about the questioners’ particular concerns, I spoke with them after the conference concluded.
REGARDING STANDARDS
Intelligent-system concepts have been around for decades, with the basis of ML and AI showing up in areas of computer science and industriaengineering statistics. The names have evolved though, often for marketing-related reasons. Consider VRML (Virtual Reality Modeling Language) models of the 1990s that became known as “digital twins” in the late 2010s, and the term “Neural Networks,” which had only been discussed in high-level engineering-society and academic research in the mid-1990s, coming into common use two decades later.
The application of those data systems has accelerated greatly in recent years, to the point that it’s virtually impossible to keep up with all the new vendors and concepts associated with “digital transformation” (formerly known as “automation”). Living and working in this period, we are truly enjoying the transition to what was only viewed as science fiction during the past four decades.
Within the United States, the U.S. Department of Commerce (commerce.gov) had the task of coordinating stop-gap frameworks for the application of smart technologies, with a focus on smart cities and smart grid. It has done so through programs primarily investigated and initiated by the National Institute of Standards and Technology (NIST, nist.gov). These efforts, in turn, helped accelerate development of standards within such organizations as the Institute of Electrical and Electronics Engineers (IEEE) for ML, AI, and IoT devices that have already been published or will be.
Other involved standards groups have included the International Electrotecchnical Commission (IEC) with some level of harmonization, and the United Nations recommending the implementation of ethical digital-transformation standards. The most significant challenge to the rapid development and deployment of standards has been harmonization issues, which can generate problems in communications between vendors and end-users when one side is referencing a standard from a certain organization and the other is using a different standard).
Several hundred standards have already been developed within IEEE. Among other things, they cover commercial, industrial, and healthcare operations, data-centers, utilities, computer science, cybersecurity, and ethical design associated with how the information is transported and managed. And more IEEE standards are on the way, with the first ones now moving through second and third iterations. Outside all the noise around standards development, however, it’s important to point out that definitive ones are being produced for use by end-users and vendors alike.
For example, IEEE 2413-2019, “IEEE Standard for an Architectural Framework for the Internet of Things (IoT),” provides an outline from everything related to common architectures for handling information across all sectors and servers to cloud and edge. Another one (just voted on) is P2994, “Standard for Security Assessment Framework for Internet of Things (IoT) Application Deployments.” It provides guidance on assessing IoT devices for cybersecurity.
Note: Regarding P2994, the ‘P’ in front of the standard number and no “-date” at the end represents an in-development standard that will then change to a number-date format once it is finally released as an official document. It’s also important to note that many of the types of standards addressed by IEEE are now harmonized, i.e., associated or adopted with those from the International Standards Organization (ISO) and IEC.
For more information on specific IEEE standards, go to https://standards.ieee.org and search by topic.
TIME TO FAILURE ESTIMATION (OR RUL)
I once presented a paper at an IEEE conference referencing predictive maintenance (PdM) of electrical- insulation systems. The challenge to this paper was that if you “predict” a failure will occur in 100 hours and it occurs in 99 or 101 hours, then the prediction was wrong. This, in essence, was a problem with the definition of terms.
In the maintenance and reliability world, the term “predictive maintenance” has a specific meaning, while in the engineering world it has a different meaning. To add to the confusion, in reliability, maintenance, and other asset-management activities, the concept of risk is often forgotten or not properly understood. To management and anyone outside our world, this seems to mean “an exact science predicting when equipment will fail.” As a result, we have often run into devices that use modeled information to predict “exactly” when something will cease to operate. And they’re not accurate in the field.
(Note: I retitled my cited IEEE paper as, “Time to Failure Estimation of Insulation Systems,” and it was reprinted several times. A 2009 copy can be accessed at this link: Time to Failure Estimation – MotorDoc LLC. I later expanded that paper under the title, “Evaluating Reliability of Insulation Systems for Electric Machines,” and it was published through IEEE’s 2014 Electrical Insulation Conference. A copy of this 2014 version can be accessed at this link: Evaluating Reliability of Insulation Systems for Electric Machines – MotorDoc LLC
As I’ve discussed in papers and articles regarding Remaining Useful Life (RUL) or Time to Failure Estimation (TTFE), we are looking at the probability of defined failure within a period of time. For this reason, we need data and some type of history or comparison, which is what AI/ML/IoT is supposed to automate. Whether we are using modeled or real data, the complexity of an actual failure and variability of real-life loading of equipment, transients, environment, and other conditions that need to be factored in for a specific device and failure mode don’t allow for reasonable exact time to failure. Instead, we deal in probability of survival or failure with a confidence band (as in there is a 60% chance the bearing will last another 60 hours, with a confidence of 80%).
If someone is guaranteeing that a bearing will last 60 hours after a fault or degradation is detected, he/she deserves to be seriously questioned. If the system in question were unsupervised (as opposed to a supervised system for which training data was provided), the accuracy for most industrial systems, such as motors, would have to be questioned further (see my series on “Machine Learning With Raw Electric Motor Data,” links below).
Click The Following Links For Articles In The Referenced Machine-Learning Series
Part I (Aug. 8, 2021)
Expectations with present AI/ML/IoT systems are probably on the high side with most of the marketing presented by the various device and software vendors suggesting perfect applications. The reality is that in discussions with data scientists the recommendation is reviewing the problem and the data in order to determine if such an approach is even necessary, or if a simple expert or alert system would suffice. This is, of course, if AI/ML/IoT are defined individually or collectively as fully autonomous systems versus an expert or alarm systems that trigger review of the data.
The most successful systems in areas of vibration and electrical signature analysis, for example, are those using industry-accepted alert and fault levels and bands. The majority of these already incorporate trending and fault-detection and -identification capabilities yet remain relatively simple to use (as opposed to systems that require users to call upon vast libraries for fault analysis [classification] and detection). At this point, the primary difference between the most successful systems and others is the use of experienced personnel versus technology to determine the risk of failure over a specific period of time. Which is more accurate?
The challenge with both supervised and unsupervised learning systems, whether you refer to them as AI or ML, is bias by the developer. Regardless of anything else, the system is still designed by a human and the installed libraries are human which means that bias will be present. In effect, the accuracy of either a human-based expert system or an AI/ML system is going to be relatively the same when you take into consideration the bias of the human, whether he/she is a technician or subject matter expert used by the programmers, at which point you also include the bias of the programmer.
WRAPPING IT ALL UP
AI/ML/IoT systems standards have been developed and are being put to use. The importance of those standards is that they provide a consensus for vendors, end-users, and other stakeholders on what is to be expected. In this article, we referenced IEEE, IEC and ISO, although there are a number of other standards-writing organizations, including UL (Underwriters Laboratories), that cover the gamut of applications. As a stakeholder, you will want to define which standards are being used for your application up-front, since some standards are not harmonized.
AI/ML deal with statistics and probabilities based upon provided data to develop algorithms used and systems developed by human beings. This means you are dealing with bias, as well as with the chance of survival or failure during a given time. Due to all the variables involved, there are no accurate systems that can provide exact remaining life or exact time to failure. In the end, due to a combination of these effects and the bias from subject matter experts and programmers, the accuracy between an expert system and technician versus an AI/ML system are subjective and based upon the human element of either one.
In the end, AI/ML/IoT are in their infancy. We are still discussing the topic of Industry 4.0, which hasn’t changed since about 2005, when these concepts began gaining ground, and the experiences and confusion of end-users caused many to pause their efforts. (Did devices or software meet their expectations?) Even now, as industrial operations move forward on their digital-transformation journeys, they can expect to be confused at times. After all, countless players are leaping into the market, and we’re onboarding a whole new group of reliability and maintenance experts known as “data scientists.”TRR
ABOUT THE AUTHOR
Howard Penrose, Ph.D., CMRP, is Founder and President of Motor Doc LLC, Lombard, IL and, among other things, a Past Chair of the Society for Reliability and Maintenance Professionals, Atlanta (smrp.org). Email him at howard@motordoc.com, or info@motordoc.com, and/or visit motordoc.com.
Tags: reliability, availability, maintenance, RAM, asset management, electrical systems, motors, drives, industry standards, Artificial/Augmented Intelligence, Machine Learning, Internet of Things, IEEE, ISO, IEC
