한지훈
(Ji-Hoon Han)
1iD
최동진
(Dong-Jin Choi)
1iD
박상욱
(Sang-Uk Park)
1iD
홍선기
(Sun-Ki Hong)
†iD
-
(Dept. of Information Control Engineering, Hoseo University, Korea.)
Copyright © The Korean Institute of Electrical Engineers(KIEE)
Key words
Motor fault diagnosis, Deep learning, GAN, Lack of the data, DT-CNN
1. Introduction
One of the areas with insufficient data is motor fault diagnosis. The normal state
data of the motors are very easy to obtain. However, fault state data is very difficult
to acquire because the fault signal of the motor changes depending on the driving
environment and the specifications of the motor. Normal state data is also affected
by these factors. The normal state signal of the motor can be collected in the driving
environment. However, the fault signal of the motor is difficult to collect in the
environment. Previous studies have not considered this problem (1)-(3).
In previous studies, experiments were carried out under the assumption that the motor
failure signal has a specific pattern for each failure. However, the fault signal
of the motor is changed by external factors such as the degree of the fault and the
sensor (4)-(5). Nevertheless, the data accumulated through the study of fault diagnosis methods
for a classic motor is meaningful. However, these are not the data of the motor to
be diagnosed and are not enough to use when developing a new diagnostic algorithm.
Therefore, some additional data are required to train the deep learning algorithm.
To solve this problem, a virtual data generation technique through a Generative Adversarial
Network (GAN) was used. Through this algorithm, insufficient motor fault data can
be complemented. Data created with GAN is created with the goal of becoming as similar
to existing data as possible. An increase in the number of most similar data does
not mean that the model’s performance will improve the most. When considering the
classification accuracy, the degree of overfitting, and the classification performance
of outlier data representing the perfor- mance of the model, the generated data is
similar to the insufficient data, but must have an appropriate level of variance.
Previous studies have not considered a method of finding data that maximizes model
performance by focusing only on increasing the number of scarce data (6)-(7). When only the data that is most similar to the insufficient data is added, the following
problems occur. Therefore, if only this algorithm is used, the following problems
arise. The generated virtual data is based on existing fault data.
There is also a problem in generating completely new fault signals when generating
data because there is no conviction that this generated data is a real fault signal.
The generated virtual data should not be too similar to existing fault data because
adding very similar data does not increase the variance of the data. However, a problem
arises if the data are too different from the existing data in that a failure state
is regarded as not occurring. Data that increase the quality and number of failure
data should be generated. This requires a technique for evaluating the generated virtual
data.
The methods of evaluating the generated virtual data include the Root Mean Square
Error (RMSE) and coherence, which are commonly used (8). These are ways to numerically assess the similarity of data. However, when using
deep learning, an important factor is the output of the model with the data as input.
The data evaluation method is used as an auxiliary method for evaluating the similarity
of data. To evaluate the performance of the generated virtual data, an overfitting
model with existing fault data is used. A technique is proposed for inputting the
generated virtual data into this model and analyzing the performance of the data according
to the accuracy of the data. A technique is also proposed for updating fault diagnosis
algorithms using deep learning by using the analyzed virtual data.
2. Proposed Method
2.1 GAN Algorithm
The GAN algorithm generates training data for deep learning algorithms. This algorithm
generally consists of two neural networks. One of the neural networks is designed
to produce data similar to the input data. However, it is difficult to evaluate generated
virtual data objectively with a single neural network. Therefore, a neural network
is also used for classifying the generated virtual data. The generator for generating
virtual data is denoted by G, and the discriminator is denoted by D.
The GAN algorithm is a breakthrough technology and is likely to be used in many fields
that use deep learning. However, training this algorithm is very difficult. To solve
this problem, some studies have been carried out to find the optimal GAN structure
[9~10]. However, the fault diagnosis signal of the motor has a relatively simple pattern
and does not require a complicated GAN model. Therefore, a GAN composed of neural
network with one layer is used. The loss function for learning this GAN is shown in
(1) (11).
The purpose of (1) is to minimize the loss function output of G and maximize the loss function output
of D. (1) trains G to produce better virtual data by sequentially learning G and D. z is a
random vector that is input to G. The training data used to train G is not the input
data x. x is used for teaching D and G is taught through a random vector z. The output
of G is checked by D. The structure of a GAN with one hidden layer is shown in Fig. 1.
그림. 1. GAN의 구조
Fig. 1. The structure of the GAN
Hyper-parameters such as the number of hidden layer neurons and the learning rate
used in the GAN are values arbitrarily set by the user. When generating virtual
data using GAN, the performance varies greatly according to these internal parameters.
Through (1), the learning progress of the GAN model can be identified. However, it is not enough
to judge whether the generated data can be used as training data for other deep learning
algorithms. This is an important issue in this area, where data must be obtained and
updated in real time. Therefore, a criterion is required to evaluate the data generated
through GAN.
2.2 Data Evaluation Method
The data evaluation method is focused on motor fault diagnosis. There is insufficient
data about a fault condition. Since this study is on fault diagnosis rather than failure
prediction, the necessary data is the signal generated when the motor has a specific
fault. The fault signal of the motor changes according to the degree of the fault
or the driving condition and fault signals have a specific pattern depending on the
fault condition. This has been demonstrated in classical fault diagnosis studies (12). The generated virtual data need to have the pattern of the existing fault signal.
However, when creating virtual data using the fault data that a developer has, the
created data should not be too similar to the input data because additional data with
the same information is created and not new data. This problem requires an algorithm
that evaluates whether the data can be used for deep learning algorithms. RMSE is
a method of numerically evaluating the data produced. The RMSE equation is shown in
(2).
(2) is the average of squared differences of the data. In this calculation, the similarity
of the data can be expressed numeri- cally. However, when only numerically expressed
similarities are used, the data cannot be objectively evaluated. Fig. 2 shows the gear fault data of a test motor and the virtual data generated by GAN. As
can be seen in Fig. 2, the 100 and 200-epoch data show a similar pattern. Table 1 shows the RMSE values calculated from the data in Fig. 2.
With only the values in Table 1, assessing the similarity of the data is objectively difficult. The calculated RMSE
values are not significantly different from each other. This means it is difficult
to determine a reference value among the calculated values. To use a calculated value
as a meaningful value, a different process is required. The simplest way to solve
this problem is to determine reference data that can be used as training data and
calculate the numerical value based on the RMSE of the reference data. The most obvious
way to ensure the generated virtual data can be used as training data is to use a
deep learning model that has been trained. In this case, the model is overfitted with
the original data to make the virtual data. That is, a model overfitting the failure
data of the motor is used. The overfitted model can be created by adjusting hyperparameters
using genetic algorithms (13).
표 1. 에폭에 따른 RMSE 값
Table 1. RMSE value at each epochs
|
|
RMSE
|
|
100 Epochs
|
0.005523194
|
|
200 Epochs
|
0.004727735
|
The reason for using the overfitted model without using only the discriminator inside
the GAN is as follows. If the model is overfitted with one class of data, the model
classifies the other data as another, even if the input data is slightly different
from the overfitted data. If the model classifies the data with high accuracy when
inputting the generated virtual data, it means that the input data is very similar
to the training data of the model. In contrast, the classification accuracy of the
model close to 100\% means that the generated virtual data is not different from the
data that the developer has. That is, the classification accuracy of the model should
be analyzed when the generated virtual data is input to the overfitted model.
그림. 2. GAN으로 생성된 가상 데이터들
Fig. 2. The generated virtual data through GAN
At first, the optimal model accuracy range is assumed to be 80 ~ 90\% because the
overfitted model is used. This range is not too similar or different from overfitted
data. The evaluating equation is recalculated based on the largest accuracy data among
the generated virtual data groups with an accuracy of 80 to 90\%.
${R M S E}_{\max}$ means the RMSE with maximum accuracy among the data that outputs
80 - 90\% accuracy. ${R M S E}_{{now}}$ is the RMSE between the data to be calculated
and ${R M S E}_{\max}$. In addition, (3) calculates an objective RMSE value using the data outputting the least accuracy among
the generated virtual data through the GAN. Fig. 3 shows the RMSE values calculated through (3).
그림. 3. 에폭 당 RRMSE 값
Fig. 3. Epoch per RRMSE value
2.3 Motor Fault Diagnosis Using GAN
This section introduces how to complement the fault diagnosis algorithm by using generated
virtual data and an overfitted deep learning model. When performing fault diagnosis
using these data, a DT-CNN is adopted (14). A DT-CNN is similar to the concept of a one-vs-many SVM algorithm. This algorithm
builds a decision tree using overfitted CNN models. The algorithm then classifies
the input data using a sequentially overfitted CNN model. The model classifies classes
sequentially and finally classifies classes that have not been trained. This is a
way to solve the problem of supervised learning method. The virtual data generated
using the GAN in this algorithm is used to further train the overfitted model of the
fault condition.
It is easy to build a CNN model that is overfitted in steady state in DT-CNN. However,
fault state data is so scarce and limited that additional data is required. Therefore,
the proposed model trains a model that is overfitted to the fault condition by using
the generated virtual data. Unused data is also very valuable in supervised learning
methods that require labeling. Thus, classified untrained data at the end of the DT-CNN
is collected. These data are then used to create virtual data using the GAN. The overfitted
model generated with initially insufficient data is supplemented by generated virtual
data. Fig. 4 shows the decision tree for fault diagnosis using DT-CNN. Fig. 5 shows a flow chart of the fault diagnosis supplementation using GAN.
The generated training data used in this case is only that with values of RRMSE
greater than 70 because the RRMSE value is 70 when the previously assumed minimum
value of the overfitted model accuracy is 80\%. This is not an absolute value and
is only validated in the experiments. This value should be verified by an experiment,
depending on the field of application. The method to verify this is as follows. At
first, the generated virtual data is used as the training data of the overfitted model.
At this point, it is better to use the kind of data where the model is overfitted
again within 5 epochs.
그림. 4. DT-CNN의 의사 결정 구조
Fig. 4. The structure of DT-CNN
그림. 5. 제안된 시스템의 블록 다이어그램
Fig. 5. A block diagram of the proposed system
3. Experiments
An experiment was carried out to train a model using the virtual data generated by
GAN. The performance of the model is evaluated by checking the classification accuracy
by inputting the existing training data and the generated training data into the trained
model. An experiment was also conducted to evaluate the model performance according
to the , which is the basis of the data to be used. The performance is assessed by
the number of iterations and the time it takes to overfit the trained model’s classification
to have accuracy above 99\%. Overfitting of the model is accomplished through the
hyperparameter optimi- zation technique using a genetic algorithm (12). In addition, the accuracy when inputting untrained data into additional trained
models is compared. This allows an appropriate level of data criteria to be evaluated.
The experimental environment used to measure the motor signal is the same as that
of the previous study (14). Fig. 6 shows the environment used in the experiment.The electric motor used in the experiment
is a 200W induction motor that is mainly used in industrial sites. The fault types
that make up the DT-CNN in Fig. 4 are gear faults, bearing faults, and a poorly fixed motor. The signal used in the
experiment is the X-axis vibration data of the motor. In this case, the data is transformed
using the FFT. The reason for using the FFT data is that the verification time is
shorter, and it is more obvious than the raw signal.
그림. 6. 실험 환경
Fig. 6. Experiments environment
In the first experiment, the generated virtual data confirms whether the overfitting
model can be further trained. The reference value of RRMSE was set to 70. When training
the model, the used data are 5,000 normal states and 50 fault states. A trained model
was created with insufficient fault data. In this case, the validation data of 5,000
normal and fault state data were not used for training. The test data were used for
5,000 normal-state and fault-condition data that were not used for learning. Unless
additional data were generated using the GAN, the model trained on 50 fault data could
not classify 5,000 fault data. Fig. 7 shows the generated virtual data for each fault condition when RRMSE is 70.
그림. 7. 각 고장 상태의 원 데이터와 생성된 데이터
Fig. 7. Real and generated data at each fault state
The second experiment was carried out to confirm the importance of selecting the RRMSE
value. In this experiment, the RRMSE values were set as 0, 60, 70, and 80 to confirm
the accuracy and time required for classifying the untrained data when the model is
further trained. The fault condition is a gear fault condition. The algorithm’s training
is repeated by adjusting the hyperparameters until the model’s accuracy is greater
than 99\%. The validation data is the same as that of Experiment 1. The results of
Experiments 1 and 2 are shown in Table 2.
표 2. 정확도 99\% 이상에서의 모델 성능 비교
Table 2. Model performance comparison with more than 99\% accuracy
|
Experiment 1
|
Fault state
|
Accuracy
|
|
Gear fault
|
99.67 %
|
|
Bearing fault
|
99.83 %
|
|
Poorly fixed
|
99.49 %
|
|
Experiment 2
|
RRMSE
|
Accuracy
|
Training Time
|
Number of Generation
|
|
80
|
99.87%
|
4795 [s]
|
1
|
|
70
|
99.88%
|
4618 [s]
|
1
|
|
60
|
99.78%
|
5213 [s]
|
1
|
|
0
|
99.80%
|
20094 [s]
|
3
|
그림. 8. t-SNE로 확인한 데이터
Fig. 8. Described data through t-SNE
From Experiment 2 in Table 2, it can be seen that if the RRMSE value is not considered, it has a negative effect
on the learning time. This is a problem that occurs as data sets with large variance
are added. It can be seen from Table 2 that the generated virtual data using the GAN can complement the overfitted model.
The results of Experiment 1 show that the model can obtain much failure data through
the virtual data generated by the GAN. The results of Experiment 2 show that the value
can be used as an indicator to evaluate the perfor- mance of the generated virtual
data. If this value is too small, the number of generations of genetic algorithms
required for hyperparameter optimization increases. This indicates that the added
data are very different from the existing data. In contrast, the larger this value
is, the fewer the generations are needed because additional data like existing data
are trained and very similar data are further trained. However, setting the value
too large may not increase the variance of data. When the value was 60, 70, and 80,
there was no big difference.
Fig. 8 shows the data used in the experiment in Table 2 when checked using the t-SNE algorithm.
The RRMSE value is calculated from the average of the data in one epoch. As can be
seen in Fig. 8, it can be seen that for each data there is data very similar to the original data.
In particular, the larger the value of RRMSE, the greater the degree of similarity
to the original data. This means that the larger the value of RRMSE, the more similar
data to the original signal. This point also appears in the experimental results in
Table 2. It can be seen that the higher the value of RRMSE, the faster the convergence speed
of transfer learning, like the data distribution confirmed by t-SNE. However, data
that are different from the original data will have an effect on learning. The results
of Experiment 2 did not fully explain the effect of these data on model performance.
Additional experiments are conducted to confirm the increase in outlier data classification
performance by varying the RRMSE value. In these experiments, data measured by two
sensors with different performance were used. One of these sensors has an isolator.
Fig. 9 shows the signal due to the sensor’s differences.
그림. 9. 센서 차이에 따른 데이터 변화
Fig. 9. Signal change due to sensor difference
As can be seen in Fig. 9, the two signals' overall patterns are very similar but show a large difference in
the driving frequency band of the motor. Previous CNN models could not classify data
measured by different sensors when trained with one sensor’s data. In this experiment,
the generated virtual data is further trained to see if it is possible to classify
the data measured by other sensors and check the effect of changing the RRMSE value
on learning data performance. The high accuracy of the trained model means that the
variance of the training data is increased because the data learned by the model are
similar to the existing data but different.
The training data used in the experiment was 850 normal and faulty data measured by
a sensor with an isolator. The data used for learning is labeled as sensor 1, and
data not used for learning is labeled as sensor 2. The initially trained model is
further trained using the virtual data of sensor 1 generated using the GAN algorithm.
Further learning is repeated until the accuracy of classifying the GAN data and sensor
1 data, which is the validation data, is greater than 99\%. The test data is 850 normal
and fault data measured on a sensor without an isolator attached. In addition, the
performance of the same model trained without using virtual data was reviewed to confirm
the increase in performance when the training data was created with GAN. In addition,
for comparison with existing studies, only the data of the point where the loss function
has the minimum value was added to proceed with learning. Table 3 shows the experimental results.
표 3. TP, FP로 확인한 이상치 검출 성능 비교
Table 3. Outlier detection performance comparison identified by TP and FP
|
Model
|
RRMSE
|
SENSOR 1
|
SENSOR 2
|
|
Normal
(TP)
|
Fault
(FP)
|
Normal
(TP)
|
Fault
(FP)
|
|
1
|
80
|
99.8
|
99.9
|
95.4
|
96.3
|
|
2
|
70
|
99.7
|
99.9
|
99.9
|
98.8
|
|
3
|
60
|
99.8
|
99.8
|
96.4
|
98.5
|
|
4
|
0
|
99.8
|
99.2
|
100
|
0
|
|
CNN
|
X
|
99.1
|
99.4
|
65.3
|
40.7
|
|
Previous
|
X
|
99.8
|
99.9
|
97.2
|
95.4
|
The results in Table 3 are TP (True positives) and FP (False positives) of the confusion matrix. The model
CNN is a model without additional training with virtual data generated by the GAN
algorithm. Model Previous is a model that is trained using only virtual data at the
minimum value of the loss function in the same way as previous studies. From Table 3, it can be seen that the virtual data generated through the GAN helps to improve
the outlier detection performance. In addition, it can be seen that the outlier detection
performance of the model is not significantly increased when only data similar to
the existing one is used as in the existing method. Table 3 indicates that if the value is too low (e.g., 0), the model cannot classify data
from sensor 2. This is a typical overfit situation. This problem is caused by learning
even patterns that are not very similar to actual data. However, the highest value
does not mean the highest accuracy of classification of sensor 2 faults because the
variance of data is not greatly increased. Finally, it is considered desirable to
select an value that divides the number of data in half within the same generation
number of the genetic algorithm.
4. Conclusions
In this paper, a study was conducted on a data evaluation technique to maximize the
performance of the model when using the GAN algorithm. Models trained with insufficient
training data through the proposed method were trained with data with greater variance.
This is an important factor in classi- fying a motor signal that changes little by
little according to noise or driving environment.
The results of Experiment 1 showed that a model trained with insufficient training
data can classify a large number of failure data by additionally learning the additionally
generated data through the GAN algorithm. In addition, the criteria for using the
virtual data generated through Experiments 2 and 3 as training data were described.
It was confirmed that the outlier detection performance of the overall model was increased
by using the GAN algorithm. In addition, the data evaluation technique using RMSE
showed up to 3\% improvement in outlier data detection performance. It is expected
that the proposed method through the results of the experiments conducted will be
a good way to solve the insufficient data in the diagnosis of motor failure using
deep learning.
Acknowledgements
This research was supported by Korea Electric Power Corporation. [Grant number : R18XA06-23].
References
B. Li, M.-Y. Chow, Y. Tipsuwan, J.C. Hung, 2000, Neural- network-based motor rolling
bearing fault diagnosis, IEEE Transactions on Industrial Electronics, Vol. 47, pp.
1060-1069
W. Sun, R. Zhao, R. Q. Yan, Jun 2017, Convolutional discrimi- native feature learning
for induction motor fault diagnosis, IEEE Trans. Ind. Informat., Vol. 13, No. 3, pp.
1350-1359
K. Kim, A. G. Parlos, Jun 2002, Induction motor fault diagnosis based on neuropredictors
and wavelet signal processing, IEEE/ASME Trans. Mechatronics, Vol. 7, No. 2, pp. 201-219
D. J. Choi, J. H. Han, S. U. Park, S. K. Hong, Oct 2019, Data Preprocessing Method
in Motor Fault Diagnosis Using Unsupervised Learning, ICCAS 2019 Conference paper,
pp. 1508-1511
J. H. Han, D. J. Choi, S. K. Hong, March 2019, Study on Develop- ment of Deep Learning
Fault Diagnosis Algorithm Consi- dering Induction Motor Speed and Load Condition,
The Transactions of The Korean Institute of Electrical Engineers, Vol. 68, No. 3
Q. Guo, Y. Li, Y. Song, D. Wang, W. Chen, 2019, Intelligent Fault Diagnosis Method
Based on Full 1D Convolutional Generative Adversarial Network, IEEE Trans. Ind Inform
Y. Xie, T. Zhang, 2018, A Transfer Learning Strategy for Rotation Machinery Fault
Diagnosis based on Cycle Con- sistent Generative Adversarial Networks, IEEE, pp. 1309-1313
N. Sriraam, C. Eswaran, 2008, Performance evaluation of neural network and linear
predictors for near-lossless compression of EEG signals, IEEE Transactions on Information
Tech- nology in Biomedicine, pp. 87-93
T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, X. Chen, 2016, Improved
techniques for training gans, arXiv preprint arXiv:1606.03498
A. Radford, L. Metz, S. Chintala, 2015, Unsupervised repre- sentation learning with
deep convolutional generative adver- sarial networks, arXiv preprint arXiv:1511.06434
J. Goodfellow Ian, Pouget-Abadie Jean, Mirza Mehdi, Xu Bing, WardeFarley David, Ozair
Sherjil, Courville Aaron, Bengio Yoshua, 2014, Generative adversarial nets., In Advances
in Neural Information Processing Systems, Curran Associates, Inc, Vol. 27, pp. 2672-2680
N. Feki, G. Clerc, P. Velex, Feb 2013, Gear and motor fault modeling and detection
based on motor current analysis, Elect. Power Syst. Res., Vol. 95, No. , pp. 28-37
J. H. Han, D. J. Choi, S. U. Park, S. K. Hong, 2020, Hyperparameter Optimization Using
a Genetic Algorithm Considering Verification Time in a Convolutional Neural Network,
Journal of Electrical Engineering & Technology, Vol. 15, pp. 721-726
J. H. Han, D. J. Choi, S. U. Park, S. K. Hong, Oct 2019, A Study on Motor Poor Maintenance
Detection Based on DT-CNN, ICCAS 2019 Conference paper, pp. 1234-1237
저자소개
He obtained his B.S. in Digital Control Engi- neering from Hoseo University, Korea
in 2019. Currently, he is pursuing the M.S. in Infor- mation Control Engineering from
Hoseo Univer- sity, Korea. His research interests include deep learning and motor
control.
He obtained his B.S. in Digital Control Engi- neering from Hoseo University, Korea
in 2019. Currently, he is pursuing the M.S. in Infor- mation Control Engineering from
Hoseo Univer- sity, Korea. His research interests include deep learning and IoT system.
He obtained his B.S. in Digital Control Engi- neering from Hoseo University, Korea
in 2020. Currently, he is pursuing the M.S. in Infor- mation Control Engineering from
Hoseo Univer- sity, Korea. His research interests include EEG signal processing and
IoT system control.
He received the B.S., M.S. and Ph.D degrees in Electric Engineering from Seoul National
University, Korea in 1987, 1989 and 1993, respectively. He joined Hoseo University,
Korea, in 1995, where he is currently a Full Professor with the Department of Digital
Engineering. His research interests include hysteresis motor analysis, electric motor
analysis and design, motor fault diagnosis, servo motor control, converter and inverter
design, deep learning and IoT.