Research Article
BibTex RIS Cite

A novel hybrid ICA-SVM method for detection and identification of shift in multivariate processes

Year 2024, Volume: 53 Issue: 2, 556 - 576, 23.04.2024

Zümre Özdemir Güler M. Akif Bakır Filiz Kardiyen

https://doi.org/10.15672/hujms.1269072

Abstract

Detecting shifts in the mean vector of a multivariate statistical process control is crucial, and equally important is identifying the source of such a signal. This study introduces a novel approach that combines independent components analysis with support vector machines to address the challenge of multivariate process monitoring. In this hybrid independent components analysis-support vector machines method, statistical metrics $I^2$ derived from the independent components extracted through independent components analysis from observed data serve as input variables for the support vector machines. The probabilistic outputs generated by the support vector machines model are utilized as monitoring statistics for the proposed control chart, referred to as $I^2-\text{PoC}$. Simulation results validate the effectiveness of the independent components analysis with support vector machines approach in both detecting and identifying shifts in multivariate control processes, whether they follow a normal or non-normal distribution. Furthermore, the results demonstrate the robustness of this method in handling various challenges, including complex relationships between process variables, shifts of varying sizes, and different distribution shapes, when compared to existing approaches in the literature.

Keywords

multivariate statistical process control shift detection shift identification independent component analysis support vector machines

References

  • [1] F.B. Alt, Multivariate Quality Control, Encyclopedia of Statistical Sciences,Vol.6, N.L. Johnson and S. Kotz, (eds.) Wiley, New York, 1985.
  • [2] L.K. Chan and G.Y. Li, A multivariate control chart for detecting linear trends, Comm. Statist. Simulation Comput. 23, 997-1012, 1994.
  • [3] Y.S. Chang and D.S. Bai, A multivariate T2 control chart for skewed populations using weighted standard deviations,Qual. Reliab. Eng. Int. 20, 3146, 2004.
  • [4] J.M. Charnes, Tests for special causes with multivariate autocorrelated data, Comput. Oper. Res. 22, 443-453, 1995.
  • [5] M.P.S. Chawla, PCA and ICA processing methods for removal of artifacts and noise in electrocardiograms: A survey and comparison, Appl. Soft Comput. 11 (2), 2216- 2226, 2011.
  • [6] L.H. Chiang, E.L. Russell and R.D. Braatz, Fault Detection and Diagnosis in Industrial Systems, London, U.K., Springer, 2000.
  • [7] S.W. Choi, C. Lee, J.M. Lee, J.H. Park and I.B. Lee, Fault detection and identification of nonlinear processes based on kernel PCA, Chemometrics Intell. Lab. Syst. 75 (1), 55-67, 2005.
  • [8] P. Chongfuangprinya, S.B. Kim, S.K. Park and T. Sukchotrat, Integration of support vector machines and control charts for multivariate process monitoring, J. Stat. Comput. Simul. 81 (9), 1157-1173, 2011.
  • [9] R.B. Crosier, Multivariate generalizations of cumulative sum quality-control schemes, Technometrics 30 (3), 291-303, 1988.
  • [10] S.X. Ding, Data-driven design of monitoring and diagnosis systems for dynamic processes: A review of subspace technique based schemes and some recent results, J. Process Control 24 (2), 431-449, 2014.
  • [11] Z. Ge, Z. Song and F. Gao, Review of recent research on data-based process monitoring, Ind. Eng. Chem. Res. 52 (10), 3543-3562, 2013.
  • [12] M. Girolami, Self-Organising Neural Networks, London, U.K., Springer, 4775, 1999.
  • [13] Z.O. Guler and M.A. Bakir, Detection and identification of mean shift using independent component analysis in multivariate processes, J. Stat. Comput. Simul. 92 (9), 1920-1940, 2022.
  • [14] T. Hastie and R. Tibshirani, Classification by pairwise coupling, Ann. Stat. 26 (2), 451-471, 1998.
  • [15] D.M. Hawkins, Regression adjustment for variables in multivariate quality control, J. Qual. Technol. 25, 170-182, 1993.
  • [16] A.J. Hayter and K.L. Tsui, Identification and quantification in multivariate quality control problems, J. Qual. Technol. 26 (3), 197,208, 1994.
  • [17] H. Hotelling, Multivariate Quality Control, Techniques of Statistical Analysis, C. Eisenhart, M. W. Hastay, ve W. A. Wallis, eds., New York: McGraw-Hill, 1947.
  • [18] C.C. Hsu, M.C. Chen and L.S. Chen, Integrating independent component analysis and support vector machine for multivariate process monitoring, Comput. Ind. Eng. 59 (1), 146-156, 2010.
  • [19] C.C. Hsu, L.S. Chen and C.H. Liu, A process monitoring scheme based on independent component analysis and adjusted outliers, Int. J. Prod. Res. 48 (6), 1727-1743, 2010.
  • [20] A. Hyvärinen and E. Oja, Independent component analysis: algorithms and applications, Neural Networks 13, 411-430, 2000.
  • [21] Q. Jiang, X. Yan and B. Huang, Performance-driven distributed PCA process monitoring based on fault-relevant variable selection and Bayesian inference, IEEE Trans. Ind. Electron. 63 (1), 377-386, 2016.
  • [22] C. Jutten and J. Herault, Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture, Signal Processing 24 (1), 1991.
  • [23] M. Kano, S. Tanaka, S. Hasebe, I. Hashimoto and H. Ohno, Monitoring independent components for fault detection, Am. J. Chem. Eng. 49 (4), 969-976, 2003.
  • [24] V. Kecman, Learning and Soft Computing: Support Vector Machines, Neural Networks, and Fuzzy Logic Models, Massachusetts Institute of Technology Press, Cambridge, Massachusetts, 2001.
  • [25] J.V. Kresta, J.F. MacGregor and T.E. Marlin, Multivariate statistical monitoring of process operating performance, Can. J. Chem. Eng. 69 (1), 35-47, 1991.
  • [26] J.M. Lee, K.Y. Chang and I.B. Lee, Statistical monitoring of dynamic processes based on dynamic independent component analysis, Chem. Eng. Sci. 59 (14), 2995-3006, 2004.
  • [27] J.M. Lee, S.J. Qin and I.B. Lee, Fault detection and diagnosis based on modified independent component analysis, Am. J. Chem. Eng. 52 (10), 3501-3514, 2006.
  • [28] J.M. Lee, C.K. Yoo and I.B. Lee, Fault detection of batch processes using multiway kernel principal component analysis, Comput. Chem. Eng. 28 (9), 1837-1847, 2004.
  • [29] J.M. Lee, C.K. Yoo and I.B. Lee, Statistical process monitoring with independent component analysis, J. Process Control 14, 467-485, 2004.
  • [30] R.Y. Liu, Control charts for multivariate processes, J. Am. Stat. Assoc. 90, 1380-1387, 1995.
  • [31] C.A. Lowry and D.C. Montgomery, A review of multivariate control charts, IIE Transactions 27 (6), 800-810, 1995.
  • [32] C.A. Lowry, W.H. Woodall, C.W. Champ and S.E. Rigdon, A multivariate exponentially weighted moving average control chart, Technometrics 34 (1), 46-53, 1992.
  • [33] C.J. Lu, Integrating independent component analysis-based denoising scheme with neural network for stock price prediction, Expert Syst Appl 37 (10), 7056-7064, 2010.
  • [34] C.J. Lu, T.S. Lee and C.C. Chiu, Financial time series forecasting using independent component analysis and support vector regression, Decis. Support Syst. 47 (2), 115- 125, 2009.
  • [35] C.J. Lu and D.M. Tsai, Independent component analysis-based defect detection in patterned liquid crystal display surfaces,Image Vis 26 (7), 955-970, 2008.
  • [36] C.J. Lu, C.M.Wu, C.J. Keng and C.C. Chiu, Integrated application of SPC/EPC/ICA and neural networks, Int. J. Prod. Res. 46 (4), 873-893, 2006.
  • [37] E. Martin, A. Morris and J. Zhang, Process performance monitoring using multivariate statistical process control in Control Theory and Applications, IEE Proceedings, 132-144, 1996.
  • [38] R.L. Mason, C.W. Champ, N.D. Tracy, R.J. Wierda and J.C. Young, Assessment of multivariate process control techniques, J. Qual. Technol. 29, 140-143, 1997.
  • [39] R.L. Mason, N.D. Tracy and J.C. Young, Decomposition of T2 for multivariate control chart interpretation, J. Qual. Technol. 27 (2), 109-119, 1995.
  • [40] R.L. Mason and J.C. Young, Multivariate Statistical Process Control With Industrial Application, Philadelphia: SIAM, 2002.
  • [41] D.A.O. Moraes, F.L.P. Oliveira and L.H. Duczmal, On the Hotellings T, MCUSUM and MEWMA control charts performance with different variability sources: A simulation study,Braz. J. Oper. Prod. Manag. 12, 196-212, 2015.
  • [42] J. Platt, Probabilistic outputs for support vector machines and comparison to regularized likelihood methods, Advances in Large Margin Classifiers, A. Smola, P. Bartlett, B. Scholkopf, and D. Schuurmans, eds., Cambridge, MA, 6174, 2000.
  • [43] E.Y. Shao, C.J. Lu and Y.C. Wang, A Hybrid ICA-SVM approach for determining the quality variables at fault in a multivariate process, Math. Probl. Eng. 18 (1), 1-12, 2012.
  • [44] J. Shawe-Taylor and N. Cristianini, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods, Cambridge University Press, New York, 2000.
  • [45] J.H. Sullivan, Z.G. Stoumbos, R.L. Mason and J.C. Young, Step-Down analysis for changes in the covariance matrix and other parameters, J. Qual. Technol. 39, 66-84, 2007.
  • [46] R. Sun and F. Tsung, A Kernel-distance-based multivariate control chart using support vector methods, Int. J. Prod. Res. 41 (13), 2975-2989, 2003.
  • [47] V.N. Vapnik, The Nature of Statistical Learning Theory, New York: Springer-Verlag, 1995.
  • [48] V.N. Vapnik and A.J. Chervonenkis, On the uniform convergence of relative frequencies of events to their probabilities, Academy of Sciences of the United Soviet Socialist Republic 4, 181, 1968.
  • [49] W.H. Woodall, D.J. Spitzner, D.C. Montgomery and S. Gupta, Using control charts to monitor process and product quality profiles, J. Qual. Technol. 36, 309-320, 2004.
  • [50] C.K. Yoo, J.M. Lee, P.A. Vanrolleghem and I.B. Lee, On-line monitoring of batch processes using multiway independent component analysis, Chemom. Intell. Lab. Syst. 71 (2), 151-163, 2004.
  • [51] Y. Zhang and Y. Zhang, Fault detection of non-Gaussian processes based on modified independent component analysis, Chem. Eng. Sci. 65 (16), 4630-4639, 2010.

Year 2024, Volume: 53 Issue: 2, 556 - 576, 23.04.2024

Zümre Özdemir Güler M. Akif Bakır Filiz Kardiyen

https://doi.org/10.15672/hujms.1269072

Abstract

References

  • [1] F.B. Alt, Multivariate Quality Control, Encyclopedia of Statistical Sciences,Vol.6, N.L. Johnson and S. Kotz, (eds.) Wiley, New York, 1985.
  • [2] L.K. Chan and G.Y. Li, A multivariate control chart for detecting linear trends, Comm. Statist. Simulation Comput. 23, 997-1012, 1994.
  • [3] Y.S. Chang and D.S. Bai, A multivariate T2 control chart for skewed populations using weighted standard deviations,Qual. Reliab. Eng. Int. 20, 3146, 2004.
  • [4] J.M. Charnes, Tests for special causes with multivariate autocorrelated data, Comput. Oper. Res. 22, 443-453, 1995.
  • [5] M.P.S. Chawla, PCA and ICA processing methods for removal of artifacts and noise in electrocardiograms: A survey and comparison, Appl. Soft Comput. 11 (2), 2216- 2226, 2011.
  • [6] L.H. Chiang, E.L. Russell and R.D. Braatz, Fault Detection and Diagnosis in Industrial Systems, London, U.K., Springer, 2000.
  • [7] S.W. Choi, C. Lee, J.M. Lee, J.H. Park and I.B. Lee, Fault detection and identification of nonlinear processes based on kernel PCA, Chemometrics Intell. Lab. Syst. 75 (1), 55-67, 2005.
  • [8] P. Chongfuangprinya, S.B. Kim, S.K. Park and T. Sukchotrat, Integration of support vector machines and control charts for multivariate process monitoring, J. Stat. Comput. Simul. 81 (9), 1157-1173, 2011.
  • [9] R.B. Crosier, Multivariate generalizations of cumulative sum quality-control schemes, Technometrics 30 (3), 291-303, 1988.
  • [10] S.X. Ding, Data-driven design of monitoring and diagnosis systems for dynamic processes: A review of subspace technique based schemes and some recent results, J. Process Control 24 (2), 431-449, 2014.
  • [11] Z. Ge, Z. Song and F. Gao, Review of recent research on data-based process monitoring, Ind. Eng. Chem. Res. 52 (10), 3543-3562, 2013.
  • [12] M. Girolami, Self-Organising Neural Networks, London, U.K., Springer, 4775, 1999.
  • [13] Z.O. Guler and M.A. Bakir, Detection and identification of mean shift using independent component analysis in multivariate processes, J. Stat. Comput. Simul. 92 (9), 1920-1940, 2022.
  • [14] T. Hastie and R. Tibshirani, Classification by pairwise coupling, Ann. Stat. 26 (2), 451-471, 1998.
  • [15] D.M. Hawkins, Regression adjustment for variables in multivariate quality control, J. Qual. Technol. 25, 170-182, 1993.
  • [16] A.J. Hayter and K.L. Tsui, Identification and quantification in multivariate quality control problems, J. Qual. Technol. 26 (3), 197,208, 1994.
  • [17] H. Hotelling, Multivariate Quality Control, Techniques of Statistical Analysis, C. Eisenhart, M. W. Hastay, ve W. A. Wallis, eds., New York: McGraw-Hill, 1947.
  • [18] C.C. Hsu, M.C. Chen and L.S. Chen, Integrating independent component analysis and support vector machine for multivariate process monitoring, Comput. Ind. Eng. 59 (1), 146-156, 2010.
  • [19] C.C. Hsu, L.S. Chen and C.H. Liu, A process monitoring scheme based on independent component analysis and adjusted outliers, Int. J. Prod. Res. 48 (6), 1727-1743, 2010.
  • [20] A. Hyvärinen and E. Oja, Independent component analysis: algorithms and applications, Neural Networks 13, 411-430, 2000.
  • [21] Q. Jiang, X. Yan and B. Huang, Performance-driven distributed PCA process monitoring based on fault-relevant variable selection and Bayesian inference, IEEE Trans. Ind. Electron. 63 (1), 377-386, 2016.
  • [22] C. Jutten and J. Herault, Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture, Signal Processing 24 (1), 1991.
  • [23] M. Kano, S. Tanaka, S. Hasebe, I. Hashimoto and H. Ohno, Monitoring independent components for fault detection, Am. J. Chem. Eng. 49 (4), 969-976, 2003.
  • [24] V. Kecman, Learning and Soft Computing: Support Vector Machines, Neural Networks, and Fuzzy Logic Models, Massachusetts Institute of Technology Press, Cambridge, Massachusetts, 2001.
  • [25] J.V. Kresta, J.F. MacGregor and T.E. Marlin, Multivariate statistical monitoring of process operating performance, Can. J. Chem. Eng. 69 (1), 35-47, 1991.
  • [26] J.M. Lee, K.Y. Chang and I.B. Lee, Statistical monitoring of dynamic processes based on dynamic independent component analysis, Chem. Eng. Sci. 59 (14), 2995-3006, 2004.
  • [27] J.M. Lee, S.J. Qin and I.B. Lee, Fault detection and diagnosis based on modified independent component analysis, Am. J. Chem. Eng. 52 (10), 3501-3514, 2006.
  • [28] J.M. Lee, C.K. Yoo and I.B. Lee, Fault detection of batch processes using multiway kernel principal component analysis, Comput. Chem. Eng. 28 (9), 1837-1847, 2004.
  • [29] J.M. Lee, C.K. Yoo and I.B. Lee, Statistical process monitoring with independent component analysis, J. Process Control 14, 467-485, 2004.
  • [30] R.Y. Liu, Control charts for multivariate processes, J. Am. Stat. Assoc. 90, 1380-1387, 1995.
  • [31] C.A. Lowry and D.C. Montgomery, A review of multivariate control charts, IIE Transactions 27 (6), 800-810, 1995.
  • [32] C.A. Lowry, W.H. Woodall, C.W. Champ and S.E. Rigdon, A multivariate exponentially weighted moving average control chart, Technometrics 34 (1), 46-53, 1992.
  • [33] C.J. Lu, Integrating independent component analysis-based denoising scheme with neural network for stock price prediction, Expert Syst Appl 37 (10), 7056-7064, 2010.
  • [34] C.J. Lu, T.S. Lee and C.C. Chiu, Financial time series forecasting using independent component analysis and support vector regression, Decis. Support Syst. 47 (2), 115- 125, 2009.
  • [35] C.J. Lu and D.M. Tsai, Independent component analysis-based defect detection in patterned liquid crystal display surfaces,Image Vis 26 (7), 955-970, 2008.
  • [36] C.J. Lu, C.M.Wu, C.J. Keng and C.C. Chiu, Integrated application of SPC/EPC/ICA and neural networks, Int. J. Prod. Res. 46 (4), 873-893, 2006.
  • [37] E. Martin, A. Morris and J. Zhang, Process performance monitoring using multivariate statistical process control in Control Theory and Applications, IEE Proceedings, 132-144, 1996.
  • [38] R.L. Mason, C.W. Champ, N.D. Tracy, R.J. Wierda and J.C. Young, Assessment of multivariate process control techniques, J. Qual. Technol. 29, 140-143, 1997.
  • [39] R.L. Mason, N.D. Tracy and J.C. Young, Decomposition of T2 for multivariate control chart interpretation, J. Qual. Technol. 27 (2), 109-119, 1995.
  • [40] R.L. Mason and J.C. Young, Multivariate Statistical Process Control With Industrial Application, Philadelphia: SIAM, 2002.
  • [41] D.A.O. Moraes, F.L.P. Oliveira and L.H. Duczmal, On the Hotellings T, MCUSUM and MEWMA control charts performance with different variability sources: A simulation study,Braz. J. Oper. Prod. Manag. 12, 196-212, 2015.
  • [42] J. Platt, Probabilistic outputs for support vector machines and comparison to regularized likelihood methods, Advances in Large Margin Classifiers, A. Smola, P. Bartlett, B. Scholkopf, and D. Schuurmans, eds., Cambridge, MA, 6174, 2000.
  • [43] E.Y. Shao, C.J. Lu and Y.C. Wang, A Hybrid ICA-SVM approach for determining the quality variables at fault in a multivariate process, Math. Probl. Eng. 18 (1), 1-12, 2012.
  • [44] J. Shawe-Taylor and N. Cristianini, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods, Cambridge University Press, New York, 2000.
  • [45] J.H. Sullivan, Z.G. Stoumbos, R.L. Mason and J.C. Young, Step-Down analysis for changes in the covariance matrix and other parameters, J. Qual. Technol. 39, 66-84, 2007.
  • [46] R. Sun and F. Tsung, A Kernel-distance-based multivariate control chart using support vector methods, Int. J. Prod. Res. 41 (13), 2975-2989, 2003.
  • [47] V.N. Vapnik, The Nature of Statistical Learning Theory, New York: Springer-Verlag, 1995.
  • [48] V.N. Vapnik and A.J. Chervonenkis, On the uniform convergence of relative frequencies of events to their probabilities, Academy of Sciences of the United Soviet Socialist Republic 4, 181, 1968.
  • [49] W.H. Woodall, D.J. Spitzner, D.C. Montgomery and S. Gupta, Using control charts to monitor process and product quality profiles, J. Qual. Technol. 36, 309-320, 2004.
  • [50] C.K. Yoo, J.M. Lee, P.A. Vanrolleghem and I.B. Lee, On-line monitoring of batch processes using multiway independent component analysis, Chemom. Intell. Lab. Syst. 71 (2), 151-163, 2004.
  • [51] Y. Zhang and Y. Zhang, Fault detection of non-Gaussian processes based on modified independent component analysis, Chem. Eng. Sci. 65 (16), 4630-4639, 2010.
There are 51 citations in total.

Details

Primary Language English
Subjects Statistics
Journal Section Statistics
Authors

Zümre Özdemir Güler 0000-0003-2730-4584

M. Akif Bakır 0000-0003-0774-0338

Filiz Kardiyen 0000-0002-8730-2751

Early Pub Date March 18, 2024
Publication Date April 23, 2024
Published in Issue Year 2024 Volume: 53 Issue: 2

Cite

APA Özdemir Güler, Z., Bakır, M. A., & Kardiyen, F. (2024). A novel hybrid ICA-SVM method for detection and identification of shift in multivariate processes. Hacettepe Journal of Mathematics and Statistics, 53(2), 556-576. https://doi.org/10.15672/hujms.1269072
AMA Özdemir Güler Z, Bakır MA, Kardiyen F. A novel hybrid ICA-SVM method for detection and identification of shift in multivariate processes. Hacettepe Journal of Mathematics and Statistics. April 2024;53(2):556-576. doi:10.15672/hujms.1269072
Chicago Özdemir Güler, Zümre, M. Akif Bakır, and Filiz Kardiyen. “A Novel Hybrid ICA-SVM Method for Detection and Identification of Shift in Multivariate Processes”. Hacettepe Journal of Mathematics and Statistics 53, no. 2 (April 2024): 556-76. https://doi.org/10.15672/hujms.1269072.
EndNote Özdemir Güler Z, Bakır MA, Kardiyen F (April 1, 2024) A novel hybrid ICA-SVM method for detection and identification of shift in multivariate processes. Hacettepe Journal of Mathematics and Statistics 53 2 556–576.
IEEE Z. Özdemir Güler, M. A. Bakır, and F. Kardiyen, “A novel hybrid ICA-SVM method for detection and identification of shift in multivariate processes”, Hacettepe Journal of Mathematics and Statistics, vol. 53, no. 2, pp. 556–576, 2024, doi: 10.15672/hujms.1269072.
ISNAD Özdemir Güler, Zümre et al. “A Novel Hybrid ICA-SVM Method for Detection and Identification of Shift in Multivariate Processes”. Hacettepe Journal of Mathematics and Statistics 53/2 (April 2024), 556-576. https://doi.org/10.15672/hujms.1269072.
JAMA Özdemir Güler Z, Bakır MA, Kardiyen F. A novel hybrid ICA-SVM method for detection and identification of shift in multivariate processes. Hacettepe Journal of Mathematics and Statistics. 2024;53:556–576.
MLA Özdemir Güler, Zümre et al. “A Novel Hybrid ICA-SVM Method for Detection and Identification of Shift in Multivariate Processes”. Hacettepe Journal of Mathematics and Statistics, vol. 53, no. 2, 2024, pp. 556-7, doi:10.15672/hujms.1269072.
Vancouver Özdemir Güler Z, Bakır MA, Kardiyen F. A novel hybrid ICA-SVM method for detection and identification of shift in multivariate processes. Hacettepe Journal of Mathematics and Statistics. 2024;53(2):556-7.

https://dergipark.org.tr/en/pub/hujms