SCALABLE INCREMENTAL DATA MINING: A REAL-TIME FRAMEWORK FOR EFFICIENT BIG DATA PROCESSING AND ANALYSIS
##plugins.themes.bootstrap3.article.main##
Abstract
Scalable Incremental Data Mining (SIDM) is a methodology designed to efficiently handle dynamic and large datasets by enabling continuous updates to the model without retraining from scratch. SIDM is particularly useful for real-time applications, such as fraud detection in financial transactions, where data streams grow exponentially. This approach incorporates incremental updates, real-time processing, and resource management strategies, including model pruning and batch size control, ensuring scalability and computational efficiency. SIDM offers significant advantages over traditional batch processing, maintaining accuracy and performance in big data environments.
##plugins.themes.bootstrap3.article.details##
Section
Articles
References
[1]. T. Matsumoto, W. Sunayama, Y. Hatanaka and K. Ogohara, "Data Analysis Support by Combining Data Mining and Text Mining," 2017 6th IIAI International Congress on Advanced Applied Informatics (IIAI-AAI), Hamamatsu, Japan, 2017, pp. 313-318, doi: 10.1109/IIAI-AAI.2017.165.
[2]. V. Putrenko, N. Pashvnska and S. Nazarenko, "Data Mining of Network Events with Space-Time Cube Application," 2018 IEEE Second International Conference on Data Stream Mining & Processing (DSMP), Lviv, Ukraine, 2018, pp. 79-83, doi: 10.1109/DSMP.2018.8478437.
[3]. M. Y. Raval, S. Yagnik and S. R. Dave, "An Effective High Utility Itemset Mining Algorithm with Big Data Based on MapReduce Framework," 2018 International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 2018, pp. 590-595, doi: 10.1109/ICIRCA. 2018.8597176.
[4]. P. T. T. Khine and H. P. P. Win, "Ensemble Framework for Big Data Stream Mining," 2020 IEEE Conference on Computer Applications(ICCA), Yangon, Myanmar, 2020, pp. 1-5, doi: 10.1109/ICCA49400.2020.9022820.
[5]. F. MartÃnez-Plumed et al., "CRISP-DM Twenty Years Later: From Data Mining Processes to Data Science Trajectories," in IEEE Transactions on Knowledge and Data Engineering, vol. 33, no. 8, pp. 3048-3061, 1 Aug. 2021, doi: 10.1109/TKDE.2019.2962680.
[6]. H. Xu, M. Fang, L. Li, Y. Tian and Y. Li, "The value of data mining for surveillance video in big data era," 2017 IEEE 2nd International Conference on Big Data Analysis (ICBDA), Beijing, China, 2017, pp. 202-206, doi: 10.1109/ICBDA.2017.8078808.
[7]. S. Roy and S. N. Singh, "Emerging trends in applications of big data in educational data mining and learning analytics," 2017 7th International Conference on Cloud Computing, Data Science & Engineering - Confluence, Noida, India, 2017, pp. 193-198, doi: 10.1109/ CONFLUENCE. 2017.7943148.
[8]. N. Ekwunife, "National Security Intelligence through Social Network Data Mining," 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 2020, pp. 2270-2273, doi: 10.1109/ Big Data 50022.2020. 9377940.
[9]. M. -H. Nadimi-Shahraki and M. Mansouri, "Hp-Apriori: Horizontal parallel-apriori algorithm for frequent itemset mining from big data," 2017 IEEE 2nd International Conference on Big Data Analysis (ICBDA), Beijing, China, 2017, pp. 286-290, doi: 10.1109/ICBDA.2017.8078825.
[10]. R. K. Shukla, P. Sharma, N. Samaiya and M. Kherajani, "WEB USAGE MINING-A Study of Web data pattern detecting methodologies and its applications in Data Mining," 2nd International Conference on Data, Engineering and Applications (IDEA), Bhopal, India, 2020, pp. 1-6, doi: 10.1109/IDEA49133.2020.9170690.
[11]. Z. Dong, "Research of Big Data Information Mining and Analysis : Technology Based on Hadoop Technology," 2022 International Conference on Big Data, Information and Computer Network (BDICN), Sanya, China, 2022, pp. 173-176, doi: 10.1109/BDICN55575.2022.00041.
[12]. L. Jijuan, H. Lirong, X. Miaohong, Z. Chi and L. Dewei, "The Application of Data Mining and Fusion Technology in the Security Management of Sensitive Data in Data Center," 2021 IEEE International Conference on Data Science and Computer Application (ICDSCA), Dalian, China, 2021, pp. 464-467, doi: 10.1109/ICDSCA53499. 2021.9650201.
[13]. S. M. Dol and P. M. Jawandhiya, "Use of Data mining Tools in Educational Data Mining," 2022 Fifth International Conference on Computational Intelligence and Communication Technologies (CCICT), Sonepat, India, 2022, pp. 380-387, doi: 10.1109/CCiCT56684.2022.00075.
[14]. J. Luo, "Modeling of Data Mining Technology in Financial Data Recognition Mining and Forecasting," 2022 4th International Conference on Smart Systems and Inventive Technology (ICSSIT), Tirunelveli, India, 2022, pp. 1168-1171, doi: 10.1109/ICSSIT53264.2022.9716308.
[15]. J. Shen, "Research on Parallel Data Mining Based on Spark," 2022 International Symposium on Advances in Informatics, Electronics and Education (ISAIEE), Frankfurt, Germany, 2022, pp. 122-126, doi: 10.1109/ ISAIEE 57420. 2022.00033.
[2]. V. Putrenko, N. Pashvnska and S. Nazarenko, "Data Mining of Network Events with Space-Time Cube Application," 2018 IEEE Second International Conference on Data Stream Mining & Processing (DSMP), Lviv, Ukraine, 2018, pp. 79-83, doi: 10.1109/DSMP.2018.8478437.
[3]. M. Y. Raval, S. Yagnik and S. R. Dave, "An Effective High Utility Itemset Mining Algorithm with Big Data Based on MapReduce Framework," 2018 International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 2018, pp. 590-595, doi: 10.1109/ICIRCA. 2018.8597176.
[4]. P. T. T. Khine and H. P. P. Win, "Ensemble Framework for Big Data Stream Mining," 2020 IEEE Conference on Computer Applications(ICCA), Yangon, Myanmar, 2020, pp. 1-5, doi: 10.1109/ICCA49400.2020.9022820.
[5]. F. MartÃnez-Plumed et al., "CRISP-DM Twenty Years Later: From Data Mining Processes to Data Science Trajectories," in IEEE Transactions on Knowledge and Data Engineering, vol. 33, no. 8, pp. 3048-3061, 1 Aug. 2021, doi: 10.1109/TKDE.2019.2962680.
[6]. H. Xu, M. Fang, L. Li, Y. Tian and Y. Li, "The value of data mining for surveillance video in big data era," 2017 IEEE 2nd International Conference on Big Data Analysis (ICBDA), Beijing, China, 2017, pp. 202-206, doi: 10.1109/ICBDA.2017.8078808.
[7]. S. Roy and S. N. Singh, "Emerging trends in applications of big data in educational data mining and learning analytics," 2017 7th International Conference on Cloud Computing, Data Science & Engineering - Confluence, Noida, India, 2017, pp. 193-198, doi: 10.1109/ CONFLUENCE. 2017.7943148.
[8]. N. Ekwunife, "National Security Intelligence through Social Network Data Mining," 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA, 2020, pp. 2270-2273, doi: 10.1109/ Big Data 50022.2020. 9377940.
[9]. M. -H. Nadimi-Shahraki and M. Mansouri, "Hp-Apriori: Horizontal parallel-apriori algorithm for frequent itemset mining from big data," 2017 IEEE 2nd International Conference on Big Data Analysis (ICBDA), Beijing, China, 2017, pp. 286-290, doi: 10.1109/ICBDA.2017.8078825.
[10]. R. K. Shukla, P. Sharma, N. Samaiya and M. Kherajani, "WEB USAGE MINING-A Study of Web data pattern detecting methodologies and its applications in Data Mining," 2nd International Conference on Data, Engineering and Applications (IDEA), Bhopal, India, 2020, pp. 1-6, doi: 10.1109/IDEA49133.2020.9170690.
[11]. Z. Dong, "Research of Big Data Information Mining and Analysis : Technology Based on Hadoop Technology," 2022 International Conference on Big Data, Information and Computer Network (BDICN), Sanya, China, 2022, pp. 173-176, doi: 10.1109/BDICN55575.2022.00041.
[12]. L. Jijuan, H. Lirong, X. Miaohong, Z. Chi and L. Dewei, "The Application of Data Mining and Fusion Technology in the Security Management of Sensitive Data in Data Center," 2021 IEEE International Conference on Data Science and Computer Application (ICDSCA), Dalian, China, 2021, pp. 464-467, doi: 10.1109/ICDSCA53499. 2021.9650201.
[13]. S. M. Dol and P. M. Jawandhiya, "Use of Data mining Tools in Educational Data Mining," 2022 Fifth International Conference on Computational Intelligence and Communication Technologies (CCICT), Sonepat, India, 2022, pp. 380-387, doi: 10.1109/CCiCT56684.2022.00075.
[14]. J. Luo, "Modeling of Data Mining Technology in Financial Data Recognition Mining and Forecasting," 2022 4th International Conference on Smart Systems and Inventive Technology (ICSSIT), Tirunelveli, India, 2022, pp. 1168-1171, doi: 10.1109/ICSSIT53264.2022.9716308.
[15]. J. Shen, "Research on Parallel Data Mining Based on Spark," 2022 International Symposium on Advances in Informatics, Electronics and Education (ISAIEE), Frankfurt, Germany, 2022, pp. 122-126, doi: 10.1109/ ISAIEE 57420. 2022.00033.