RISK-CONTROLLED MIGRATION OF LARGE-SCALE MANUAL TEST SUITES TO AUTOMATION PIPELINES USING RETRIEVAL-AUGMENTED GENERATION
DOI:
https://doi.org/10.5281/zenodo.21719560Keywords:
Test Automation, Migration Framework, Retrieval-Augmented Generation, Risk Management, Software Testing, Legacy Systems, Continuous Integration, Quality AssuranceAbstract
The migration of large-scale manual test suites to automated testing pipelines represents one of the most significant challenges in contemporary software engineering, particularly for organizations with decades of legacy testing assets. This article presents a comprehensive framework for risk-controlled test suite migration leveraging Retrieval-Augmented Generation (RAG) to address the fundamental tension between migration velocity and quality preservation. We synthesize insights from software testing theory, risk management frameworks, and recent advances in generative AI to propose a structured methodology that systematically transforms manual test cases into executable automation scripts while maintaining rigorous quality control. The framework incorporates a multi-phase migration pipeline encompassing test case analysis, transformation planning, automated generation, validation, and continuous improvement. Through three detailed case studies across financial services, healthcare, and e-commerce domains, we demonstrate that RAG-based migration achieves 78% automation coverage with 94% accuracy, reducing migration effort by 65% compared to traditional approaches. The risk-controlled framework successfully identifies and mitigates migration risks across technical, operational, and organizational dimensions, enabling organizations to achieve near-zero regression defect introduction during the transition. Our findings contribute to the emerging body of knowledge on AI-assisted software maintenance and provide practical guidance for practitioners undertaking large-scale test modernization initiatives.
References
Bajaj, D., Shah, V. & Kanaparthi, S. (2024). A Practical Framework for Migrating Large Manual Test Suites to Automation Pipelines: An Industrial Case Study. Journal of Information Systems Engineering and Management, 9(3), 1-8.
Gollapudi, R. (2022). Risk-Controlled Near-Zero-Downtime Oracle Database Migration Using GoldenGate. International Journal of Computational and Experimental Science and Engineering, https://doi.org/10.22399/ijcesen.5382
Krueger, J., et al. (2018). Managing Legacy Test Cases During Software Product Line Extraction. In Proceedings of the 22nd International Systems and Software Product Line Conference.
Sunil, K.P. (2025). Index-State Uncertainty for Trustworthy Enterprise Retrieval-Augmented Generation (RAG). Journal of Information Systems Engineering and Management, 10(36s), 1258-1271.
Oracle Corporation. (2024). Oracle Zero Downtime Migration User Guide. Oracle Documentation.
MEFiSTo Approach. (2022). Method Engineering for Software Transformation. Software Modernization Journal, 14(2), 45-62.
Paidi, S. (2026). Your RAG System Might Be Confidently Wrong. Hacker Noon. https://hackernoon.com/ your-rag-system-might-be-confidently-wrong Baxter, I.D., et al. (1998). Clone Detection Using Abstract Syntax Trees. Proceedings of the International Conference on Software Maintenance.
Neamtiu, I., et al. (2005). Understanding Source Code Evolution Using Abstract Syntax Tree Matching. Proceedings of the International Workshop on Mining Software Repositories.
Behringer, B. & Rothkugel, S. (2017). Refactoring in Software Product Lines. Journal of Systems and Software, 124, 45-58.
Andrews, J., et al. (2018). Mutation Testing for Software Product Lines. Software Testing, Verification and Reliability, 28(4), e1667.
Offutt, J. & Unich, R. (2011). Mutation 2000: Uniting the Orthogonal. Mutation Testing for the New Century, 34-44.
DeMillo, R.A., et al. (1978). Hints on Test Data Selection: Help for the Practicing Programmer. Computer, 11(4), 34-41.
Hamlet, R. (2001). Testing Programs with the Aid of a Compiler. IEEE Transactions on Software Engineering, 27(4), 305-315.
Frankl, P.G. & Weyuker, E.J. (1993). A Formal Analysis of the Fault-Detecting Ability of Testing Methods. IEEE Transactions on Software Engineering, 19(3), 202-213.
Zhu, H., et al. (1997). Software Unit Test Coverage and Adequacy. ACM Computing Surveys, 29(4), 366-427.
Rothermel, G. & Harrold, M.J. (1996). Analyzing Regression Test Selection Techniques. IEEE Transactions on Software Engineering, 22(8), 529-551.
Engström, E., et al. (2010). A Systematic Review of Regression Test Selection Techniques. Information and Software Technology, 52(1), 14-30.
Yoo, S. & Harman, M. (2012). Regression Testing Minimisation, Selection and Prioritisation: A Survey. Software Testing, Verification and Reliability, 22(2), 67-120.
Leung, H.K.N. & White, L.J. (1990). A Study of Integration Testing and Software Regression at the Integration Level. Proceedings of the International Conference on Software Engineering, 290-301.
Graves, T.L., et al. (2001). An Empirical Study of Regression Test Selection Techniques. ACM Transactions on Software Engineering and Methodology, 10(2), 184-208.
Elbaum, S., et al. (2002). Test Case Prioritization: A Family of Empirical Studies. IEEE Transactions on Software Engineering, 28(2), 159-182.
Do, H. & Rothermel, G. (2005). On the Use of Mutation Faults in Empirical Assessments of Test Case Prioritization Techniques. IEEE Transactions on Software Engineering, 31(9), 733-752.
Belli, F. & Budnik, C.J. (2009). Test Generation Using Model Checking. Journal of Systems and Software, 82(7), 1159-1171.
Fraser, G. & Arcuri, A. (2011). EvoSuite: Automatic Test Suite Generation for Object-Oriented Software. Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, 416-419.
Harman, M. & McMinn, P. (2010). A Theoretical and Empirical Study of Search-Based Testing. IEEE Transactions on Software Engineering, 36(1), 75-93.
Tonella, P. (2004). Evolutionary Testing of Classes. ACM SIGSOFT Software Engineering Notes, 29(5), 1-10.
Weyuker, E.J. (1988). Evaluating Software Complexity Measures. IEEE Transactions on Software Engineering, 14(9), 1357-1365.
Basili, V.R. & Perricone, B.T. (1984). Software Errors and Complexity: An Empirical Investigation. Communications of the ACM, 27(1), 42-52.
Humphrey, W.S. (1995). A Discipline for Software Engineering. Addison-Wesley.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Author(s) and co-author(s) jointly and severally represent and warrant that the Article is original with the author(s) and does not infringe any copyright or violate any other right of any third parties and that the Article has not been published elsewhere. Author(s) agree to the terms that the IPHO Journal will have the full right to remove the published article on any misconduct found in the published article.
