Enhancing augmented reality experiences through advanced computer vision techniques

  • Chairuddin Chairuddin 
    Sekolah Tinggi Manajemen Informatika dan Komputer IM
    Indonesia
  • Ardi Azhar Nampira
    Institute Teknologi Sepuluh November
    Indonesia
  • Jarot Budiasto
    Information Systems Department, Faculty of Engineering, Universitas Musamus
    Indonesia

Abstract

This study investigates the integration of advanced computer vision techniques, specifically semantic segmentation, depth estimation, and object detection, into augmented reality (AR) systems to enhance user immersion, contextual interaction, and real-world adaptability. The research aims to address the limitations of existing AR systems, which often struggle with real-time performance and context-aware interaction in dynamic environments. Using a design-based research (DBR) approach, the study involved testing the proposed AR system on mobile devices under controlled and real-world conditions. Performance was assessed using specific metrics: frame rate (FPS), inference latency, semantic accuracy, and user experience. The sample included 20 participants, recruited through convenience sampling, with inclusion criteria focused on individuals familiar with mobile AR applications. Ethical approval was obtained for the study, and informed consent was provided by all participants. The testing involved both simulation trials and real-world prototyping in varied environmental conditions, such as indoor and outdoor settings with different lighting and motion dynamics. Results indicate that the integrated of the vision stack significantly enhanced scene understanding and enabling stable, context-aware digital overlays, with the system maintaining a real-time frame rate of >27 FPS. User feedback, measured through a 5-point Likert scale survey, confirmed improved immersion, visual coherence, and satisfaction compared to baseline AR systems, with an average increase of 35% in perceived realism. The analysis also revealed that the system's performance remained consistent across varying environmental conditions, with minimal latency (less than 300ms) in dynamic re-anchoring of AR elements. Statistical tests (paired t-tests) confirmed the significance of these improvements, with p-values < 0.05 for all key metrics. This research contributes a scalable framework that bridges artificial intelligence, user experience design, and mobile AR deployment. It provides empirical evidence supporting the integration of computer vision techniques into AR systems, with practical implications for applications in education, healthcare, and industry. Future work will focus on expanding the user base, exploring hardware compatibility, and investigating multimodal AR interactions.

Keywords
  • Augmented reality
  • Semantic segmentation
  • Depth estimation
  • Real-time performance
  • User immersion
Lisensing Information
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

References
  1. Adilzhan, A., & Omarov, N. (2024). Development of Augmented Reality Game Using Computer Vision Technology. 2024 IEEE 4th International Conference on Smart Information Systems and Technologies (SIST), 386–391.
  2. Alsadik, B., & Karam, S. (2021). The simultaneous localization and mapping (SLAM): An overview. Surveying and Geospatial Engineering Journal, 1(2), 1–12.
  3. Amparore, D., Sica, M., Verri, P., Piramide, F., Checcucci, E., De Cillis, S., Piana, A., Campobasso, D., Burgio, M., & Cisero, E. (2024). Computer vision and machine-learning techniques for automatic 3D virtual images overlapping during augmented reality guided robotic partial nephrectomy. Technology in Cancer Research & Treatment, 23, 15330338241229368.
  4. Appavu, N. (2025). Analysing the Effect of Edge-Optimized Deep Learning Models on Improving Low-Powered Iot Devices Real-Time Object Detection. 2025 9th International Conference on Inventive Systems and Control (ICISC), 1663–1669.
  5. Bahar, A. A., Fauzi, U. A., Al Anwar, S. F., Istama, N. R., Maulana, A. H., & Nargis, N. (2025). Youth slang as a reflection of social identity in teenage communities. Variable Research Journal, 2(01), 186–194.
  6. Bhowmik, A. K. (2024). Virtual and augmented reality: Human sensory‐perceptual requirements and trends for immersive spatial computing experiences. Journal of the Society for Information Display, 32(8), 605–646.
  7. Cao, J., Lam, K.-Y., Lee, L.-H., Liu, X., Hui, P., & Su, X. (2023). Mobile augmented reality: User interfaces, frameworks, and intelligence. ACM Computing Surveys, 55(9), 1–36.
  8. Carmigniani, J., Furht, B., Anisetti, M., Ceravolo, P., Damiani, E., & Ivkovic, M. (2011). Augmented reality technologies, systems and applications. Multimedia Tools and Applications, 51, 341–377.
  9. Chen, X. (2023). Real-Time Semantic Segmentation Algorithms for Enhanced Augmented Reality. Journal of Computational Innovation, 3(1).
  10. Cheng, P., Xiong, Z., Bao, Y., Zhuang, P., Zhang, Y., Blasch, E., & Chen, G. (2023). A deep learning-enhanced multi-modal sensing platform for robust human object detection and tracking in challenging environments. Electronics, 12(16), 3423.
  11. Divya, B., Asha, R. M., Thara, P., & Prabha, R. (2023). An IoT Enabled Smart Home Automation Using Augmented Reality in Computer Vision. 2023 International Conference on Self Sustainable Artificial Intelligence Systems (ICSSAS), 1755–1761.
  12. Doskarayev, B., Omarov, N., Omarov, B., Ismagulova, Z., Kozhamkulova, Z., Nurlybaeva, E., & Kasimova, G. (2023). Development of computer vision-enabled augmented reality games to increase motivation for sports. International Journal of Advanced Computer Science and Applications, 14(4).
  13. Ghasemi, Y., Jeong, H., Choi, S. H., Park, K.-B., & Lee, J. Y. (2022). Deep learning-based object detection in augmented reality: A systematic review. Computers in Industry, 139, 103661.
  14. Hernández, N., Almeida, F., & Blanco, V. (2024). Optimizing convolutional neural networks for IoT devices: performance and energy efficiency of quantization techniques. Journal of Supercomputing, 9.
  15. Jacobs, C., Foote, G., & Williams, M. (2023). Evaluating user experience with immersive technology in simulation-based education: A modified Delphi study with qualitative analysis. Plos One, 18(8), e0275766.
  16. Kaatrasalo, V. (2022). Computer vision methods for augmented reality.
  17. Kavitha, M. G. (2024). Development of Augmented Reality Platform Using Image Processing with Deep Learning Techniques. J. Electrical Systems, 20(3s), 249–256.
  18. Lasfar, R., & Tóth, G. (2024). The difference of model robustness assessment using cross‐validation and bootstrap methods. Journal of Chemometrics, 38(6), e3530.
  19. Lu, W. (2023). The Integration of Artificial Intelligence and Computer Vision Technologies in Virtual Reality and Augmented Reality Applications. International Conference on Big Data Analytics for Cyber-Physical System in Smart City, 79–87.
  20. Omarov, N., Omarov, B., Baibaktina, A., Abilmazhinova, B., Abdimukhan, T., Doskarayev, B., & Adilzhan, A. (2023). Applying Artificial Intelligence and Computer Vision for Augmented Reality Game Development in Sports. International Journal of Advanced Computer Science and Applications, 14(8).
  21. Perera, P., Perera, S., Jin, X., Rashidi, M., Nanayakkara, S., Yazbek, G., & Yazbek, A. (2025). An Innovative Software Development Methodology for Deep Learning-Driven Visual Computing in Built Environment Applications. Journal of Information Technology in Construction (ITcon), 30(41), 1017–1040.
  22. Pflieger, L. C. J., Hartmann, C., & Bannert, M. (2024). Enhancing knowledge construction in emerging technologies: The role of imagination training in immersive virtual reality environments. Discover Education, 3(1), 65.
  23. Rohan, A., Hasan, M. J., & Petrovski, A. (2025). A systematic literature review on deep learning-based depth estimation in computer vision. ArXiv Preprint ArXiv:2501.05147.
  24. Shewail, A. S., Zayed, H. H., & Elsayed, N. A. M. (2024). Real-time indoor tracking for augmented reality using computer vision technique. Int J Artif Intell, 13(2), 1845–1857.
  25. Shi, R. (2024). Integrating computer vision and AI for interactive augmented reality experiences in new media. Applied and Computational Engineering, 102, 49–54.
  26. Silalahi, E. B., & Silalahi, N. (2023). Linguistics Realization Analysis on Slang Word; Social Media Whatsapp. JETAL: Journal of English Teaching & Applied Linguistic, 5(1), 8–13.
  27. Subrahmanyam, S. (2025). Experimental and Simulation-Based Research: Designing Controlled Experiments for Maritime Training and Safety. In Research Methods for Advancing the Maritime Industry (pp. 145–170). IGI Global Scientific Publishing.
  28. Swami, S., Singh, P., & Singh, B. (2025). Revolutionizing E-Commerce: The Synergy of Computer Vision and Augmented Reality. In Creating AI Synergy Through Business Technology Transformation (pp. 115–136). IGI Global.
  29. Syed, T. A., Siddiqui, M. S., Abdullah, H. B., Jan, S., Namoun, A., Alzahrani, A., Nadeem, A., & Alkhodre, A. B. (2022). In-depth review of augmented reality: Tracking technologies, development tools, AR displays, collaborative AR, and security concerns. Sensors, 23(1), 146.
  30. Theodorou, C., Velisavljevic, V., Dyo, V., & Nonyelu, F. (2022). Visual SLAM algorithms and their application for AR, mapping, localization and wayfinding. Array, 15, 100222.
  31. Urquhart, L., Petrakis, K., & Wodehouse, A. (2024). User engagement in physical-digital interaction design. DS 131: Proceedings of the International Conference on Engineering and Product Design Education (E&PDE 2024), 187–192.
  32. Wang, S., Zargar, S. A., & Yuan, F.-G. (2021). Augmented reality for enhanced visual inspection through knowledge-based deep learning. Structural Health Monitoring, 20(1), 426–442.
  33. Yang, L., Kumar, R., Kaur, R., Babbar, A., Makhanshahi, G. S., Singh, A., Kumar, R., Bhowmik, A., & Alawadi, A. H. (2024). Exploring the role of computer vision in product design and development: a comprehensive review. International Journal on Interactive Design and Manufacturing (IJIDeM), 18(6), 3633–3680.
  34. Zhou, B., & Güven, S. (2020). Fine-grained visual recognition in mobile augmented reality for technical support. IEEE Transactions on Visualization and Computer Graphics, 26(12), 3514–3523.

Downloads

PDF

Statistic

Abstract Views
: 9 times
PDF
: 2 times