NONINTRUSIVE VIRTUAL TRAINER PROTOTYPE FOR EXERCISE ROUTINES IN UNITY USING MOTION CAPTURE

NONINTRUSIVE VIRTUAL TRAINER PROTOTYPE FOR EXERCISE ROUTINES IN UNITY USING MOTION CAPTURE

Virtuality is an innovative process seen in recent years. This process allows transforming scenarios to a virtual environment for simulations, practices, or tests so that all results can be studied. In turn, these environments can be supported by new tools and methodologies that increase the analysis capabilities, such as motion capture to generate animations, tracking of robotic equipment and the study of human movement, which benefits the development of systems with these objectives. A prototype of a nonintrusive virtual trainer capable of capturing motion and determining the correct execution of exercise routines is presented, using a virtual environment developed in the Unity video game engine. The system employs a SHDR webcam for real-time capture of the movement performed by the user, which is processed to track the pose and joints using Machine Learning through the MediaPipe library. This article explains the construction of the prototype and presents the results of the project.

DOI: http://dx.doi.org/10.21017/rimci.2023.v10.n19.a125

1. Y. Barniv, M. Aguilar, and E. Hasanbelliu, “Using EMG to anticipate head motion for virtual environment applications,” IEEE Trans. Biomed. Eng., vol. 52, no. 6, pp. 1078–1093, 2005, doi:10.1109/TBME.2005.848378.
2. L. Li et al., “Application of virtual reality technology in clinical medicine.,” Am. J. Transl. Res., vol. 9, no. 9, pp. 3867–3880, 2017. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5622235/.
3. A. Rizzo et al., “Virtual environment applications in clinical neuropsychology,” in Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048), 2000, pp. 63–70. doi: 10.1109/VR.2000.840364.
4. G. Nagymate and R. Kiss, “Application of OptiTrack motion capture systems in human movement analysis A systematic literature review,” Recent Innov. Mechatronics, vol. 5, 2018, doi: 10.17667/riim.2018.1/13.
5. A. Bottino and A. Laurentini, “Experimenting with nonintrusive motion capture in a virtual environment,” Vis. Comput., vol. 17, pp. 14–29, 2001, doi: 10.1007/s003710000091.
6. D. Osokin, “Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose.” arXiv, 2018. doi: 10.48550/ARXIV.1811.12004.
7. K. E. Lara, “El doble desafío de la Educación Física virtual,” Panamá América: Salud y bienestar, Panamá, Sep. 07, 2020. [Online]. Available: https://www.panamaamerica.com.pa/deportes/el-dobledesafio-de-la-educacion-fisica-virtual-1171545
8. L. P. Berg and J. M. Vance, “Industry use of virtual reality in product design and manufacturing: a survey,” Virtual Real., vol. 21, no. 1, pp. 1–17, 2017. https://link.springer.com/article/10.1007/s10055-016-0293-9
9. W. A. IJsselsteijn, Y. A. W. de Kort, J. Westerink, M. de Jager, and R. Bonants, “Virtual Fitness: Stimulating Exercise Behavior through Media Technology,” Presence Teleoperators Virtual Environ., vol. 15, no. 6, pp. 688–698, Dec. 2006.
10. J. C. P. Chan, H. Leung, J. K. T. Tang, and T. Komura, “A Virtual Reality Dance Training System Using Motion Capture Technology,” IEEE Trans. Learn. Technol., vol. 4, no. 2, pp. 187–195, 2011, doi:10.1109/TLT.2010.27.
11. T. Hayami, M. Tanaka, M. Okutomi, T. Shibata, and S. Senda, “Super-high Dynamic Range Imaging,” in 2014 22nd International Conference on Pattern Recognition, 2014, pp. 720–725. doi: 10.1109/ICPR.2014.134.
12. P. Galindo Salgado, “What’s New In Python 3.10,” 2022. https://docs.python.org/3/whatsnew/3.10.html
13. Unity Technologies, “Unity,” 2022,[Online]. Available: https://unity.com/
14. Blender Foundation, “Blender: Introducing blender 3.1,” 2022. https://www.blender.org/
15. G. Bradski, “The OpenCV Library” Dr. Dobb’s J. Softw. Tools, 2000.
16. C. Lugaresi et al., “MediaPipe: A Framework for Building Perception Pipelines.” arXiv, 2019, https://arxiv.org/abs/1906.08172.
17. Python, “Socket - Low-level networking interface,”2022. https://docs.python.org/3/library/socket