VOLKOV O.Ye., PhD (Engineering), Senior Researcher,
Director
https://orcid.org/0000-0002-5418-6723, e-mail: alexvolk@ukr.net

KOMAR M.M., PhD (Engineering), Senior Researcher
Deputy Director for Scientific and Organizational Work,
https://orcid.org/0000-0001-9194-2850, e-mail: nickkomar08@gmail.com

POPOV I.V., PhD Student,
Researcher of the Intelligent Control Department
https://orcid.org/0009-0009-7961-9431, e-mail: popigor7@gmail.com

SIMAKHIN V.M., PhD (Engineering),
Senior Researcher of the Research Laboratory of Unmanned Complexes and System
https://orcid.org/0000-0003-4497-0925, e-mail: thevladsima@gmail.com

International Research and Training Center
for Information Technologies and Systems
of the National Academy of Sciences
of Ukraine and the Ministry of Education and Science of Ukraine,
40, Acad. Glushkov av., 03187, Kyiv, Ukraine

APPROACHES TO ADAPTATION OF OPERATORS’ FIELD OF VISION SETTINGS DURING TRAINING AND OPERATION OF DRONES

Introduction. The widespread use of unmanned aerial vehicles (UAVs), including first-person view drones (FPV drones), makes it more urgent to create modern training complexes for training UAV operators and ensuring their optimal performance.

The purpose of the paper is to investigate ways and methods of using artificial intelligence to reduce the load on the vestibular apparatus of external pilots of FPV drones.

Results. Various factors influencing the operator’s vestibular apparatus have been studied, especially those that allow the use of artificial intelligence (AI) tools. The analysis of a group of software and hardware factors and separately the means of adjusting the operator’s working field of vision is relevant. The proposed method is effective both during operation and during training on training software and hardware complexes, where this is especially important, since operators train extremely intensively.

Conclusions. Individual adaptation of field of view settings using AI methods has significant potential to improve the efficiency and comfort of external pilots of FPV drones, although there are certain shortcomings and limitations that should be taken into account.

Ensuring data confidentiality, managing computing resources and algorithm efficiency are key aspects that require attention for the successful implementation of such solutions. It is possible to ensure optimal operator performance when performing tasks that require vestibular functions. This will reduce the requirements for external pilots and, accordingly, increase the number of people capable of controlling FPV drones. Methods for using AI to optimize the operator’s field of view work successfully both during the operation of real drones and on training software and hardware complexes.

Keywords: FPV drone, training complex, unmanned aerial vehicle, external pilot, operator, virtual reality, artificial intelligence, method, visual-vestibular conflict, operator` field of view, optimal operator performance.

REFERENCES

1. FPV Drone Market: Global Industry Analysis and Forecast (2023-2029), https://www.maximizemarketresearch.com/market-report/fpv-drone-market/211053/ 

2. P. Wojciechowski and K. Wojtowicz, “Simulator sickness and cybersickness as significant indicators in a primary selection of candidates for FPV drone piloting,” 2022 IEEE 9th International Workshop on Metrology for AeroSpace (MetroAeroSpace), Pisa, Italy, 2022, pp. 130-133.
https://doi.org/10.1109/MetroAeroSpace54187.2022.9856141 

3. Nesthus, Thomas & Fercho, Kelene & Durham, Justin & Mofle, Theodore & Nesmith, Blake & Hu, Peter. (2021). Summary Final Report for Unmanned Aircraft Systems in Air Carrier Operations: UAS Operator Fatigue, DOT/FAA/AM-21/16, https://www.faa.gov/ data_research/research/med_humanfacs/oamtechreports/2020s/media/202116.pdf 

4. Song, Zhe & Dong, Jiaoyang & Zhang, Xiting & Xu, Xiaotian & Pu, Fang & Fan, Xiaoya & Li, Shuyu. (2024). Effects of Visual-Vestibular Conflicts Caused by Visual Input on Out-of-Body Experience Induced by Visual-Tactile Stimulation in Virtual Reality. 345-352.
https://doi.org/10.1007/978-3-031-51455-5_38 

5. Ding F, Sun C, He S. Anti-Swing Control for Quadrotor-Slung Load Transportation System with Underactuated State Constraints. Sensors. 2023; 23(21):8995.
https://doi.org/10.3390/s23218995 

6. Javaid, A.; Rasool, S.; Maqsood, A. Analysis of Visual and Vestibular Information on Motion Sickness in Flight Simulation. Aerospace 2024, 11, 139.
https://doi.org/10.3390/aerospace11020139 

7. Deng, Zilong, Dongxiao Yang, Xiaohu Zhang, Yuguang Dong, Chengbo Liu, and Qiang Shen. 2020. “Real-Time Image Stabilization Method Based on Optical Flow and Binary Point Feature Matching” Electronics 9, no. 1: 198.
https://doi.org/10.3390/electronics9010198 

8. Jarron, D., Shahbazi, M., Lichti, D., Radovanovic, R.: MODELLING WIDE-ANGLE LENS CAMERAS FOR METROLOGY AND MAPPING APPLICATIONS, ISPRS 2019 Ann. Photogramm. Remote Sens. Spatial Inf. Sci., IV-2/W7, 79-86.
https://doi.org/10.5194/isprs-annals-IV-2-W7-79-2019 

9. Watanabe, K., Takahashi, M. Head-synced Drone Control for Reducing Virtual Reality Sickness. J Intell Robot Syst 97, 733-744 (2020).
https://doi.org/10.1007/s10846-019-01054-6 

10. Arshad, I., De Mello, P., Ender, M., McEwen, J. D., & Ferré, E. R. (2021). Reducing Cybersickness in 360-Degree Virtual Reality. Multisensory Research, 35(2), 203-219.
https://doi.org/10.1163/22134808-bja10066 

11. Anne Harrington, Vasha DuTell, Mark Hamilton, Ayush Tewari, Simon Stent, William T. Freeman, Ruth Rosenholtz COCO-Periph: Bridging the Gap Between Human and Machine Perception in the Periphery ICLR 2024 Conference, ICLR 2024 https://openreview.net/pdf?id=MiRPBbQNHv 

12. E. Wen et al., “VR.net: A Real-world Dataset for Virtual Reality Motion Sickness Research,” in IEEE Transactions on Visualization and Computer Graphics, vol. 30, no. 5, pp. 2330-2336, May 2024.
https://doi.org/10.1109/TVCG.2024.3372044 

13. W. Kim, “Low-Light Image Enhancement: A Comparative Review and Prospects,” in IEEE Access, vol. 10, pp. 84535-84557, 2022.
https://doi.org/10.1109/ACCESS.2022.3197629 

14. Ilesanmi, A.E., Ilesanmi, T.O. Methods for image denoising using convolutional neural network: a review. Complex Intell. Syst. 7, 2179-2198 (2021).
https://doi.org/10.1007/s40747-021-00428-4 

15. Hüsem, Hürkal & Orman, Zeynep. (2020). A Survey on Image Super-Resolution with Generative Adversarial Networks. Acta INFOLOGICA.
https://doi.org/10.26650/acin.765320 

16. Wang, X. et al. (2019). ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks. In: Leal-Taixé, L., Roth, S. (eds) Computer Vision – ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science, vol 11133. Springer, Cham.
https://doi.org/10.1007/978-3-030-11021-5_5 

17. Jung, Felix, Kevin Frey, David Zimmer, and Timo Mühlhaus. 2023. “DeepSTABp: A Deep Learning Approach for the Prediction of Thermal Protein Stability” International Journal of Molecular Sciences 24, no. 8: 7444.
https://doi.org/10.3390/ijms24087444 

18. J. Yu and R. Ramamoorthi, “Learning Video Stabilization Using Optical Flow,” 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 8156-8164.
https://doi.org/10.1109/CVPR42600.2020.00818 

19. T. Liu, G. Wan, H. Bai, X. Kong, B. Tang and F. Wang, “Real-Time Video Stabilization Algorithm Based on SuperPoint,” in IEEE Transactions on Instrumentation and Measurement, vol. 73, pp. 1-13, 2024, Art no. 5004113.
https://doi.org/10.1109/TIM.2023.3342849 

20. A. Censi, A. Fusiello and V. Roberto, “Image stabilization by features tracking,” Proceedings 10th International Conference on Image Analysis and Processing, Venice, Italy, 1999, pp. 665-667.
https://doi.org/10.1109/ICIAP.1999.797671 

21. https://www.dji.com/global 

22. https://www.skydio.com/

Received 27.09.2024