Faculty of Mechanical Engineering Niš, University of Niš

Faculty of Mechanical Engineering Niš, University of Niš

Faculty of Mechanical Engineering Niš, University of Niš

Faculty of Mechanical Engineering Niš, University of Niš

Faculty of Mechanical Engineering Niš, University of Niš

The paper is available at the following link: PDF

Abstract:

Modern robotic systems increasingly couple motion planning with rich visual perception, yet experimenting directly on hardware is often costly and inflexible. This poster presents a study on robotic motion simulation with integrated computer vision, focusing on how robotic simulation environment such as RoboDK supports coupling with OpenCV and YOLO- based object detection. We implement a pipeline in which simulated cameras provide image streams to an external vision stack for preprocessing, detection, and tracking, while simulated robots execute corresponding motion commands in closed loop. We report quantitative metrics such as frame rate, end-to-end response time, and detection reliability, as well as qualitative observations on development complexity and sim-to-real transfer potential. The results highlight practical trade-offs of a simulator and provide guidelines for configuring a simulation framework when rapid experimentation with vision-guided robot motion is required

Keywords:

Robotic motion simulation; Computer vision; Vision-guided robotics.

REFERENCES:

[1] Y. Liu, Z. Yan, J. Ju, and X. Su, "Trajectory correction strategy for vision-guided industrial robot operating system with continuous operating trajectories," Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, 2025.
[2] K. Artemov and S. Kolyubin, "Collaborative robot manipulator control in dynamic environment using computer vision system," Journal of Physics: Conference Series, 2021.
[4] C. Cristoiu and A. M. Ivan, "Integration of real signals acquired through external sensors into RoboDK simulation of robotic industrial applications," Multidisciplinary Digital Publishing Institute, 2025.
[5] J. Semjon, M. Kocan, and R. Demko, "Creation of a 3d model of robot activity assembly in the RoboDK environment," 2021.
[6] O. M. Nwakeze, O. C. Okeke, and I. J. Mgbemfulike, "Intelligent robotic object grasping system using computer vision and deep reinforcement learning techniques," 2025.
[7] J. Vasić, L. Milić, and M. Raković, "Real-time object detection for mobile robot manipulation trained on synthetic data from unreal engine," 2025 12th International Conference on Electrical, Electronic and Computing Engineering (IcETRAN), 2025.
[8] J. Zhao, "Adaptive robust learning control for a 6- DOF robotic arm with real-time object detection using YOLO v10," Applied and Computational Engineering, 2025.
[9] J. Guo, H.-T. Nguyen, C. Liu, and C. Cheah, "Convolutional neural network-based robot control for an eye-in-hand camera," IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023.
[10] C. Imai et al., "Vision-guided quadrupedal locomotion in the wild with multi-modal delay randomization," IEEE/RJS International Conference on Intelligent RObots and Systems, 2021.
[11] S.-H. Han, W. H. Seo, K. Yoon, and M. Lee, "Real- time control of an industrial robot using image-based visual servoing," 2003.
[12] A. V. Pitkevich and I. Makarov, "A survey on sim- to-real transfer methods for robotic manipulation," Symposium on Intelligent Systems and Informatics, 2024.
[13] E. Aljalbout et al., "The reality gap in robotics: Challenges, solutions, and best practices," Annual Review of Control Robotics and Autonomous Systems, 2025.
[14] M. Pollk and K. Goryl, "Development of a simulation computational model for hole detection and generation of robot tool movement for fitting mold preparation nozzles," Machines, 2025.
[15] H. Soebhakti, S. Prayoga, R. Fatekha, and M. B. Fashla, "The real-time object detection system on mobile soccer robot using YOLO v3," International Conference on New Technology of Agricultural Engineering, 2019.
[16] L. Zhao, H. Zhao, C. Chen, X. Rong, and Y. Li, "YOLO-GRCNN based object detection and grasp detection on distribution line," IEEE International Conference on Robotics and Biomimetics, 2024.
[17] J. Zhou and G. Zhang, "Design of a ROS-based vision-guided mobile robot for object transportation," IEEE Advanced Information Technology, Electronic and Automation Control Conference, 2025.
[18] J. Zhang et al., "HiRT: Enhancing robotic control with hierarchical robot transformers," Conference on Robot Learning, 2024.
[19] X. Chen et al., "InternVLA-M1: A spatially guided vision-language-action framework for generalist robot policy," arXiv.org, 2025.
[20] Z. Yang, C. R. Garrett, D. Fox, T. Lozano-P"erez, and L. Kaelbling, "Guiding long-horizon task and motion planning with vision language models," IEEE International Conference on Robotics and Automation, 2024.