Skip to main content
CISTI'2020 - 15th Iberian Conference on Information Systems and Technologies

Full Program »

Using machine learning for cognitive Robotic Process Automation (RPA)

There are many business routine tasks and processes which are performed by qualified resources which can be reallocated, allowing qualified workers to dedicate their effort to other fields/tasks with more relevance. With Robotic Process Automation (RPA), repetitive tasks, even complex computer routine processes, can be automated and improved, independently of the interface or menu location.\\ This paper proposes an RPA application, which in real-time, dynamically detects objects in software applications interface, allowing flexibility, performance, and accuracy regardless of the OS, location of interface tools, and necessary menus to reach it. For this purpose, a Convolution Neural Network (CNN) is trained with several interfaces and menus and used to classify software interfaces in real-time. Furthermore, a developed software takes automated actions, moving the mouse pointer, clicking, editing text, and performing any necessary action.\\ Results show that the proposed RPA technique based on deep learning is capable of detecting objects in real-time, classify them, with outstanding accuracy, and take dynamic actions. This way, it is possible to automate any computer task.

Pedro Martins
ESTGV
Portugal

Filipe Sá
ESTGV
Portugal

Francisco Morgado
ESTGV
Portugal

Carlos Cunha
ESTGV
Portugal

 


Powered by OpenConf®
Copyright ©2002-2018 Zakon Group LLC