MotionInput: software that allows you to interact with a computer without touching it

Paris Baker, 31 years old and mother of two children, was an elite acrobatic gymnast who represented Great Britain and won silver in the European and World Championships. At 26 years old, Paris received a diagnosis of motor neuron disease (EMN), which causes muscular weakness that worsens gradually with time and leads to disability. De las manyas cosas que cambiaron en su vida, un elemento essencial fue perder la capacidad de jugar videojuegos con sus hijos.

Eso fue hasta que descubrió MotionInput.

Developed by academics and students from the Department of Computing Sciences at University College London (UCL), in collaboration with Intel, Microsoft and IBM, UCL MotionInput V3 allows truly contactless computing. With MotionInput and a common web camera, a user can control a PC making gestures with the hands, the head, the face and the full body using speech. The software analyzes these interactions and converts them into mouse, keyboard and joystick signals making full use of the existing software.

Intel has a close relationship with UCL to advise computer science projects, says Phillippa Chick, director of global accounts, health and life sciences, Intel UK. «Trabajamos con el professor Dean Mohamedally, el professor Graham Roberts and la Sra. Sheena Visram in tutoring projects, as well as in the support structure for students. This idea was proposed for the first time by the UCL team, in the summer of 2020, as a series of student projects from UCL Computer Science IXN [Industry Exchange Network] y surgió de la necesidad de la necesidad de las trabajadores de la salud durante COVID-19 cuando era necesario mantener las computadoras compartidad limpias y libres de germenes». The team brought in Atia Rafiq, NHS chief medical officer, to improve the clinical requirements that front-line medical care workers need.

MotionInput puede abrir un mundo de caseso de uso mediate el uso de manos u ojos simultamente con el habla. Cada juego ahora puede ser accesible, el progresso de los movimientos de los pacientes se puede registrar en fisiotherapy y, en un un environmento hospitalario, los rujanos pueden tomar notas través de gestos con las manos y el habla without having to touch a computer. La solution no requiere connectivati ​​ni un servicio en la nube, lo que hace que sea mucho más fácil de implementar.

“You have a great opportunity to positively impact the lives of people with chronic conditions that affect the movement,” says Phillippa.

Intel provides UCL students with tutoring and technology, including hardware and software capabilities such as Intel’s OpenVINO™ tool kit. The tool kit facilitates the development of AI-based applications and helps increase performance.

The previously trained models provided by OpenVINO™ allowed the fastest development of the diverse components and characteristics of MotionInput, which allowed the students to advance without training their own models, generally a long and intensive computation process.

Costas Stylianou, technical specialist in health and life sciences at Intel UK, explains that the optimization means that MotionInput V3 “has several improvements of orders of magnitude in efficiency and an architecture to support the growth of computer applications without contact as ecosystem”. Software engineering and architecture development for V3 was directed by UCL students Sinead V. Tattan and Carmen Meinson. Together they directed more than 50 students from UCL in various courses in computer science from UCL to build on the work. The team also worked with mentors from Microsoft and IBM, in particular Prof. Prof. Lee Stott et al. John McNamara.

The solution employs a combination of automatic learning models and computer vision to allow receptive interaction. It is customizable to allow the user to choose between a variety of modules, such as:

  • Facial navigation: the user can use his nose and eyes and a set of facial expressions to trigger actions such as clicks on the mouse button, and with the word “clic”.
  • Hand gestures: it can recognize and assign a selection of hand gestures to specific keyboard commands and shortcuts, mouse movements, native multi-touch detection and digital pencils with depth in the air.
  • Gaze with grid and magnet modes: to align the cursor in accessibility scenarios, an automatic calibration method is implemented for eye tracking that obtains gaze estimation, including grid mode and magnetic mode.
  • Seguimiento de cuerpo completo: los usuarios pueden establish ejercicios físicos y etiquetar regions en su espacio circunda para jugar juegos de computadora existentes.
  • Keyboards for quick access to voice and subtitles in vivo: Ask-KITA (Know-It-All) allows users to interact with the computer from a set of voice commands, subtitles in vivo and keyboard shortcuts that can be canceled.
  • Joypad en el aire: Users can play games with the usual buttons of the joypad ABXY en el aire with analog shooting controls.

“What makes this software so special is that it’s totally accessible,” says Phillippa. “The code does not require expensive equipment to function. It works with any standard web camera, including your portable computer. Es solo un caso de descarga y estás listo para commensar».

Due to the fact that MotionInput allows facial navigation using the nose, eyes and mouth, adds Costas, “it is ideal for people who suffer from EMN”.

What follows for MotionInput?

“The project will continue and seek to collaborate with industrial sectors. The academics and mentors are investigating what they can do to expand the cases of use and improve the user experience continuously,” says Phillippa. “Nos encanta trabajar con los estudiantes y el staff docente de UCL, ya que es inspirador ver lo que pueden hacer con la tecnología”.

O como dice Paris, mientras juega un videojuego con sus hijos, «The potential of UCL MotionInput para cambiar vidas es ilimitado».

Leave a Comment

Your email address will not be published. Required fields are marked *