The A(I)BILITIES project team participated in the International Conference on User-System Interaction, where it presented a series of recent results in digital accessibility, artificial intelligence, and human–computer interaction.

At the current stage of the project, the presented contributions reflect the maturity of the developed solutions and the consolidation of the results achieved through the partnership between Ștefan cel Mare University of Suceava and ASSIST Software. The papers address the automatic generation of adaptive mobile user interfaces, gesture recognition, the reuse of touchscreen interaction data, and voice-based control of digital environments.

Through these results, the A(I)BILITIES project reaffirms its objective of contributing to the development of more inclusive digital technologies, adapted to the diverse abilities and needs of users.

The A(I)BILITIES Platform Validated at TRL 5

During the conference, the team presented the A(I)BILITIES platform, designed to automatically generate adaptive mobile user interfaces using artificial intelligence.

The platform is built around three main directions:

• digital accessibility;
• ability-centered design;
• artificial intelligence as a service.

The paper presented the validation of the platform at Technology Readiness Level 5 (TRL 5). At this level, the solution’s components were validated, integrated, and implemented in a functional prototype.

The platform includes a software architecture and a software development kit (SDK). These enable developers to integrate adaptive interfaces into mobile applications. Its functionality was demonstrated through a WCAG-based evaluation of a mobile graphical user interface, conducted before and after adaptation.ionalitatea a fost demonstrată printr-o evaluare a unei interfețe grafice mobile, înainte și după adaptare, realizată pe baza criteriilor WCAG.

Gesture-A11Y Receives Recognition for Its Demonstration

The team also presented Gesture-A11Y, a web platform that supports the exploration and reuse of touchscreen gesture datasets collected from people with different impairments.

The platform enables users to search, filter, visualize, and export data collected from smartphones, tablets, smartwatches, and other touch-enabled devices. As a result, researchers can more easily prepare gesture studies, while developers can build applications with interaction methods tailored to specific user groups.

The paper on Gesture-A11Y received the Best Demo Honorable Mention Award.

Gesture Recognition for More Accessible Interaction

Another paper presented at the International Conference on User-System Interaction evaluated several deep learning architectures for recognizing touchscreen gestures and mid-air movements.

The study used data collected from people with upper-body motor impairments. Among the evaluated models, the 1D-CNN architecture achieved the best results:

• 94.7% accuracy for stroke gestures;
• 84.0% accuracy for mid-air motion gestures.

The findings show that gesture recognition performance cannot be explained solely by medical diagnoses or general disability scores. This conclusion highlights the need to design digital technologies around each user’s actual abilities.

A Local-First Voice Assistant for Digital Environment Control

The conference also featured a local-first voice assistant capable of recognizing the speaker and controlling digital environments through natural-language commands. The solution integrates speech recognition, speaker identification, large language models, task automation, and text-to-speech technology. To protect users, the system classifies commands according to their level of risk and requests explicit confirmation before performing actions with potentially harmful effects.

The preliminary evaluation indicated an overall tool invocation success rate of 95.5%. This result highlights the potential of local-first voice assistants to provide digital interactions that are more accessible and more privacy-friendly.

Digital Technologies Adapted to Users

The team’s participation in the International Conference on User-System Interaction confirms the direction of the A(I)BILITIES project: developing accessible, adaptive, and secure technologies.

Through AI-generated interfaces, gesture-based interaction methods, and voice solutions, the project contributes to creating digital experiences that better respond to users’ individual needs and abilities.

All papers presented at the conference can be accessed in the Project Status section of the A(I)BILITIES website, where all publications produced within the project are available.