Breaking Bad (Design): Challenging AI User Interface Accessibility Guardrails

Abstract: What happens when we prompt AI to create “bad” design? To find out, we challenged four AI-driven design tools to create user interfaces that explicitly violate established accessibility criteria, only to discover them as prisoners of their usability-oriented training. This finding raises a critical question: How can we develop AI that understands accessibility deeply enough to know when to comply and when to thoughtfully challenge established design principles? Through systematic attempts to subvert AI tools and make them follow our request, we found them both rigid and limited: capable of reproducing accessible patterns, but incapable of thoughtful deviation when context demanded it. By adopting the lens of intentional inaccessibility as an investigation method, we raise questions about the nature of design intelligence that demand reconsideration of how design knowledge is integrated into AI-driven design tools.

Authors: Alexandra-Elena Gurita, Radu-Daniel Vatavu

Conference: CHI Conference on Human Factors in Computing Systems

Publication: Association for Computing Machinery, New York, NY, USA, Article 624, 1–7

Link: https://doi.org/10.1145/3706599.3716220