Evaluating the usability of a Quranic theme extraction and visualization system using task-based usability testing

Usability testing is crucial for evaluating the effectiveness and user experience of information retrieval systems. This study presents a task-based usability evaluation of a Quranic Theme Extraction and Visualization System, which integrates Natural Language Processing (NLP) techniques which are RAKE algorithm for keyword extraction and DistilBERT for theme classification. The system was assessed using Task-Based Usability Testing, where 20 participants completed predefined tasks while key usability metrics were recorded, including task success rate, task completion time, error rate, relative efficiency, and user satisfaction. The results indicate a task success rate of 91.25%, an overall relative efficiency of 96.12%, and a high user satisfaction score, demonstrating the system’s usability and effectiveness. However, challenges in handling misspelled queries and complex synonym recognition were identified, highlighting areas for improvement. These findings underscore the importance of usability-driven evaluation in the development of Quranic information retrieval systems to enhance accessibility and user interaction.