RetailARVA: AR-based Virtual Assistant with Conversational Capabilities to Enhance Retail Customer Experience

Abstract

Abstract Despite the rise of e-commerce, 94% of consumers continue to shop at brick-and-mortar stores, with 90% of millennial retail spending occurring in physical locations. However, in-store experiences often fail to meet the expectations shaped by online shopping, lacking real-time assistance, detailed product information, and personalized support. Studies show that 56% of in-store shoppers now use their smartphones to supplement product information, highlighting a growing informational gap in traditional retail. This research introduces RetailARVA, a smartphone-based Augmented Reality (AR) virtual assistant enhanced with Large Language Models (LLMs), designed to bridge this gap. Focused initially on the skincare domain, RetailARVA enables real-time product identification through barcode scanning, delivers personalized recommendations based on user preferences, and provides natural conversational support via an interactive 3D avatar — all without the need for wearable hardware. A controlled user study involving 30 participants demonstrated that users employing RetailARVA completed shopping tasks much faster, at best one task simulated in the user study was completed 80.32% faster. The study also reports a lower cognitive load (NASA-TLX scores), and the users rated the system’s usability as ”Excellent” with a mean System Usability Scale (SUS) score of 82.5. User engagement metrics from the User Experience Questionnaire (UEQ) showed significant gains in Attractiveness, Efficiency , and Stimulation compared to the control group. These results strongly support the effectiveness of integrating AR and conversational AI into physical retail environments to enhance customer experience, improve decision-making, and boost purchase confidence.