Motivation
In my initial project brief, I wrote: '...imagination plays a crucial role in addressing the complexities and interconnectedness of human behavior in the real world, where I could further develop my skills in helping organizations navigate AI development with a designer's empathy, human-centered design methodologies, and creative problem-solving...' What began as an intuition about creativity's role in designing LLM systems has evolved into 7 prototype designs a comprehensive user study.
Early in this journey, I came to realize that LLMs are neither inherently good nor bad as people often debate, but rather tools that people use to create. This shift in my perspective has been motivating me to create, with LLMs as design materials.
Methodology
This project was a collision between making and measuring, both explorative and hypothesis-driven. I prototyped in the beginning and pushed against the boundaries of what LLMs could become. Each prototype was a question made tangible: How might we leverage LLM technology to improve navigation experience for Decathlon website users?
But prototypes could lie, in the sense that we do not necessarily know how users will interact with them. To understand the user experience, I conducted a user study that compared the new interface with a more traditional chat interface.
Prototyping generated hypotheses. Scientific research tested and validated them. The two approaches created a loop that iterated - unexpected behaviors in prototypes led to focused studies, while patterns in the data inspired new design directions.
Prototyping
In the early prototypes, I started by exploring with LLMs in unconventional ways, from direct to non-direct, such as using LLM as a backend for chat that understood user intent, or as a keyword generator that surprises users with delight, even expading the output from text to 3D location coordinates that helps users explore the store digitally.
The ealy explorations were not perfect in pixels, but the process was such a joy to me, where I have adopted the mindset of a creative, who needs no permission to make or worries about the outcome, but rather focus on the process of playing and improving.
After this exploratory phase, I shifted focus to my second research question and created a more Decathlon-specific prototype. Users could ask product questions and receive insights synthesized from user reviews, with the LLM acting as an intelligent aggregator rather than a simple chatbot.
Through user testing and reflection, I iterated on the design, developing the feature where the system would analyze user previous queires and make prediction when the usr travels to another product page.
In the final stage, I created a control condition of conventional chat ineterface, in order to compare with the non-linear interface.
Insights
Firstly, the result from the user study (N=7) shows that the structured interface showed better usability, however, this comparison is limited since we only tested it on review synthesis. More interestingly, through the qualitative data, I started to form an understanding of the tensions behind the comparison between the conventional chat inetrface and non-linear interface.
Cognitive load, boundary, affordance, and spatial memory are the concepts at the center of the tensions, and they are, not new concepts, but rather the same concepts that have been discussed in the field of HCI and cognitive psychology.
Spatial memory emerged as a crucial factor in e-commerce navigation throughout the user study. While linear conversational interactions proved effective for small, time-consuming validation tasks—like checking product specifications or availability—the user study revealed that people still fundamentally rely on spatial memory when navigating e-commerce environments.
This reliance on spatial memory highlighted the importance of visual affordances in interface design. While chat feels natural to users, it isn't the right solution for every interaction problem. When designing AI systems for specific functions rather than general-purpose assistance, visual affordances often communicate possibilities and constraints more effectively than conversational prompts.
Effective affordances also communicate boundaries. While the tech industry often builds products to showcase capabilities—demonstrating how powerful or versatile a system can be—what users actually need is clarity about the product's limitations.
Chat interfaces also potentially impose continuous cognitive load. Unlike clicking through familiar visual patterns where users can rely on muscle memory and established habits, conversational AI requires constant mental engagement—formulating queries, interpreting responses, maintaining conversational context.
At the same time that LLMs are changing how we communicate with our machines, they have also expanded my creative toolbox and the scope of my responsibility as a product designer to understand this two-way complexity. This research illuminated that complexity: LLMs aren't just new interface options, they're design materials that require us to think differently about spatial memory, affordances, boundaries, and cognitive load. The future of AI interaction lies not in choosing between conversational and visual design, but in understanding when each serves human needs best.
Resources
🖲️ Prototype - Chat Interface: https://tinyurl.com/4d462jk6
🖲️ Prototype - Redesigned Interface: https://tinyurl.com/r9jxf9zb