Oracle CX Commerce Cloud Mobile Storefront

Overview

Course: Needs Assessment and Usability Evaluation Team size: 4

Duration: Jan, 2020 - Apr, 2020 My Role: UX Researcher, Communication Lead

Skills learned: Customer experience (CX) research, Report generation, Interviews, Surveys, Usability studies, Heuristic analysis, Presentation of research results

Problem

Our client would like a team to evaluate the usability of Oracle’s CX commerce cloud mobile storefront, help determine the most important features that will differentiate a retailer from other e-commerce platforms, and move the customer forward in the purchase path when accessing an e-commerce site from a mobile device.

How might we improve the current prototype so that it differentiates itself from other e-commerce platforms and attracts online shoppers to shop and complete their purchase on a mobile device?

Process

Interaction maps

To better understand the flow of the product and evaluate what is involved in the project, we created interaction maps based on current prototype of the e-commerce storefront. After an exploratory search of the current working prototype, we created interaction maps based on four categories: Home/Navigation, Registration/Login, Product details, and Check out. Due to the NDA, I cannot show the maps publicly. However, we generated valuable insights through our exploration on these four flows carried by mobile shopping users: we found potential pain points in the navigation bar as some labels were not consistent. We also found that the “add to wishlist” function might differentiate the prototype from other e-commerce platforms as customers are able to choose different categories of wishlists.

Interviews

Based on our client’s objective, we came up with the following research questions to better understand the needs of mobile shopping customers:

• What are the motivations of customers to shop on a mobile device?

• What are the pain points when customers shop on a mobile device?

• What might prevent customers from completing their purchase on a mobile device?

• What features might motivate customers to shop more frequently on a mobile device?

We defined our target audience as any age groups who have shopped or haven’t shopped on a mobile device. The decision was based on our research questions. Please see our interview protocol here. Our group interviewed 4 participants through our personal networks. Each interview lasted about 30 to 40 minutes. This process also helped our team generate 3 personas of our target customer:

Comparative analysis

To gain a deeper insights about current e-commerce platforms available in the market, we selected six e-commerce platforms and differentiated them into five different competitor types.

I was responsible for evaluating SAP commerce cloud and Salesforce platforms. I selected three mobile apps for each platform to evaluate their customer-facing storefront. Each of us finished the evaluation chart before concluding our findings and recommendations.

Surveys

While interviews give us a rich and detailed understanding of why users do the things they do on a e-commerce site, surveys allow us to gain the prevalence and frequency of the key attitudes, behaviors, and characteristics of our target population.

Therefore, we developed a single survey. The survey contains an introduction to participants, followed by questions designed to collect information about user preferences, attitudes regarding particular features, general shopping behaviors, and shopper demographic characteristics.

The survey was created using the Qualtrics Online Survey platform.  It was distributed to a broad sampling of potential participants for a period of approximately one week, and we received a total of 57 responses. Below are some of our findings that is helpful for answering our research questions:

Heuristic evaluation

After gaining a better understanding of the needs and motivations of mobile shoppers, we then moved to the evaluative stage where we conducted a heuristic analysis to examine whether the interfaces meet certain usability standards.

I led the heuristic evaluation and assessed the mobile storefront using Nielson’s Ten Heuristics (Nielson, 1994). As a team, we first reviewed the system to develop questions specific to our product that aligned with the heuristics.

Some of our questions were broad to cover general issues in the interface, while others focused more specifically on tasks and features that were highlighted to be of importance from our previous research including surveys, comparative analysis, and user interviews. The main tasks we focused on for our questions were concerned with login and registration, search and browsing, product details, and checkout and payment procedures. Please see a complete list of the questions.

Next, each of us evaluate the prototype using the definition of those heuristics. Please see our complete group heuristic matrix here.

Usability test

To gain more direct feedback from target audience, our group conducted 4 moderated usability test to evaluate current e-commerce site. Due to the pandemic caused by COVID-19, all of our usability test were conducted remotely via a video call with participants. We had participants sign the NDA form and send back to us before each scheduled test.

During the test, we had users view the prototype through our screen and asked them to direct the test moderator to guide the mouse while reading aloud what they were thinking. One of us served as moderators to interact with the prototype based on what participants said, one served as helpers to log the time when each participants responded to each task. Two observers were responsible for taking additional notes. Please see a complete usability protocol here. Due to the NDA, I cannot show the findings publicly. However, I would be happy to share the details of how we conducted those usability tests remotely with a working prototype and things I have learned. Please feel free to get in touch!

Limitations for usability test

There are some shortcomings with our method. First of all, since users were not able to interact with the prototype directly, it is very possible that our moderators accidentally showed some subtle cues while navigating the screens for the users. Because of this, users may have been directed to complete the tasks due to those visual cues rather than their own perceptions. Moreover, when the moderator clicked on the elements following users’ direct instructions, if the interaction was not yet being prototyped, the hotspots that appeared in other place of the page may have affected users’ perceptions of how they wanted to proceed with the tasks. Additionally, the users we recruited may not have been representative of the wider variety of online shoppers in the real-world. Demographically, the five users recruited were of a limited age range (22-25 years old), and there were significantly more females than males.

In the future, we would like to conduct more usability tests in person with a larger and more diverse sample of participants to generate more findings and minimize our moderator bias. In addition, we look forward to seeing a more complete prototype in the future so we can test more user tasks and scenarios.

Project Take Aways

This project provided me with the opportunity to work with Oracle’s cloud-based platform to examine the customer experience of an e-commerce site. I gained deeper understanding on when to use which research methods and how to apply them based on research questions we had during each step. I was able to use research methods such as interaction mapping, interviews, comparative analysis, surveys, heuristic evaluation and usability tests. Those methods helped us better triangulate our findings based on our research objectives. Having the opportunity to write different parts of the research reports such as Summary, Introduction, Methods, Findings, Recommendations and Future Discussions has also helped improve my writing and synthesis skills dramatically. Lastly, we have a very supportive and engaged client from Oracle and serving as a communication lead for this project also helped me realize that timely and proactive communication with the client will help move our team project forward.