When I first came to NWEA it was as the only designer on the student experience from login to the final screen of the assessment. My design leadership role on this team required a designer with a background in cognitive psychology and cognitive development as well as a strong understanding of usability basics, since the target audience were children between the ages of 4 and 18. I knew we would need a dedicated team to scale up our research efforts in order to keep up with the industry. Within a year there were three designers in this team and a dedicated User Experience Researcher.
Designing a Testing Process to Isolate Usability
In an assessment, students are contending with three separate issues: the content that tests their knowledge, the interaction patterns used to ask the questions, and the social-emotional strains of the school environment. This environment presents challenges to the typical usability testing process.
Challenge
We couldn’t ensure that our interaction patterns (our UI) were not impacting students’ ability to answer the academic questions posed to them. If our UI was impacting their ability to answer the question, then our data was compromised as it was not a true reflection of that student’s learning. In order to confirm our interaction patterns were not a variable making its way into the assessment data, we needed to remove those other two variables (school environment and academic content) and then reintroduce them one at a time.
Solution
I designed an item testing process that would ensure that our interaction patterns were separated from academic content and the testing environment. Each phase of the testing process gives us different, valuable data on our interaction patterns when married with the other variables. We were able to test the first phase in a mall. We chose the mall because it was an environment that wasn’t academic, but yet still distracting. Our content for this first phase was non-academic. For instance, if the interaction pattern we were testing was drag and drop, we had students drag apples to a bucket or some other behavior that didn’t require an understanding of on-grade content.
