Note: Because of non-disclosure agreements signed by me and the members of my team, some details, such as specific findings and recommendations, are not available for this project.
As part of Evaluation of Systems and Services course, my group performed a semester-long usability analysis of the Quicken Loans mobile website that was launched in November 2010. I mapped the existing site, performed initial user interviews, created personas and scenarios, performed several different analyses, performed user surveying and user testing, and offered findings and recommendations.
The Quicken Loans mobile site was launched in November 2010, so I was in a unique position to influence an important mobile product in its nascent phase. The initial version of the site was fairly simple with a collection of static informational pages and a simple mortgage calculator. Quicken Loans was interested in learning who their users were, what potential customers look for in a mobile site, and which specific features might entice them to use the site.
My first step was to map the current Quicken Loans mobile site and the interactions between all of the pages. I laid out the screenshots into a map which used color-coded boxes to show depth and complexity without cluttering the map with many arrows and lines.
Click the image above to see the full, scrollable version
In order to learn more about potential customers, my group interviewed 6 people who met the demographic data provided by Quicken Loans. In a series of group discussion and whiteboarding sessions after the interviews, I synthesized our interview notes into three personas, each with a corresponding scenario. The personas and scenarios were centered around the idea of Life, Experience, and End Goals, to provide a sharp focus on our specific project.
Click the image above to view all 3 full personas in a PDF
In addition to creating some of the personas, I also created the template we used to create all of the personas, based somewhat on recommendations provided by Dan M. Brown in his book Communicating Design. These personas and scenarios would be used for the rest of the semester to inform our findings and recommendations at every step.
After creating personas, I performed a comparative analysis of the Quicken Loans mobile site. My group started with a very large pool of sites which I contributed to a preliminary investigation of. After deciding on metrics to rate each site by, I helped pare down the large pool of sites into a smaller list of six sites to compare to Quicken Loans mobile.
Quicken Loans occupies a unique space in that it is currently the only loan site I could find that offers a mobile-optimized version. This meant that analyses of direct and partial competitors who offered similar loan services were not enough to generate good design suggestions. To this end, I came up with the idea of relying heavily on analogous sites: mobile-optimized sites providing information and a contact point about a product, but without any built-in sales element or advanced interactive functionality.
In writing the comparative analysis, I decided that a comparative rubric chart would be useful, and created one using the page layout features of Apple Pages.
Click the image above to view the chart at full size
After finishing the comparative analysis, my team and I performed a heuristic evaluation of the Quicken Loans mobile site. I used Jakob Nielsen's heuristics as a baseline, and then came up with a set of custom heuristics that were tailored to the Quicken Loans site. The entire site was evaluated according to these heuristics by each of my team members, and then the individual results were consolidated and collated into a set of recommendations.
In order to canvass a broad group of people on their experience and opinions about mobile sites and mortgage sites, I participated in the design of a survey, which asked respondents to rank the importance of certain features of mobile websites and mortgage websites, and also asked general preference and demographic information. By leveraging social media, list-servs, personal contacts, and link-sharing sites like Reddit and Digg, I was able to collect over four-hundred survey responses. These responses were then filtered by relevant demographic information, linked to the personas created earlier. The results were then collated and interpreted, and compared with findings from the various earlier analyses.
The last evaluation I performed was a hands-on usability test of the existing Quicken Loans site with users who matched the demographics of my personas. I developed sample tasks, making sure that relevant areas of the site (identified with earlier analyses) were being tested. From these tasks I developed an interview script, as well as pre- and post-test questionnaires built using Google Forms. Recruiting was a challenge, as our demographics showed that most users and potential users were middle-aged and relatively wealthy, and incentivizing these sorts of users was difficult, but my team managed to recruit and conduct tests on five individuals, all of which were videotaped and recorded by members of my team.
My team and I created a concise presentation which was given to our clients at Quicken Loans. The findings were organized into four main improvement areas: Navigation, Presentation, Design, and the Mortgage Calculator. Our presentation was well-received, and some of our recommendations, especially in the area of Design, can be seen on the Quicken Loans mobile site today.