Groperz.com Website Evaluation

Executive Summary

The following report describes the usability testing of Groperz.com, a web site that will serve as Everlast Climbing Industries' virtual presence on the World Wide Web. The purpose of this section is to provide a brief overview of our logistics, major findings, recommendations, and overall benefits of testing.

Test Logistics:

This was the first combined development and usability testing effort for all of the individuals assigned to this project. Therefore, a certain amount of flexibility and creative problem solving was employed by HCI 460 members to map incipient development information to testing methodology.

Major Findings:

The "learn by doing" learning style is characteristic to this user population. In other words, the users needed to be able to dive right in and accomplish their tasks with minimal struggle. Therefore, usability concerns are critical to the success of this site. We found that there was some significant confusion on how to order products and navigate through the site in general.

Recommendations:

Based on our findings, we recommend that the basic high-level design structure be simplified. This would be accomplished by having:

"Company info", and "about Groperz" should be place into the home page.

"Ordering", "products", and "build a wall" under one link called "Products and ordering". Or the group should provide better links between these categories to streamline users’ workflow.

Overall Benefits:

Because of the nature of Groperz's users, our tests have provided valuable design information that is necessary to the success of the purpose and objective of this effort to establish an effective virtual presence. Our Conclusions on the usability of Groperz site were established from quantitative and qualitative data, which can be found in the Appendices section of this report. The Methods section describes the nature of our tests, how they were set up, profiles of users, and how our data was collected. The Results, Findings & Recommendations section grew from our inspection of the raw data. following report as well as logistics specific to this project.

Method

The users of this site are climbers, people who are interested in climbing and people who are responsible for buying this level of athletic equipment. In an effort to understand our users as best we could, we conducted some research through an Athletic Director of a local college. We wanted to define the characteristics of climbers, since we figured it might be difficult to find enough people, with enough climbing experience willing to participate. We found that the climber's defining characteristics matched other groups of people involved in similar types of athletic activity and we used this profile in the recruiting and screening of perspective participants.

Personal History

Age: 20’s-40

Gender: both

Positive attitude toward computers & the Internet

Learning Style:

learn by doing

Educational History

Highest grades completed:

College and up

Subjects Studied:

varied, but must be athletic

Computer Experience

20% low 0-2 yrs

60% intermediate 2-5yrs

20% high 5+ yrs

Athletic Experience

20% semi active, >3 times a week

60% active, 3 times a week

20% very active, < 3 times a week

To recruit participants we contacted friends, acquaintances and co-workers via email and word of mouth. The athletic director previously mentioned, also assisted by sending us folks who fit our profile.

On November 1st, testing day eight minutes before our first test, we received a few new screen designs, which did not coordinate with the rest of the system. This new challenge ment:

Positive- Comparing both sets of screen designs allowed us to learn more about the 555's mindset

Positive- It initiated more in depth information from participants

Negative- It swung our assessment tests to include more exploratory-like activity

* Since participants were confused, they often pumped the monitor for info

Negative- Adjusting to this unexpected change threw-off the planned structure of our testing

* We struggled to maintain composure, an unforeseen challenge

Five people participated to test the usability of the Groperz site. They received no prerequisite training, because they were required to have at least a basic computer and web experience.

The test setup consisted of a low fidelity prototype with no support materials (e.g., documentation, quick reference) in an attempt to reflect the typical human-internet session. The test itself consisted of a series of tasks of varying complexity, typically preformed by Groperz's web user customer base.

Prior to the test, the monitor oriented the participants in the nature of their participation, and delivered each task one at a time. Two 460 personnel observed the participant, noting such items as errors of commission and omission, excessive time to search through the site, requests for additional information or other questions, points of confusion or hesitation, etc. The last test was videotaped on VHS from an overhead and 45 degree angles that revealed the buttons and keys that this participant pressed as well as verbal and non-verbal communication between the testers and participant.

After the test, the participants were removed from the testing area to be given a short break. They were then asked to complete a questionnaire, which collected subjective data about their performance and the site's design. The post test questionnaire explored their perception of general usability of the site. While the participants filled out the questionnaire, the 460 team reviewed their notes to ask the participant about unresolved errors or activities during the debriefing session.

After the participants completed the questionnaire, the monitor debriefed the participant, to clarify unresolved issues and answer any questions. Changes by the development team to some of the screens were incorporated in a "fast iteration" process, due to time constraints, to make the architecture diagram, and both sets of screens work together.

In the end, there were three cycles of iteration: 1 practice test, 1 cycle, 2 tests, 1 cycle, 2more tests, 1 cycle, 1 last test.

Results

Quantitative Data

Participant	Years Using Internet	Years Rock Climbing	Internet purchases	Number of tasks correct
P1	5	0	0	7 of 7
P2	2	0	2	6 of 7
P3	3	1	2	7 of 7
P4	3	0	0	5 of 7
P5	4	1	3	5 of 7
			0= Never
			5= Often

Task	Screens Tested	Average Time To Complete	Success Rate	Recovery Rate
1	Browser	4.8 seconds	100%	NA
2	Company Information	45 seconds	100%	NA
3	Company Information	25 seconds	100%	NA
4	Ordering	8 minutes 5 seconds	60%	100%
5	Build-A-Wall	6 minutes 36 seconds	60%	100%
6	Company Information	1 minute 28 seconds	60%	NA
7	Company Information	45 seconds	100%	NA

Qualitative Data

The following responses were gathered after the participants had completed the usability test.

User	Difficulty of Web Site	Rating of Web Site	Usability Test Rating
1	4	2	5
2	5	4	5
3	2	3	4
4	4	5	4
5	3	3	3
Average	3.6	3.4	4.2
	1= Hard	1= Dislike	1= Hard
	5= Easy	5=Like	5= Easy

Q: Describe what you would change in the design of the site?

1. Better presentation of information when connecting to different sections

2. Product and Ordering together

3. Better definition of Build-A-Wall

4. View shopping cart at any time

5. Display actual testimonies instead of downloading

6. show if ordering is secure or not

7. Can I subscribe to a catalog?

8. More visually creative

9. Streaming video

10. Get rid of “than you for ordering” screen

11. Show shopping cart in the tool bar w/selected items and prices

Additional Comments:

1. Quick search function- if I want to go to a certain products

2. Links to other “cool” climbing sites

3. I only want to log-in once, even as a first time user

4. Should make site more interesting, a place where a climber would given if they didn’t want to buy something

5. Loose the scroll down stuff

6. Better home page with embedded animation, like video clips of crazy climbing shots

7. It was very good and basic design, very easy to use

8. I would use this site again

Findings &Recommendations

Form the studies that we have conducted, we have concluded the Company Information, and Product Ordering and Build-A-Wall screen need some modifications prior to release. The following is a detailed description of the modifications and evidence supporting our claims.

.

Screen	Company Information
Recommendation	Insert the word “Fax” in front of the fax number
Findings	The telephone number has “Tel” directly in front of the number and the fax number doesn’t. One of our users was a little confused when they saw this and it was suggested the word “fax” be placed in front of the fax numbers.
Screen	Product Ordering
Recommendations	1) Create a “Check Out” button so the user can verify and submit their order 2) Move the “Build A Wall” link within the “Ordering” link this will maintain proper navigation 3) Utilize the same product description screens for the “Products” link and the “Ordering” link 4) Upon entering the new “product” link set default the view to the product groups (Mini, Footgrips, Poprock, jumbos etc..). 5) Clarify the process of selecting an item and specifying the quantity 6) Incorporate a “shopping cart” list so the user can keep track of what the ordered 7) You may want to incorporate the Login feature after all of the items have been selected and totaled. This will enable you to utilize the same ordering process in the “Build A Wall” module.
Findings	1) There is no “submit order” button 2) 40% of the users selected “Build A Wall” as opposed to “Ordering “ 3) 20% of the users entered the product screen thinking they would find pricing 4) 20 % of the users did not want to fill out the customer information form- 5) 60 % of the users were confused in how they selected the item to order it- 6) Users expressed some confusion with entering the Product screens at the top of the list
Screen	Build A Wall
Recommendations	1) Remove the start Building button or Input a confirmation screen after the user selects the type of wall they want 2) Utilize the same product selection and ordering features of the “Ordering” module 3) Incorporate a “Next Product” button rather than the scroll arrow
Findings	1) Users thought selecting “45 degree” would start the process rather than the “Start Building Button” 2) Confusion with how to select the products 3) Down Arrow scroll button not visible
Screen	Company Information
Recommendation	Use pictures and written testimonials for the users to view rather than Movie Download
Findings	1) 60% of users would not download plug-in-in to view testimonial 2) 80% of users would not wait more then 3 minutes for the file to download