A quick overview - in usability testing, testers are chosen, given an introduction to the software, and observed while they try to get stuff done with your product. This ideally involves two people aside from the tester - the "greeter", who interacts with the user/tester and guides them through the process, and an observer who focuses completely on watching the user at work.
Usually, two types of measures should be taken: numerical (number of errors, amount of time taken to find functionality, etc.), and qualitative (the user's attitude, comments - all the things you didn't think to measure in advance).
Quantitative measures are very important because then comparisons can be made - do users find functionality more easily/complete tasks faster with the "Edit" button hidden? By how much? How much longer do they take (if at all) to find that functionality when the button is hidden?
In sum, a usability study should be systematic - more like a psychology experiment or ethnography study than a product demo.
Usability testing should involve more than just "time taken for a new user to become familiar with the interface". Experienced users also need to be observed to ensure that prolonged use also shows desirable usability results.
Pop-ups providing first use advice to users may improve usability results for new users. For experienced users, pop-ups are likely to decrease usability.
Placing many buttons on a screen will likely make it harder for new users to start using the interface. They will have to investigate many more options amongst the clutter, leading to more decision making ("what is this button? do I click it?") and a corresponding decrease in usability.
In the case of GitHub, the question of whether new and/or experienced users regularly need to use the "Edit" button needs to be investigated. The cost of making an interface control highly visible to new users may exceed the cost of experienced users needing to search for help on how to perform the rare action.
No dispute with you there - I mentioned some possible metrics, but those you mentioned are also clearly important.
I was commenting on the question about usability testing (which yes, should include testing of experienced users); I don't have the information to make a judgment on the Edit button; I presume Google and Github both saw something in their testing that led them to make their design decisions. However, I am interested to know, in Google's case, what improvement, and what problems, they saw from this particular change. Anyone work on the new Google Contacts?
Comments
See http://en.wikipedia.org/wiki/Usability_testing for a description.
A quick overview - in usability testing, testers are chosen, given an introduction to the software, and observed while they try to get stuff done with your product. This ideally involves two people aside from the tester - the "greeter", who interacts with the user/tester and guides them through the process, and an observer who focuses completely on watching the user at work.
Usually, two types of measures should be taken: numerical (number of errors, amount of time taken to find functionality, etc.), and qualitative (the user's attitude, comments - all the things you didn't think to measure in advance).
Quantitative measures are very important because then comparisons can be made - do users find functionality more easily/complete tasks faster with the "Edit" button hidden? By how much? How much longer do they take (if at all) to find that functionality when the button is hidden?
In sum, a usability study should be systematic - more like a psychology experiment or ethnography study than a product demo.
Usability testing should involve more than just "time taken for a new user to become familiar with the interface". Experienced users also need to be observed to ensure that prolonged use also shows desirable usability results.
Pop-ups providing first use advice to users may improve usability results for new users. For experienced users, pop-ups are likely to decrease usability.
Placing many buttons on a screen will likely make it harder for new users to start using the interface. They will have to investigate many more options amongst the clutter, leading to more decision making ("what is this button? do I click it?") and a corresponding decrease in usability.
In the case of GitHub, the question of whether new and/or experienced users regularly need to use the "Edit" button needs to be investigated. The cost of making an interface control highly visible to new users may exceed the cost of experienced users needing to search for help on how to perform the rare action.
No dispute with you there - I mentioned some possible metrics, but those you mentioned are also clearly important.
I was commenting on the question about usability testing (which yes, should include testing of experienced users); I don't have the information to make a judgment on the Edit button; I presume Google and Github both saw something in their testing that led them to make their design decisions. However, I am interested to know, in Google's case, what improvement, and what problems, they saw from this particular change. Anyone work on the new Google Contacts?