Defining Performance In Computer Systems Computer Science

a. i. Performance can be defined as the speed and efficiency with which a task is accomplished using the user interface. This is usually measured with respect to system specifications or compared to other systems.

ii. Accessibility can be taken to mean the extent to which the user interface enables users with disabilities to perform tasks - usually compared with non-disabled users.

b. You would use the specifications for the system to determine the tasks required of the user interface and any specified time requirements/restrictions. In addition, any comparative systems either in the specifications or otherwise can be identified.

The main metric will be time to completion of tasks using the prototype interface, while other metrics would include number and types of errors, error recovery and use of help systems.

A protocol would be designed which includes an appropriate number and range of tasks to cover key required functionality. The tasks should evaluate all forms of interaction in the interface, including any buttons or touch screen elements.

The evaluation would then be carried out with a representative sample of users, recording times and errors as above. Results would be compared with the specification or with comparative systems tested in the same way.

c. Usually accessibility issues would be considered throughout the design and development of the product, and should be written into the specification. Any evaluation should ensure that the user interface meets the specification. Any legislative or regulatory accessibility requirements applicable to the location the machines will be used in must be researched and compliance with these included in the evaluation.

Evaluation by users with a range would commonly be used, but users with or specific disabilities may be required. Such users are commonly recruited through disability support organisations, students with disabilities, local rehabilitation or disability services departments, seniors organisations centres and independent living organizations.

A suitable protocol would be designed which includes an appropriate number and range of tasks to cover key required functionality and the issues faced by disabled users. This must include evaluation of all forms of interaction in the interface such as buttons and touch screen elements.

When carrying out the evaluation, specific data regarding the users’ disabilities should be gathered. The location for testing - i.e. lab or field must be physically accessible, and any assistive technology required such as magnifiers must be available.

Pilot tests should be used to determine if extra time is required for the evaluation and evaluators must be aware that fatigue is a common problem in disabilities particularly when using assistive technology and therefore breaks and support may be required.

Some devices are available that attempt to imitate the difficulties of disabled users, such as gloves to make fingers clumsy and spectacles to simulate particular visual problems, but these are often not realistic evaluation tools.

This question required candidates to apply their knowledge of usability and evaluation to a specific scenario and a particular group of users. Answers had to address the issues raised by dealing with a wide-ranging and very varied group of users, and the issues of accessibility when doing standard usability evaluations.

Candidates were required to demonstrate that they understood the basic metrics used in usability evaluation. Given that proof of accessibility is a legal requirement in the development of systems in most countries, candidates were expected to show how usability evaluations are extended to address accessibility issues.

Many candidates did not address the question on evaluation of accessibility as asked, and instead of answering in the context of the question, simply gave answers reflecting very general usability evaluation theory and methodology.

2.3 You are hired as an HCI expert to undertake a user-centred design of the user interface for a touch screen kiosk displaying local hospital information for patients and carers. In the course of this task, you need to obtain user requirements from a group of users and test the design with the group of users.

a) Describe the criteria you will use to select your user group.

b) You have a choice between 2 methods to obtain the initial user requirements: focus groups or postal questionnaires. Describe the advantages and disadvantages of each of these methods.

c) Describe the advantages and disadvantages of using icons in this user interface.

d) Describe a suitable design for the usability evaluation of the icons for the user interface.

Answer Pointers

a. Ideally genuine patients and carers should be used. The user sample should be as broad as possible in terms of: age, different departments of the hospital etc. The ratio of sexes should be balanced. Patients with disabilities should be represented.

b. Focus groups:

Advantages:

- direct contact with users

- can pursue supplementary/additional information or details

- no delay in obtaining data.

Disadvantages:

- requires specialist skill

- usually personnel intensive

- expensive

- cannot always control format of responses

- large samples usually difficult and expensive.

Postal questionnaires:

Advantages:

- relatively cheap

- tight control of questions and format of response

- can easily be run by single person

- can use large sample.

Disadvantages:

- no direct contact with users

- cannot easily clarify details/ask supplementary questions

- return rates often poor

- skilled design needed

- delay in obtaining data.

c. Advantages:

- helps overcome language barriers

- helps with poor literacy

- can help with visual impairment

- if correctly done can speed up navigation

- enables of colour coding/categorisation

- recognition on subsequent use improves usability.

Disadvantages:

- can be confusing for some users

- can be difficult to design appropriate icons.

d. Prototype icons with a variety of different symbols, images and colour representations would usually be created. A group of users should be used for the evaluation and they can be presented with the prototype icons on paper or on screen. Typically they would be asked what the icons represented to them, or to choose between options to get the most representative icons. The results would be analysed to identify an appropriate and usable set of icons. These can then be retested as part of the prototype interface testing.

The answers to the question on the advantages and disadvantages of icons were mixed, with a number of candidates describing what icons are, and how they are used rather than addressing the question asked.

Answers to the question asking for a design of a usability evaluation of icons was not well answered, with most candidates merely restating basic, general usability theory and methodology rather than applying it to the specific context of icons as asked.

2.4 The company you work for is currently engaged in designing and developing a suite of small educational games to support children aged 5-7 years old in learning basic mathematical skills such as counting and simple addition and subtraction. The games will be distributed via CD-ROM and via a website. They are intended to be used both in school classrooms as well as at home.

You have been given the task of identifying potential users of the games.

a) You work for a small games design company and have been given the task of writing for the company website a description of how your company uses each of the following three techniques and/or products in its design activities:

i) Storyboard

ii) Moodboard

iii) Wireframes

b)In your description of each, include a brief summary of the main characteristics of the technique, its uses and the point at which it is used in the design process. Use simple illustrations where appropriate.

i) List three potential USER GROUPS that should be considered when designing the games.

ii) Create two USER PERSONAS (each from a different user group identified in i. above) that may be used in the design process. Limit your personas to 100 words for each.

iii) Create two realistic SCENARIOS each involving one of the personas identified in ii. above. Limit your scenarios to 50 words each.

Answer Pointersai. Storyboard

Eg. A storyboard is a series of drawings or images that represents how an interface would be used to accomplish a particular task. Cf flowchart. Lo-fi prototype used in early development stages.

(0-4 marks according to appropriateness of answer)

aii. Moodboard

Eg. A mood board is a type of poster design that may consist of images, text, and samples of objects in a composition of the choice of the mood board creator. Designers and others use mood boards to develop their design concepts and to communicate to other members of the design team. Used to visually explain a certain style of writing, or an imaginary setting for an artefact such as a game. Lo-fi prototype, used very early in the design process.

(0-4 marks according to appropriateness of answer)

aiii. Wireframes

Eg. Defines a layout, typically of a web page or other interactive artefact that shows what content goes where, basic layout, navigation and interaction. The amount of content included may vary, according to stage of the design process a wireframe is used. Can be paper-based or developed using software tools.

(0-4 marks according to appropriateness of answer)

3.2 a) Explain how paper prototyping and walkthroughs can be used to evaluate a

design.[p8]

b) Using a walkthrough, evaluate the design shown below and offer your design

team some feedback on any potential weaknesses along with some sketches

to show possible improvements in usability.[p8]

3.3 Briefly outline the reasons why prototyping is used in industry to develop user

interfaces. [p9]

bi. User groups

Potential user groups for this game include:

children aged 5-7

School teachers

Parents/carers

(1 mark for each)

bii. User personas

Two detailed personas one from each user group eg child user, a school teacher user, a parent. Marks awarded for the richness of the personas created and demonstration of understanding, through the personas created, of their role in the design process.

(0-5 marks according to appropriateness of answers)

biii. Scenarios

Two detailed and realistic scenarios involving the characters indentified by the personas above. Eg Teacher wants to view the performances of all the children in her class, sort them and identify areas of individual weakness.

(0-5 marks according to appropriateness of answers)

Examiners’ Comments

This question was attempted by only one-third of candidates and it had the second lowest pass rate. In principle, this question is straightforward and the multipart structure should have given candidates an opportunity to score well. However, with notable exceptions, candidates were not sufficiently familiar with the range of techniques covered in the question and were therefore unable to pick up marks throughout the sections. The techniques covered in this question are well-known, often-used tools of the interaction designer. Answers from candidates indicated a lack of knowledge of these techniques, particularly wire-frames, scenarios and personas. This in turn suggests a lack of understanding and practical application of the contemporary methods employed by designers.

Article name: Defining Performance In Computer Systems Computer Science essay, research paper, dissertation