Full list of contents

Foreword
Copyright
Dedication
Acknowledgements
Abstract

1 Introduction and study context

1.1 Overview (thesis chapter guide)

1.2 The apparent problem area
1.2.1 The problem in general
1.2.2 Automation
1.2.3 Particular problems
1.2.3.1 Maritime collisions
1.2.3.2 Nuclear power plants
1.2.3.3 The study of errors

1.3 Placing this study in its general context
1.3.1 Control tasks
1.3.2 Complexity
1.3.3 Cognitive aspects
1.3.4 Modelling

1.4 Structural outline
Figure 1.1: One possible view of the structure of the thesis
— A note on terminology used in this study

2 Mental models and cognitive task analysis literature

2.1 An outline of the complexity of the literature
2.1.1 Introduction
— A note on some of the terms in the literature
2.1.2 Mental models and (cognitive) task analysis
2.1.3 Formalisable models
2.1.4 Engineering and operator models
2.1.5 More general mental models
2.1.6 Models used in training or learning
2.1.7 Models derived from real users or operators
2.1.8 Expert systems for modelling operators
2.1.9 Models of error-prone human operators
2.1.10 Ragged edges

2.2 Classifying mental models and their literature
2.2.1 The owner and the object of a model
2.2.2 The purpose of a model

2.3 A more detailed review of exemplary literature
2.3.1 Decomposition formalisms
2.3.1.1 The GOMS family of models
— The Keystroke-Level Model (KLM)
2.3.1.2 Command Language Grammar (CLG)
2.3.1.3 Cognitive Complexity Theory
2.3.1.4 Task-Action Grammars (TAG)
2.3.1.5 General points about formalisms
2.3.2 Models of cognition
2.3.2.1 The Model Human Processor (MHP)
2.3.2.2 Programmable User Models (PUMs)
2.3.2.3 ACT*
2.3.2.4 Interacting Cognitive Subsystems
2.3.2.5 General points about models of cognition
2.3.3 Important features of cognition in complex systems
2.3.3.1 Individuality
2.3.3.2 Skills, rules and knowledge
2.3.3.3 Mapping cognitive demands
2.3.3.4 Modelling the operator's view of the structure of a system
2.3.3.5 Qualitative models and reasoning
— Non-monotonic reasoning
2.3.3.6 General points about important features of cognition

2.4 Reviews and criticisms of the literature
2.4.1 A “trade-off” analysis of cognitive models
2.4.2 “Literal models” promise more than they can deliver
2.4.3 They are not yet of practical use to designers
2.4.4 Why formal models are not very useful
2.4.5 “Design and evaluation techniques” are not used anyway
2.4.6 Failure to appreciate variety of possible solutions
2.4.7 Problems of validation without theoretical foundations
2.4.8 Ease of formal analysis suggests simplicity of the system
2.4.9 The expert is a learner at the edges

2.5 Where does the literature point to?
2.5.1 What do authors consider desirable?
2.5.2 Needs

2.6 Representation in mental models literature
2.6.1 Difficulties with formal techniques
2.6.2 Consistency in HCI design
2.6.3 Representation in complex tasks
2.6.4 Representation in context

3 Early studies

3.1 Maritime collision avoidance
3.1.1 The nature of collision avoidance
3.1.2 Representations in collision avoidance
3.1.2.1 The literature on possible revisions to the collision regulations
3.1.2.2 The evidence of differences between individual watchkeeping officers
3.1.2.3 Work on collision avoidance advice systems
3.1.2.4 Undocumented considerations
3.1.3 Difficulty in collection of data
3.1.4 Difficulty in simulation

3.2 Dynamic control and machine learning
Figure 3.1: The pole and cart, or inverted pendulum
3.2.1 Fundamental ideas in dynamic control
3.2.2 The BOXES approach to pole-balancing
3.2.2.1 Chambers & Michie's ideas for cooperation
3.2.2.2 Recent work with the pole-and-cart system
3.2.3 Representation in machine learning generally
3.2.4 Commentary on relevance

4 The Simple Unstable Vehicle : a manual control task

4.1 Testing control rules for the SUV
4.1.1 Description of the operation of the test program
4.1.2 Results and discussion

4.2 Aims of the human control experiment

4.3 Method and results
4.3.1 The interface
4.3.2 Collection of data
4.3.3 Processing of data
4.3.3.1 Control as handlebar angle setting
4.3.3.2 Control as setting the rate of handlebar movement
4.3.4 Comparison with hand-written control rules

4.4 Discussion
4.4.1 Problems in the experimental design
4.4.2 Manual control as a hindrance
4.4.3 Implications for this study

5 Non-manual control task selection

5.1 Criteria of suitability for study
5.1.1 The level of complexity of the target system and task
5.1.2 Level of control
5.1.3 Independence from psycho-motor limits
5.1.4 Realism, task definition and feedback
5.1.5 Adaptability of the task and the interface
5.1.6 Logging
5.1.7 Obtainability

5.2 Choice of experimental system
Table 5.1: Comparing experimental options against criteria
5.2.1 A nautical simulator
5.2.2 STEAMER
5.2.3 A toy flight simulator
5.2.4 Other existing computer games
5.2.5 A nuclear power plant simulation
5.2.6 Another nautical task simulation
5.2.7 Decision, and implementation implications

6 The Sea-Searching Simulation task and first experiment

6.1 Aims

6.2 The design and implementation of the task and interface
6.2.1 General implementation details
6.2.2 Description of the task
6.2.2.1 The task scenario
6.2.2.2 The simulation of the objects in the task
6.2.3 Description of the interface
Figure 6.1: The interface in the first sea-searching experiment
6.2.3.1 Implementation of the interface
6.2.3.2 Logging data and replaying runs
6.2.4 Portability

6.3 Methods and results
6.3.1 Game organisation
6.3.2 Subjects
6.3.3 Data collection
6.3.4 Analysis
6.3.4.1 Analysis of the actions
6.3.4.2 Expanding the trace files
6.3.4.3 Effecting the action changes
6.3.4.4 Selection of the desired attributes
6.3.4.5 Evaluating representation primitives using rule induction
6.3.4.6 Evidence for development of rules, and representational effects

6.4 Further discussion
6.4.1 Uncertainty in scores
6.4.2 Types of action
6.4.3 Evaluating the information provided by an interface
6.4.4 Other difficulties with representation
6.4.5 Limited nature of interesting results
6.4.6 Need for further experiments

7 Sea-Searching Simulation task : second experiment

7.1 Experimental methodology and the implementation of changes
7.1.1 Costing the information
7.1.2 Rearranging the ROV turn controls
Figure 7.1: The interface in the second sea-searching experiment
7.1.3 Introducing weather
7.1.4 New arrangements for subjects
7.1.5 Other changes

7.2 Analytic methods and results
7.2.1 Analysis of sensor usage
7.2.2 The idea of context applied to this analysis
7.2.3 Analytic approach
7.2.4 Analysis structure
7.2.4.1 Finding context structure
7.2.4.2 Using context structure in the remaining analysis
7.2.5 Analysis of data from subject AJ
7.2.6 Analysis of data from subject MT
7.2.7 Deriving rules for contexts
7.2.8 Further analysis of the ROV data
7.2.9 Verbal reports of task performance
7.2.9.1 Distinguishing contexts where there is no difference in sensor usage
7.2.9.2 High-level concepts in ship searching
7.2.9.3 Using information from a combination of sensors
7.2.9.4 Verbal reports of context structure
7.2.9.5 Conscious changes in strategy or tactics
7.2.9.6 Other points

7.3 Discussion
7.3.1 Main findings of this experiment
7.3.2 Justification of results in terms of other work
7.3.3 Problems and direct remedies
7.3.4 Other possible direct extensions to the study

8 Overall interpretation of results, conclusions and directions

8.1 Conclusions on human representations of complex systems
8.1.1 Collected salient important findings
8.1.2 Variation between individuals and situations
8.1.3 What is modelled?
8.1.4 Generalising the methodology
8.1.4.1 Removal of information hiding
8.1.4.2 Removal of restriction on interaction timing
8.1.4.3 Including analogue control inputs
8.1.4.4 Finding new representational primitives
8.1.5 Conjectures about contexts
8.1.5.1 The articulation of contexts
8.1.5.2 The development of contexts
8.1.5.3 Types of context

8.2 Further implications for systems design, decision aids, and training
8.2.1 Preconditions for applying the methodology
8.2.2 Interface redesign
8.2.3 Safety
8.2.4 The Guardian Angel support paradigm
8.2.5 Training and assessment
8.2.6 Early design

8.3 Still further work
8.3.1 Recreating context structure without explicit data on information usage
8.3.2 Further refinements of the context structure
8.3.2.1 Refining the quantities into qualitative ranges
8.3.2.2 Re-examination of actions
8.3.3 Directions for machine learning
8.3.4 Prospects for contributing to the study of human learning

A The help content in the second sea-searching experiment

How to Use the help screens
Beginners' Introduction
Scenario
How it is done
How to start learning
Game Object
Scoring
Purpose
Design
Interface Principles
General Display
Click Response
Sea Bed
Targets
Ship
ROV
Cable
Start, Stop and Replay
Starting a new run
Replaying an old run
Stopping
Glossary

B A small case study of differing representations (overview of appendix)

B.1 First analysis of ROV turn actions with CN2
B.2 Rework using the ordered mode in CN2

C Analysis of concurrent sub-tasks in ROV control

References