The Role of Theory
Choosing Human-Computer Interaction (HCI) Appropriate Research Methods
The Role of Theory in HCI
A. Sasha Giacoppo (firstname.lastname@example.org)
Human-Computer Interaction (HCI) research is performed to provide and promote a scientific understanding of the interaction between humans and the computer technology and tools that we use. HCI research can be heavily steeped in the principles and controls of the scientific method. However, the methodologies of many non-scientific disciplines have also been adapted for use in HCI research and practice. The methodologies that HCI researchers select and use have been adapted from the research of other scientific fields, many of which come from the psychological sciences.
Many of the principles that have driven the differing canons of HCI (e.g. user interface design, information visualization, etc) grew from the scientific research and theories from psychology. The varying domains of psychology are interdisciplinary, and quite often, research spans into other disciplines. Fields such as mathematics, design, architecture, arts, linguistics, etc. all have relevant contributions to HCI. Findings from these various domains seek to explain the human-computer interaction in terms of the psychological mechanisms of the user.
The domains of psychology that are considered a component of HCI are described here, as well as the most influential theories, models, and research of those areas. In addition, the major theories and models that have come from HCI research are described. They illustrate the breadth and depth of HCI research, and will help you make decisions regarding the direction of your investigations.
Ultimately, generating theories from your own research will provide the greater HCI community with valuable knowledge. Theories and models that come about from your research will be one of three kinds:
Explanatory theories seek to explain the behavior of our world; they tend to provide a more conceptual model of the world. Predictive theories seek to predict outcomes based on the changing values of component variables. Predictive theories provide an extremely high level of utility to HCI, they allow designers to directly predict the outcomes of their designs on user performance variables. Generative theories generate guidelines and principles that provide useful and applicable knowledge and models. The type of theory a researcher intends to produce guides the selection of appropriate research methodologies.
2.1 Foundations of HCI
2.1.1 Cognitive Psychology
Simply put, cognitive psychology is the study of how people think and learn. The goal of cognitive psychology is to understand the psychological processes involved in the acquisition and use of knowledge by people. This includes domains such as perception, attention, memory, learning, thinking, and the importance of social and environmental influences on those domains. Cognitive psychology is a major contributor to HCI research by providing and applying psychological principles to understand and help develop models that explain and predict human performance.
22.214.171.124 Information processing model
The information processing model, developed by Lindsay & Norman (1977), characterizes humans as information processors. The model explains the movement of information from input to output within a human, via a series of processing stages. The stages of processing are encoding, comparison, response selection, and response execution.
Figure 1. Information Processing Model
Later, the basic model was extended (Barber, 1988) to include the processes of attention and memory, which interact with information processing stages of processing.
Figure 2: Extended Information Processing Model
The information processing model has been a major influence in HCI by categorizing user behavior, and in turn, being able to predict user performance.
Provides a good overview of the information processing model, as well as the models relation to attention and memory.
A great website where the user can go through a slideshow that explains the various components of the information processing model.
126.96.36.199 Hackers Action Theory (Hacker, 1973, 1978,1985,1986)
Action theory explains the determinants, processes, and consequences of work behavior. It explains human activity in the context of goal directed behavior, and that people tend to maximize the efficiency of their actions. The main components of Action Theory are acts, actions, and operations.
Action Theory deconstructs the process of translating an intention into an action. By doing so, it helps researchers investigate the components of an activity, beginning at the formation of a users goals, passing through the user deciding on and selecting the appropriate methods/tools to achieve that goal, and ending at execution, and iterative refinement, of those methods and actions.
Users constantly change, alter, vary the actions used to achieve a goal. Rasmussens model of cognitive regulation (Rasmussen, 1982,1983, 1986), which supports Action Theory, provides a detailed description of the information processes that occur at three-levels of action selection. The information processes of the users regulate the types of actions selected. The highest level is the knowledge-based level, which involves the development and deployment of strategies and heuristics to achieve goals. Problem solving processes typify this level, where a users analysis of stimuli from the environment is used to choose an optimum strategy, mediated by the users model about the potential outcomes of available strategies. The next-lowest level is the rule-based level, where actions are controlled by rules or procedures stored earlier (Izso, 2001). Incoming stimuli are consciously organized by the user and trigger certain actions when the criteria for those actions are met. The lowest level of action regulation occurs at the skill-based level, where actions are executed as a result of a trigger not conscious to the user. Sensory motor reactions represent this level.
Rasmussens theory of cognitive regulation is important to HCI because it describes and categorizes user activity and behavior. This allows researchers to understand the cognitive processes that are really occurring, at different levels of interaction, within a user during use/interaction of a computer/interface.
188.8.131.52 Andersens ACT-R
The Adaptive Character of Thought Rational (ACT-R) theory describes a cognitive architecture, based on how cognitive skills are realized by production rules. It is about how humans/users acquire skills through the gradual acquisition and integration of production rules, which are if-then pairs. The combination of multiple production rules can be applied to wide variety of complex problems. The theory describes how knowledge from production rules begins as more skill-based, declarative knowledge and transforms into more problem-solving, conceptual, procedural knowledge.
The ACT-R provides HCI researchers with an understanding of how knowledge is built, and how it transforms. Application of ACT-R in designing systems and interfaces that properly match and enhance the users acquisition of knowledge and development of skills can provide very powerful products.
Similar to the ACT-R is Normans Stages of Action Model (1988), which provides a list of the stages that users go through in trying to use a system:
This model deconstructs user behavior and activity as well, however, it provides a much more practical guide to user behavior. This explanatory model can also guide an HCI researcher when deciding on specific components of an interface to investigate.
A comprehensive site for everything about the ACT-R, including contemporary extensions to the model.
Provides a good overview of Normans model, as well as some graphics that help users understand the model better.
184.108.40.206 Knowledge and Mental Models
Research into how humans construct information into knowledge also has major contributions to HCI. Understanding how knowledge is represented and organized is critical in the design and development of technology and tools. The applications of research findings intend to maximize the amount of knowledge and utility gained from interfaces.
Much research in knowledge representation, with relation to HCI, has focused on mental models. Mental models are dynamic constructions of knowledge that guide us (and sometimes misguide us!) in understanding and acting within our own worlds. Donald Norman (1998) provides a good definition of mental models:
the model people have of themselves, others, the environment, and the thing with which they interact. People form mental models through experience, training, and instruction
An excellent beginners explanation of mental models.
Claiming to be a gentle introduction to mental models, it is anything but. However, the website does provide an excellent example of how to understand what a mental model is.
220.127.116.11 Attention & Memory
Research on the attention & memory of humans investigates the limits of human information processing, including the limits and structures of storing memory. This domain has helped HCI researchers define the limits of what an interface can provide, and has helped developed guidelines regarding what can and should be presented in an interface. Heavy research, in areas like focused/divided attention (e.g. Baddeley, 1986, 1990), or supervisory attention (Norman, D.A., & Shallice, T. ,1980), still continue to produce major findings that can be of great use with regards to HCI issues.
2.1.2 Perceptual Psychology
Perceptual psychology studies how people acquire information from their environment, through their senses, and how that knowledge is then encoded for use in future processes. Perceptual psychology, by nature, has been a major contributor to HCI research because it investigates how people acquire information from graphical representations, and interfaces.
Within the arena of perceptual psychology, there are two opposing paradigms of theorists, the constructivists, and the ecologists (sometimes called Gibsonians). Constructivist theories take the position that seeing is an active process in which perception is constructed from the environmental information before us AND from previously stored knowledge (e.g. Gregory, 1970). Ecological theorists (e.g. Gibson, 1979) take the position that perception of the environment can be solely derived from the environment itself, and does not require construction or previous knowledge on the behalf of the human.
Regardless of which camp you agree with, the theories are important when it comes to HCI. An excellent example of a perceptual theory are the Gestalt laws of perceptual organization, which are considered constructivist. Gestalt psychologists (e.g. Koffka, 1935; Kohler, 1947) believed that the interpretation (read: perception) of patterns of stimuli is based on innate laws of organization. In other words, humans are wired to perceive organizations of information to make sense. Gestalt laws, though perhaps antiquated, provide an excellent model for presenting information at an interface.
An entertaining site that explains some common illusions in the context of perceptual psychology, including a description and comparison of the major paradigms of perception.
2.1.3 Social Psychology
Social psychology studies how people behave in the context of their interactions with others. Social psychology is important to HCI research because it highlights the importance of social interactions when people are involved in tasks. The development of the internet into an online community is an important concentration of social psychology research. Understanding the new forms of social interaction and behaviors that occur within online communities will be critical to the design and development of tools and technologies. The theories and models of human behavior that come from social psychology promote researchers to investigate not only users by themselves, but how they operate within their natural, social environments.
2.2 HCI Research
By incorporating the theories and models from the various domains of psychology, HCI researchers have been able to investigate more specific and applied issues related to the human-computer interaction. HCI theories and models can be explanatory, but most tend to provide guidelines and principles for design and development of interfaces. Below is a list of the major areas of HCI research, their descriptions, as well as examples of theories and models.
2.2.1 Design Theories
Design theories help guide the design of interfaces and the selection of components contained within interfaces. Provided is a list of the areas within interface design that have been subject to investigation and research.
18.104.22.168 GOMS (goals, operators, methods, selection rules) Model
Proposed by Card, Moran, and Newell (1980), the GOMS model deconstructs the activities of a user task into components of activity and the respective information processes. Users formulate goals (and subgoals), achieve those goals by using methods and procedures, via operators (e.g. move mouse), and use selection rules to choose appropriate methods and operators.
From this model, the keystroke-level predictive model was developed. It predicts performance time of tasks by calculating the sum of the lesser parts, which included time for keystrokes, pointing, thinking, waiting, etc. From this predictive model, designers of software were able to improve their products based on user performance at each and every level. For HCI researchers, the GOMS model provided a guide for areas that needed investigation.
A great comprehensive site that takes through GOMS from start to finish, including examples, and slide shows.
Though it requires access to the Association for Computing Machinerys (ACM) digital library, this link provides a great explanation of the GOMS model. At the very least, it provides an excellent, linked page of references to numerous additional GOMS research articles.
22.214.171.124 Menu Design
The organization and selection of information is another area of HCI investigation. Menu structures are an important component of interfaces, and the appropriate menus can greatly enhance or detract from the users experience and efficiency. The combination of the variety of menus (e.g. between scrolling, two-dimensional, alphasliders, pop-up menus), menu phrasing, presentation sequence, etc. offer many trade-offs between speed, efficiency, and time.
An excellent example of how research within HCI can produce useful predictive models is from Landauer and Nachbar(1985). Landauer and Nachbars predictive equation for selection time within menus is an excellent example of how research within the various fields of HCI can provide useful, empirical models in determining the effectiveness of differing designs and prototypes.. By varying the number of items per menu level to reach 4096 target items, the investigators were able to develop a predictive model for selection time.
T = k + c^log b
T = time to achieve selection
This link provides a nice table of the advantages and disadvantages of the various menu options.
126.96.36.199. Input devices
Much research in HCI has been dedicated to predicting the effectiveness of input devices. Various input devices (e.g. mice, joysticks, trackballs) have investigated on measures of ease of use, accuracy, and speed. Determining the performance of these devices has resulted in the development of motoric theories.
A landmark theory is Fitts Law, an important predictive equation for pointing time, developed by Paul Fitts (1954). Fitts Law predicts the amount of time to move a given distance to a target of specific width. It predicts that pointing time increases as a target is further away and smaller targets take longer to point to.
188.8.131.52. User Control/ Direct Manipulation
The ability to enable a user with control over the tool/interface is an essential component of the HCI interaction. Rather than leaving users with a feeling that they are slaves to a technology, users should have feelings of control and mastery over an interface. Shneiderman (1983) explained the central ideas of user control, which include:
A good explanation of direct manipulation.
A great collection of some papers regarding design theory, all published from research at the University of Maryland Human Computer Interaction Lab (HCIL)
184.108.40.206 Anthropomorphic design
Designing technology to act and behave like humans is another debate of HCI. Many people feel that computers should be more human like, which is generally a naïve interpretation of making technology user-friendly. Shneiderman (1998) describes several major problems of anthropomorphic design. Anthropomorphism projects false models of the extent of a computers usefulness, doesnt clarify that there ARE differences between humans and computers, and can injure people emotionally by making them feel dumb or inadequate when things dont work correctly. Quantitatively, anthropomorphic design can even have negative effects on performance (Quintanar, et al., 1982). The principles of designing proper interfaces and tools involves understanding user behavior, not making or portraying computers to act like humans.
Tools and technology are not limited to visual stimulus. Interfaces can also include sound, touch, gesture, tactile inputs and outputs. Multimodal interfaces are really about making the most efficient use of the human senses. Investigating the effects of mode regarding user performance and activity within an interface is extremely important. Understanding multimodality, especially as an interface component, requires investigating human information processing, perceptual psychology, and the attentional and working memory processes of users. The utility of multimodality extends beyond enhancing performance for the typical user by being able to provide equivalent access of technology for people with disabilities.
This site provides a good picture of the types of research and applications that is involved with multi-modal interfaces.
2.3 Additional Domains of HCI Research
2.3.1 Information Visualization
Information Visualization is the use of computer-supported interactive visual representations of abstract data to amplify cognition (Xerox: Palo Alto Research Lab (PARC)). As opposed to scientific information, which is generally a representation of physical information, information visualization techniques are used to present more abstract information in a much more effective manner. Techniques of information visualization help interface designers present data to users in meaningful and resourceful ways. These techniques, like dynamic queries (Shneiderman, 1994) and the fisheye lens (Furnas, 1981), illustrate how to successfully present huge amounts of data (read: the internet), in confined spaces, like the internet. These techniques are based upon underlying theories of vision, cognition, and perception
A comprehensive site about information visualization, including lists and descriptions of the multitude of information visualization techniques.
Another comprehensive information visualization site, including numerous examples, techniques, and slide shows.
2.3.2. Ubiquitous & Pervasive computing
Ubiquitous computing is about the omni-presence of invisible technology in our environments. The term pervasive computing means that technology permeates our lives. It is the strong, emerging trend towards, described by the National Institute of Standards and Technology,
The goal is to make computing and technology simple to use, everywhere around us, accessible to people with minimal technical expertise, reliable and more intuitive. Computing has shifted from one computer, many users (e.g. mainframe computing) to one person, one computer (e.g. PCs) to the future of ubiquitous computing, where it will be one person, many computers.
Figure 3. The Growth of Ubiquitous Computing
Leaping ahead into a future world of technological ether is a daunting task. The task of anyone developing this world should be to make technology that disappears, a world where our lives are enhanced by technology that doesnt invade, but allows us to accomplish our tasks. This is a fundamental axiom of the human-computer interaction.
Ubiquitous and pervasive technology will not succeed if the interface between this technology and the human-user is not properly designed. It is the onus of researcher to understand what makes an interface user-friendly, and, in the case of ubiquitous and pervasive computing, it is paramount to understand the massive importance of the social and environmental impacts.
A good discussion of ubiquitous computing, as well as informative documents and slideshows.
2.3.2. Universal usability
Universal usability is an extremely wide, and under-developed area, of HCI research. It deals with providing equivalent access to technology and tools for people with impairments or disabilities.
The immensity of developing tools and interfaces that are usable by everyone is daunting, let alone designing tools that are specifically designed for people with disabilities. The onus is on the shoulder of the designer to use research, thoughtfulness at the very least, to understand how different users, especially people with disabilities, interact with an interface.
THE website to go to for anything you need answered about universal usability.
Examples of universal usability research projects performed by University of Maryland students.
2.3.4. Artificial intelligence (AI)
As the MIT Artificial Intelligence Laboratory puts it, AI seeks to understand the nature of intelligence & to engineer systems that exhibit intelligence. Research in artificial intelligence spans multiple disciplines, and can certainly delve into the metaphysical. AI seeks to define and understand intelligence, the deep, interweaving roles and importance of memory, emotion, knowledge, perception, etc, on intelligence, and the reasons and motivations to create intelligent systems. Products of HCI/AI research, and the theories that spring from AI, drive some of the most advanced technologies in areas like imaging, robotics, and language recognition.
Some pundits argue that the motivations of artificial intelligence to design and develop technologies that intend to mimic the processing of humans are misguided, because we should not be wasting our efforts and resources on designing computers to be human-like. Proponents believe that attempting to understand the complexities of the intelligent human via AI research can not only lead us to a greater understanding of ourselves, but can lead to advanced forms of computation and technology.
MITs Artificial Intelligence website. It describes the past, present, and future of artificial intelligence.
The SRI Artificial Intelligence Center website. Provides links that describe the major areas of research in AI
2.3.5 Supervisory control/Autonomous agents
An autonomous agent is a system situated within and a part of an environment that senses that environment and acts on it, over time, in pursuit of its own agenda and so as to effect what it senses in the future. (Franklin, S. & Graesser, A., 1996)
Autonomous agents represent technologies, usually applications, that make decisions for users based on use and intentions. Autonomous agents are really about predictive technologies. The ability for a computer to predict human behavior is a very complex task. Incorporating predictive technologies like autonomous agents into user-interfaces can have very serious drawbacks, not the least of which is a serious loss of user locus of control. It is important for researchers to investigate the true impacts of autonomous agents on user performance.
Figure 4. Hierarchy of Agents
This website provides a list of definitions and terms of autonomous agents, as well as some UI design principles.
This website describes the various types and applications of autonomous agents, as well as provide useful definitions.
2.4 Social and Cultural Theories
There are large amounts of information related to the human computer interaction that investigate the social and cultural aspects of technology use and design. A list of these domains, including some links, are provided.
Sociology and social psychological theories
Research regarding the Human-Computer Interaction will always have a foundation of work done in the fields of psychology. Understanding these foundations can help guide your HCI research interests, and help highlight the methodologies that have helped improve understanding of HCI issues in the past. Further research should focus on areas that need to be explained, and ultimately, provide predictive models for issues within those areas.
Exposing yourself to the wide range of research throughout the domains of HCI will help you envision the breadth of the applicability of your personal research. Understanding the interrelationships between disciplines, and the importance of research within those disciplines, will soon build an understanding of the importance of HCI research, and hopefully inspire you to utilize research as a tool for advancing the quality of your work.
Lindsay, P.H. and Norman, D.A. (1977). Human Information Processing: An Introduction to Psychology, 2nd edition. New York: Academic Press.
Andersen, J.R. (1990). The adaptive character of thought. Hillsdale, NJ: Lawrence Erlbaum Assoc..
Baddeley, A.D. (1990). Human memory: Theory and practice,. Oxford: Oxford University Press.
Baddeley, A.D. (1986). Working memory. Oxford: Clarendon Press.
Barber, P (1988). Applied Cognitive Psychology. London: Methuen.
Card, S.K., Moran, T.P., and Newell, A. (1980). The Keystroke-level model for user performance with interactive systems, Communications of the ACM, vol 23, 396-410.
Fitts, P.M. (1954). The information capacity of the human motor system in controlling amplitude of movement, Journal of Experimental Psychology, 47, 381-391.
Franklin, S. and Graesser, A., Institute for Intelligent Systems, University of Memphis
Furnas, G.W. (1981). The FISHEYE View: A New Look at Structured Files. Murray Hill, NJ: AT&T Bell Laboratories.
Gibson, J.J. (1979). The Ecological Approach to Visual Perception. Boston, MA: Houghton Mifflin.
Gregory, R.L. (1970). The Intelligent Eye. New York: McGraw-Hill.
Hacker, W. (1973). Arbeits und Ingenieurpsychologische Beitrage zur socialistischen Rationalisierung der Volkswirtschaft. In Hacker, W., Timpe, P., Vorwerg, M. (eds.) Arbeits-, ingenieur- und sozialpsychologische Beitrage zur sozialistischen Rationalisierung. VEB Verlag der Wissenschaften: Berlin.
Hacker, W. (1978). Allgemeine Arbeits und Ingenieurpsychologische. Psychische Struktur und Regulation von Arbeitstatigkeiten. Bern: Huber Verlag (2nd Ed.).
Hacker, W. (1985). Activity: a fruitful concept in industrial psychology. In: Frese, M., Sabini (eds.). Goal directed behavior: the concept of action psychology. Hillsdale , New Jersey: Lawrence Erlbaum Associates, 262-283.
Hacker, W. (1986), Arbeitpsychologie. Psychische Regulation von Arbeitstatigkeiten. Bern: Huber Verlag (2nd Ed.).
Izso, L. (2001). Developing Evaluation Methodologies for Human-Computer Interaction. Delft University Press, The Netherlands.
Koffka, K. (1935). Principles of Gestalt Psychology. New York: Harcourt Brace.
Kohler (1947). Gestalth Psychology. Princeton, NJ: Princeton University Press.
Landauer, T.K., and Nachbar, D.W. (1985) Selection from alphabetic and numeric menu trees using a touch screen: Breadth, depth, and width, Proc. CHI85, Human Factors in Computing Systems, ACM, New York, 73-78.
MIT Artificial Intelligence Laboratory: From http://www.ai.mit.edu/
National Institute of Standards and Technology. From http://www.nist.gov/pc2001
Norman, D.A., & Shallice, T. (1980). Attention to action. Willed and automatic control of behavior. University of California San Diego CHIP Report 99.
Norman, D.A., The Psychology of Everyday Things, Basic Books, New York, NY, 1988.
Quintanar, L.R., Crowell, C.R., and Pryor, J.B. (1982). Human-Computer Interaction: A preliminary social psychological analysis, Behavior Research Methods and Instrumentation, 14(2), 210-220.
Rasmussen, J. (1982) Human errors. A taxonomy for describing human malfunction in industrial installations. Journal of Occupational Accidents, 4, 311-333.
Rasmussen, J. (1983). Skills, Rules, and Knowledge; Signals, Signs, and Symbols, and Other Distinctions in Human Performance Models. IEEE Transactions on Systems, Man, and Cybernetics, Vol 13(3).
Rasmussen, J. (1986). Information Processing and Human-Machine Interaction. An approach to cognitive engineering. North-Holland.
Shneiderman, B. (1983). Direct Manipulation: A step beyond programming languages, IEEE Computer, 16(8), 57-69.
Shneiderman, B. (1998). Designing the User Interface. Addison-Wesley (3rd Ed.).
Shneiderman, B. (1994). Dynamic Queries for Visual Information Seeking. IEEE Software, 11(6), 70-77.
Xerox: Palo Alto Research Lab. From http://www.parc.xerox.com/istl/projects/uir/projects/InformationVisualization.html
A exhaustive list of the major theories of psychology.
An excellent illustration of how to incorporate research and research methodologies into your own work.
An exhaustive list and description of the psychological domains related to HCI, including an excellent listing and description of HCI research regarding design principles.