The Role of Theory
in HCI

Development Methods



Survey Methods

Logging &
Automated Metrics

Choosing Human-Computer Interaction (HCI) Appropriate Research Methods

The Role of Theory in HCI

A. Sasha Giacoppo (
Department of Psychology
Catholic University
Washington, DC 20064 USA
October 2001

1. Introduction

Human-Computer Interaction (HCI) research is performed to provide and promote a scientific understanding of the interaction between humans and the computer technology and tools that we use. HCI research can be heavily steeped in the principles and controls of the scientific method. However, the methodologies of many non-scientific disciplines have also been adapted for use in HCI research and practice. The methodologies that HCI researchers select and use have been adapted from the research of other scientific fields, many of which come from the psychological sciences.

Many of the principles that have driven the differing canons of HCI (e.g. user interface design, information visualization, etc) grew from the scientific research and theories from psychology. The varying domains of psychology are interdisciplinary, and quite often, research spans into other disciplines. Fields such as mathematics, design, architecture, arts, linguistics, etc. all have relevant contributions to HCI. Findings from these various domains seek to explain the human-computer interaction in terms of the psychological mechanisms of the user.

The domains of psychology that are considered a component of HCI are described here, as well as the most influential theories, models, and research of those areas. In addition, the major theories and models that have come from HCI research are described. They illustrate the breadth and depth of HCI research, and will help you make decisions regarding the direction of your investigations.

Ultimately, generating theories from your own research will provide the greater HCI community with valuable knowledge. Theories and models that come about from your research will be one of three kinds:

  • explanatory theories
  • predictive theories
  • generative theories

Explanatory theories seek to explain the behavior of our world; they tend to provide a more conceptual model of the world. Predictive theories seek to predict outcomes based on the changing values of component variables. Predictive theories provide an extremely high level of utility to HCI, they allow designers to directly predict the outcomes of their designs on user performance variables. Generative theories generate guidelines and principles that provide useful and applicable knowledge and models. The type of theory a researcher intends to produce guides the selection of appropriate research methodologies.

2 Theories

2.1 Foundations of HCI

2.1.1 Cognitive Psychology

Simply put, cognitive psychology is the study of how people think and learn. The goal of cognitive psychology is to understand the psychological processes involved in the acquisition and use of knowledge by people. This includes domains such as perception, attention, memory, learning, thinking, and the importance of social and environmental influences on those domains. Cognitive psychology is a major contributor to HCI research by providing and applying psychological principles to understand and help develop models that explain and predict human performance. Information processing model

The information processing model, developed by Lindsay & Norman (1977), characterizes humans as information processors. The model explains the movement of information from input to output within a human, via a series of processing stages. The stages of processing are encoding, comparison, response selection, and response execution.

Figure 1. Information Processing Model

Later, the basic model was extended (Barber, 1988) to include the processes of attention and memory, which interact with information processing stages of processing.

Figure 2: Extended Information Processing Model

The information processing model has been a major influence in HCI by categorizing user behavior, and in turn, being able to predict user performance.


Provides a good overview of the information processing model, as well as the model’s relation to attention and memory.    Cognition&instruction_files/Infoprocessmodel/Infroprocessmodel/

A great website where the user can go through a slideshow that explains the various components of the information processing model. Hacker’s Action Theory (Hacker, 1973, 1978,1985,1986)

Action theory explains the determinants, processes, and consequences of work behavior. It explains human activity in the context of goal directed behavior, and that people tend to maximize the efficiency of their actions. The main components of Action Theory are acts, actions, and operations.

  • Acts: motivated and regulated by intentions (i.e. higher order goals), and realized through actions.
  • Actions: The smallest units of cognitive and sensory-motor processes that are oriented towards conscious goals.
  • Operations: Components of actions that have no independent goals

Action Theory deconstructs the process of translating an intention into an action. By doing so, it helps researchers investigate the components of an activity, beginning at the formation of a user’s goals, passing through the user deciding on and selecting the appropriate methods/tools to achieve that goal, and ending at execution, and iterative refinement, of those methods and actions.

User’s constantly change, alter, vary the actions used to achieve a goal. Rasmussen’s model of cognitive regulation (Rasmussen, 1982,1983, 1986), which supports Action Theory, provides a detailed description of the information processes that occur at three-levels of action selection. The information processes of the users “regulate” the types of actions selected. The highest level is the knowledge-based level, which involves the development and deployment of strategies and heuristics to achieve goals. Problem solving processes typify this level, where a user’s analysis of stimuli from the environment is used to choose an optimum strategy, mediated by the user’s model about the potential outcomes of available strategies. The next-lowest level is the rule-based level, where actions are controlled by rules or procedures stored earlier (Izso, 2001). Incoming stimuli are consciously organized by the user and trigger certain actions when the criteria for those actions are met. The lowest level of action regulation occurs at the skill-based level, where actions are executed as a result of a trigger not conscious to the user. Sensory motor reactions represent this level.

Rasmussen’s theory of cognitive regulation is important to HCI because it describes and categorizes user activity and behavior. This allows researchers to understand the cognitive processes that are really occurring, at different levels of interaction, within a user during use/interaction of a computer/interface. Andersen’s ACT-R

The Adaptive Character of Thought – Rational (ACT-R) theory describes a cognitive architecture, based on how cognitive skills are realized by “production rules”. It is about how humans/users acquire skills through the gradual acquisition and integration of production rules, which are if-then pairs. The combination of multiple production rules can be applied to wide variety of complex problems. The theory describes how knowledge from production rules begins as more skill-based, declarative knowledge and transforms into more problem-solving, conceptual, procedural knowledge.

The ACT-R provides HCI researchers with an understanding of how knowledge is built, and how it transforms. Application of ACT-R in designing systems and interfaces that properly match and enhance the user’s acquisition of knowledge and development of skills can provide very powerful products.

Similar to the ACT-R is Norman’s Stages of Action Model (1988), which provides a list of the stages that users go through in trying to use a system:

  • Forming the goal
  • Forming the intention
  • Specifying the action
  • Executing the action
  • Perceiving the system state
  • Interpreting the system state
  • Evaluating the outcome

This model deconstructs user behavior and activity as well, however, it provides a much more practical guide to user behavior. This explanatory model can also guide an HCI researcher when deciding on specific components of an interface to investigate.


A comprehensive site for everything about the ACT-R, including contemporary extensions to the model.

Provides a good overview of Norman’s model, as well as some graphics that help users understand the model better. Knowledge and Mental Models

Research into how humans construct information into knowledge also has major contributions to HCI. Understanding how knowledge is represented and organized is critical in the design and development of technology and tools. The applications of research findings intend to maximize the amount of knowledge and utility gained from interfaces.

Much research in knowledge representation, with relation to HCI, has focused on mental models. Mental models are dynamic constructions of knowledge that guide us (and sometimes misguide us!) in understanding and acting within our own worlds. Donald Norman (1998) provides a good definition of mental models:

“the model people have of themselves, others, the environment, and the thing with which they interact. People form mental models through experience, training, and instruction”


An excellent “beginner’s” explanation of mental models.

Claiming to be a “gentle introduction” to mental models, it is anything but. However, the website does provide an excellent example of how to understand what a mental model is. Attention & Memory

Research on the attention & memory of humans investigates the limits of human information processing, including the limits and structures of storing memory. This domain has helped HCI researchers define the limits of what an interface can provide, and has helped developed guidelines regarding what can and should be presented in an interface. Heavy research, in areas like focused/divided attention (e.g. Baddeley, 1986, 1990), or supervisory attention (Norman, D.A., & Shallice, T. ,1980), still continue to produce major findings that can be of great use with regards to HCI issues.

2.1.2 Perceptual Psychology

Perceptual psychology studies how people acquire information from their environment, through their senses, and how that knowledge is then encoded for use in future processes. Perceptual psychology, by nature, has been a major contributor to HCI research because it investigates how people acquire information from graphical representations, and interfaces.

Within the arena of perceptual psychology, there are two opposing paradigms of theorists, the constructivists, and the ecologists (sometimes called Gibsonians). Constructivist theories take the position that seeing is an active process in which perception is constructed from the environmental information before us AND from previously stored knowledge (e.g. Gregory, 1970). Ecological theorists (e.g. Gibson, 1979) take the position that perception of the environment can be solely derived from the environment itself, and does not require construction or previous knowledge on the behalf of the human.

Regardless of which camp you agree with, the theories are important when it comes to HCI. An excellent example of a perceptual theory are the Gestalt laws of perceptual organization, which are considered constructivist. Gestalt psychologists (e.g. Koffka, 1935; Kohler, 1947) believed that the interpretation (read: perception) of patterns of stimuli is based on innate laws of organization. In other words, humans are wired to perceive organizations of information to make sense. Gestalt laws, though perhaps antiquated, provide an excellent model for presenting information at an interface.


An entertaining site that explains some common illusions in the context of perceptual psychology, including a description and comparison of the major paradigms of perception.

2.1.3 Social Psychology

Social psychology studies how people behave in the context of their interactions with others. Social psychology is important to HCI research because it highlights the importance of social interactions when people are involved in tasks. The development of the internet into an online community is an important concentration of social psychology research. Understanding the new forms of social interaction and behaviors that occur within online communities will be critical to the design and development of tools and technologies. The theories and models of human behavior that come from social psychology promote researchers to investigate not only users by themselves, but how they operate within their natural, social environments.

2.2 HCI Research

By incorporating the theories and models from the various domains of psychology, HCI researchers have been able to investigate more specific and applied issues related to the human-computer interaction. HCI theories and models can be explanatory, but most tend to provide guidelines and principles for design and development of interfaces. Below is a list of the major areas of HCI research, their descriptions, as well as examples of theories and models.

2.2.1 Design Theories

Design theories help guide the design of interfaces and the selection of components contained within interfaces. Provided is a list of the areas within interface design that have been subject to investigation and research. GOMS (goals, operators, methods, selection rules) Model

Proposed by Card, Moran, and Newell (1980), the GOMS model deconstructs the activities of a user task into components of activity and the respective information processes. Users formulate goals (and subgoals), achieve those goals by using methods and procedures, via operators (e.g. move mouse), and use selection rules to choose appropriate methods and operators.

From this model, the keystroke-level predictive model was developed. It predicts performance time of tasks by calculating the sum of the lesser parts, which included time for keystrokes, pointing, thinking, waiting, etc. From this predictive model, designers of software were able to improve their products based on user performance at each and every level. For HCI researchers, the GOMS model provided a guide for areas that needed investigation.


A great comprehensive site that takes through GOMS from start to finish, including examples, and slide shows.

Though it requires access to the Association for Computing Machinery’s (ACM) digital library, this link provides a great explanation of the GOMS model. At the very least, it provides an excellent, linked page of references to numerous additional GOMS research articles. Menu Design

The organization and selection of information is another area of HCI investigation. Menu structures are an important component of interfaces, and the appropriate menus can greatly enhance or detract from the user’s experience and efficiency. The combination of the variety of menus (e.g. between scrolling, two-dimensional, alphasliders, pop-up menus), menu phrasing, presentation sequence, etc. offer many trade-offs between speed, efficiency, and time.

An excellent example of how research within HCI can produce useful predictive models is from Landauer and Nachbar(1985). Landauer and Nachbar’s predictive equation for selection time within menus is an excellent example of how research within the various fields of HCI can provide useful, empirical models in determining the effectiveness of differing designs and prototypes.. By varying the number of items per menu level to reach 4096 target items, the investigators were able to develop a predictive model for selection time.

T = k + c^log b

T = time to achieve selection
k, c = constants
b = breadth (number of items per level)


This link provides a nice table of the advantages and disadvantages of the various menu options. Input devices

Much research in HCI has been dedicated to predicting the effectiveness of input devices. Various input devices (e.g. mice, joysticks, trackballs) have investigated on measures of ease of use, accuracy, and speed. Determining the performance of these devices has resulted in the development of motoric theories.

A landmark theory is Fitt’s Law, an important predictive equation for pointing time, developed by Paul Fitts (1954). Fitt’s Law predicts the amount of time to move a given distance to a target of specific width. It predicts that pointing time increases as a target is further away and smaller targets take longer to point to.
Fitts’ Law: Time to point = C1 + C2(index of difficulty) User Control/ Direct Manipulation

The ability to enable a user with control over the tool/interface is an essential component of the HCI interaction. Rather than leaving users with a feeling that they are slaves to a technology, users should have feelings of control and mastery over an interface. Shneiderman (1983) explained the central ideas of user control, which include:

  • Visibility of Object and Actions
  • Rapid, reversible, incremental actions
  • Replacement of complex command-language syntax with direct, visual manipulation of the object of interest.


A good explanation of direct manipulation.

A great collection of some papers regarding design theory, all published from research at the University of Maryland Human Computer Interaction Lab (HCIL) Anthropomorphic design

Designing technology to act and behave like humans is another debate of HCI. Many people feel that computers should be more human like, which is generally a naïve interpretation of making technology “user-friendly”. Shneiderman (1998) describes several major problems of anthropomorphic design. Anthropomorphism projects false models of the extent of a computer’s usefulness, doesn’t clarify that there ARE differences between humans and computers, and can injure people emotionally by making them feel dumb or inadequate when things don’t work correctly. Quantitatively, anthropomorphic design can even have negative effects on performance (Quintanar, et al., 1982). The principles of designing proper interfaces and tools involves understanding user behavior, not making or portraying computers to act like humans. Multimodality

Tools and technology are not limited to visual stimulus. Interfaces can also include sound, touch, gesture, tactile inputs and outputs. Multimodal interfaces are really about making the most efficient use of the human senses. Investigating the effects of mode regarding user performance and activity within an interface is extremely important. Understanding multimodality, especially as an interface component, requires investigating human information processing, perceptual psychology, and the attentional and working memory processes of users. The utility of multimodality extends beyond enhancing performance for the typical user by being able to provide equivalent access of technology for people with disabilities.


This site provides a good picture of the types of research and applications that is involved with multi-modal interfaces.

2.3 Additional Domains of HCI Research

2.3.1 Information Visualization

“Information Visualization is the use of computer-supported interactive visual representations of abstract data to amplify cognition” (Xerox: Palo Alto Research Lab (PARC)). As opposed to scientific information, which is generally a representation of physical information, information visualization techniques are used to present more abstract information in a much more effective manner. Techniques of information visualization help interface designers present data to users in meaningful and resourceful ways. These techniques, like dynamic queries (Shneiderman, 1994) and the fisheye lens (Furnas, 1981), illustrate how to successfully present huge amounts of data (read: the internet), in confined spaces, like the internet. These techniques are based upon underlying theories of vision, cognition, and perception


A comprehensive site about information visualization, including lists and descriptions of the multitude of information visualization techniques.

Another comprehensive information visualization site, including numerous examples, techniques, and slide shows.

2.3.2. Ubiquitous & Pervasive computing

Ubiquitous computing is about the omni-presence of invisible technology in our environments. The term pervasive computing means that technology permeates our lives. It is the strong, emerging trend towards, described by the National Institute of Standards and Technology,

  • Numerous, casually accessible, often invisible computing devices
  • Frequently mobile or imbedded in the environment
  • Connected to an increasingly ubiquitous network structure

The goal is to make computing and technology simple to use, everywhere around us, accessible to people with minimal technical expertise, reliable and more intuitive. Computing has shifted from one computer, many users (e.g. mainframe computing) to one person, one computer (e.g. PCs) to the future of ubiquitous computing, where it will be one person, many computers.

Figure 3. The Growth of Ubiquitous Computing

Leaping ahead into a future world of technological ether is a daunting task. The task of anyone developing this world should be to make technology that disappears, a world where our lives are enhanced by technology that doesn’t invade, but allows us to accomplish our tasks. This is a fundamental axiom of the human-computer interaction.

Ubiquitous and pervasive technology will not succeed if the interface between this technology and the human-user is not properly designed. It is the onus of researcher to understand what makes an interface user-friendly, and, in the case of ubiquitous and pervasive computing, it is paramount to understand the massive importance of the social and environmental impacts.


A good discussion of ubiquitous computing, as well as informative documents and slideshows.

2.3.2. Universal usability

Universal usability is an extremely wide, and under-developed area, of HCI research. It deals with providing equivalent access to technology and tools for people with impairments or disabilities.

The immensity of developing tools and interfaces that are usable by everyone is daunting, let alone designing tools that are specifically designed for people with disabilities. The onus is on the shoulder of the designer to use research, thoughtfulness at the very least, to understand how different users, especially people with disabilities, interact with an interface.


THE website to go to for anything you need answered about universal usability.

Examples of universal usability research projects performed by University of Maryland students.

2.3.4. Artificial intelligence (AI)

As the MIT Artificial Intelligence Laboratory puts it, AI seeks “to understand the nature of intelligence & to engineer systems that exhibit intelligence”. Research in artificial intelligence spans multiple disciplines, and can certainly delve into the metaphysical. AI seeks to define and understand intelligence, the deep, interweaving roles and importance of memory, emotion, knowledge, perception, etc, on intelligence, and the reasons and motivations to create intelligent systems. Products of HCI/AI research, and the theories that spring from AI, drive some of the most advanced technologies in areas like imaging, robotics, and language recognition.

Some pundits argue that the motivations of artificial intelligence to design and develop technologies that intend to mimic the processing of humans are misguided, because we should not be wasting our efforts and resources on designing computers to be human-like. Proponents believe that attempting to understand the complexities of the “intelligent human” via AI research can not only lead us to a greater understanding of ourselves, but can lead to advanced forms of computation and technology.


MIT’s Artificial Intelligence website. It describes the past, present, and future of artificial intelligence.

The SRI Artificial Intelligence Center website. Provides links that describe the major areas of research in AI

2.3.5 Supervisory control/Autonomous agents

“An autonomous agent is a system situated within and a part of an environment that senses that environment and acts on it, over time, in pursuit of its own agenda and so as to effect what it senses in the future.” (Franklin, S. & Graesser, A., 1996)

Autonomous agents represent technologies, usually applications, that make decisions for users based on use and intentions. Autonomous agents are really about predictive technologies. The ability for a computer to predict human behavior is a very complex task. Incorporating predictive technologies like autonomous agents into user-interfaces can have very serious drawbacks, not the least of which is a serious loss of user locus of control. It is important for researchers to investigate the true impacts of autonomous agents on user performance.

Figure 4. Hierarchy of Agents


This website provides a list of definitions and terms of autonomous agents, as well as some UI design principles.

This website describes the various types and applications of autonomous agents, as well as provide useful definitions.

2.4 Social and Cultural Theories

There are large amounts of information related to the human computer interaction that investigate the social and cultural aspects of technology use and design. A list of these domains, including some links, are provided.

Sociology and social psychological theories
Connections: New Ways of Working in the Networked Organization, Sproull & Kiesler, 1991.
An Electronic Group is Virtually a Social Network, Wellman, 1997
WebUse Research Project Resources, Univ of Maryland

Social Networks

Activity theory
What is Activity Theory?
Context and Consciousness: Activity Theory and Human-Computer Interactions, Bonnie A. Nardi

Distributed Cognition
A Brief Introduction to Distributed Cognition, Yvonne Rogers

Cultural theories
Cultural Dimensions and Global Web User-Interface Design: What? So What? Now What? Aaron Marcus
Hofstede, Geert, Cultures and Organizations: Software of the Mind, McGraw-Hill, New York, 1997.

Social informatics
What is Social Informatics and Why Does it Matter? Rob Kling
Edinburgh Social Informatics Site at Napier University School of Computing

Online communities
Strengthening Communities
Online Communities: Designing Usability, Supporting Sociability, Preece, 2000
New Community Networks: Wired for Change ,Schuler, 1996

3. Recommendations

Research regarding the Human-Computer Interaction will always have a foundation of work done in the fields of psychology. Understanding these foundations can help guide your HCI research interests, and help highlight the methodologies that have helped improve understanding of HCI issues in the past. Further research should focus on areas that need to be explained, and ultimately, provide predictive models for issues within those areas.

Exposing yourself to the wide range of research throughout the domains of HCI will help you envision the breadth of the applicability of your personal research. Understanding the interrelationships between disciplines, and the importance of research within those disciplines, will soon build an understanding of the importance of HCI research, and hopefully inspire you to utilize research as a tool for advancing the quality of your work.

3.1 References:

Lindsay, P.H. and Norman, D.A. (1977). Human Information Processing: An Introduction to Psychology, 2nd edition. New York: Academic Press.

Andersen, J.R. (1990). The adaptive character of thought. Hillsdale, NJ: Lawrence Erlbaum Assoc..

Baddeley, A.D. (1990). Human memory: Theory and practice,. Oxford: Oxford University Press.

Baddeley, A.D. (1986). Working memory. Oxford: Clarendon Press.

Barber, P (1988). Applied Cognitive Psychology. London: Methuen.

Card, S.K., Moran, T.P., and Newell, A. (1980). The Keystroke-level model for user performance with interactive systems, Communications of the ACM, vol 23, 396-410.

Fitts, P.M. (1954). The information capacity of the human motor system in controlling amplitude of movement, Journal of Experimental Psychology, 47, 381-391.

Franklin, S. and Graesser, A., Institute for Intelligent Systems, University of Memphis

Furnas, G.W. (1981). The FISHEYE View: A New Look at Structured Files. Murray Hill, NJ: AT&T Bell Laboratories.

Gibson, J.J. (1979). The Ecological Approach to Visual Perception. Boston, MA: Houghton Mifflin.

Gregory, R.L. (1970). The Intelligent Eye. New York: McGraw-Hill.

Hacker, W. (1973). Arbeits – und Ingenieurpsychologische Beitrage zur socialistischen Rationalisierung der Volkswirtschaft. In Hacker, W., Timpe, P., Vorwerg, M. (eds.) Arbeits-, ingenieur- und sozialpsychologische Beitrage zur sozialistischen Rationalisierung. VEB Verlag der Wissenschaften: Berlin.

Hacker, W. (1978). Allgemeine Arbeits – und Ingenieurpsychologische. Psychische Struktur und Regulation von Arbeitstatigkeiten. Bern: Huber Verlag (2nd Ed.).

Hacker, W. (1985). Activity: a fruitful concept in industrial psychology. In: Frese, M., Sabini (eds.). Goal directed behavior: the concept of action psychology. Hillsdale , New Jersey: Lawrence Erlbaum Associates, 262-283.

Hacker, W. (1986), Arbeitpsychologie. Psychische Regulation von Arbeitstatigkeiten. Bern: Huber Verlag (2nd Ed.).

Izso, L. (2001). Developing Evaluation Methodologies for Human-Computer Interaction. Delft University Press, The Netherlands.

Koffka, K. (1935). Principles of Gestalt Psychology. New York: Harcourt Brace.

Kohler (1947). Gestalth Psychology. Princeton, NJ: Princeton University Press.

Landauer, T.K., and Nachbar, D.W. (1985) Selection from alphabetic and numeric menu trees using a touch screen: Breadth, depth, and width, Proc. CHI’85, Human Factors in Computing Systems, ACM, New York, 73-78.

MIT Artificial Intelligence Laboratory: From

National Institute of Standards and Technology. From

Norman, D.A., & Shallice, T. (1980). Attention to action. Willed and automatic control of behavior. University of California San Diego CHIP Report 99.

Norman, D.A., The Psychology of Everyday Things, Basic Books, New York, NY, 1988.

Quintanar, L.R., Crowell, C.R., and Pryor, J.B. (1982). Human-Computer Interaction: A preliminary social psychological analysis, Behavior Research Methods and Instrumentation, 14(2), 210-220.

Rasmussen, J. (1982) Human errors. A taxonomy for describing human malfunction in industrial installations. Journal of Occupational Accidents, 4, 311-333.

Rasmussen, J. (1983). Skills, Rules, and Knowledge; Signals, Signs, and Symbols, and Other Distinctions in Human Performance Models. IEEE Transactions on Systems, Man, and Cybernetics, Vol 13(3).

Rasmussen, J. (1986). Information Processing and Human-Machine Interaction. An approach to cognitive engineering. North-Holland.

Shneiderman, B. (1983). Direct Manipulation: A step beyond programming languages, IEEE Computer, 16(8), 57-69.

Shneiderman, B. (1998). Designing the User Interface. Addison-Wesley (3rd Ed.).

Shneiderman, B. (1994). Dynamic Queries for Visual Information Seeking. IEEE Software, 11(6), 70-77.

Xerox: Palo Alto Research Lab. From

4. Links

A exhaustive list of the major theories of psychology.

An excellent illustration of how to incorporate research and research methodologies into your own work.

An exhaustive list and description of the psychological domains related to HCI, including an excellent listing and description of HCI research regarding design principles.