Does one size fit all? - Aptitude treatment interaction and error management training

Justus-Liebig-Universität Gießen

Does one size fit all ? –
Aptitude-treatment-interactions and
error-management-training

Diploma Thesis
Department of Psychology
University of Gießen

vorgelegt von

Heiko Müller

2003

Abstract

Error-management-training - which explicitly allows participants to make errors during training - has been shown to lead to better test performance than error-avoidant training which tries to prevent participants from making errors. Recently some authors suggest to take personal characteristics of the learner into account when choosing the appropriate training for different individuals. We examined the effect of error-management-instructions (rules of thumb that encourage trainees to make errors and learn from them) and the impact of cognitive ability, neuroticism, openness to experience, conscientiousness, goal orientation, achievement motivation and learning strategies on performance depending on the training method. 109 participants took part in a group software training. The groups were randomly assigned to one of 4 training conditions: error training with and without error-management-instructions and error-avoidant training with and without error-management-instructions. Results showed no difference between the 4 training conditions in task performance and declarative knowledge. The error-management-instructions seem to be crucial to error training but obviously did not work in this study. We found 4 interactions that indicate a wipe out effect of error-management-training for the impact of personal characteristics on learning success.

Note. The term error-management-training implies that the error-management-instructions are presented to the participants. Because we used two different error training conditions in our study – one with error-management-instructions and one without – we will use the terms error training with error-management-instructions and error training without error-management-instructions when we refer to our training conditions and not to the original error-management-training.

Content

1. INTRODUCTION ... 4
1.1 Error-Management-Training ... 5
1.2 Error-Management-Instructions ... 8
1.3 Aptitude-Treatment-Interactions (ATI) ... 10
1.3.1 Learning strategies ... 13
1.3.2 Cognitive ability ... 14
1.3.3 Conscientiousness ... 15
1.3.4 Neuroticism ... 15
1.3.5 Openness to experience ... 16
1.3.6 Goal orientation ... 16
1.3.7 Achievement motivation ... 18

2. METHOD ... 20
2.1 Participants ... 20
2.2 Training Design ... 20
2.3 Procedure ... 23
2.4 Personal Variables ... 26
2.5 Dependant Variables ... 29
2.6 Manipulation Checks and Controls ... 31

3. RESULTS ... 33
3.1 Preliminary Analysis ... 33
3.2 Manipulation Checks ... 36
3.3 Performance Effects ... 37
3.4 Interactions of Personal Variables and Training Method ... 37

4. DISCUSSION ... 44
4.1 Strengths and Limitations of the Study ... 49
4.2 Implications for Theory and Future Research ... 51

5. REFERENCES ... 54

1. Introduction

In many fields of modern life, training has become a very prominent method of personnel development. There are training programs for jobless persons to be reintegrated into workforce, office workers who need to be trained in new data processing systems, software experts to keep up with the rapid development of technology and many, many more. One of the largest area of training is the field of software training (Graf, 2001). We cannot ignore the presence of personal computers in nearly every aspect of modern life. But in times when money is limited, it is essential to look for the most cost effective forms of training.
One very effective method that has been examined for nearly fifteen years in the field of software training is the error-management-training. This training sets its focus on free exploration and the making of errors, who are seen as something useful to learn from. Trainees are confronted with so called error-management-instructions ("You can just learn from errors.", "There is always a way to leave the error situation – don′t panic.") that should take away the negative emotionality of the error. Trainees are encouraged to explore issues on their own and use errors as feedback that shows the trainee what is still to learn. Several studies in the past were able to show that this kind of training leads to better performance than classic tutorial trainings that guide the trainee step by step to the solution of a task (e.g., Greif & Janikowski, 1987; Dormann & Frese, 1994; Frese, 1995; Ivancic & Hesketh, 2000). We will refer to this guided approach as error-avoidant training because the detailed instructions should prevent the learner from making errors. Some authors plea for a combination of both methods (Debowski et al., 2001).
Recently, these findings have been reconsidered according to the idea that there are personal characteristics that might interact with the training method. Cronbach and Snow (1981) called such interactions of personal characteristics, ability, motivation or attitudes with a certain treatment aptitude-treatment-interactions (ATI). Gully, Payne, Koles, and Whiteman (2002) found such interactions in the way that error-management-training should be beneficial for persons with high cognitive ability and openness to experience whereas error avoidant training should be beneficial for persons with high conscientiousness. Unexpectedly there was no general performance superiority of the error-management-training in this study. An explanation for this missing effect could be that Gully et al. (2002) used the last training trial in a radar tracking task as performance measure. A consequence of this procedure is that error-management-training participants were encouraged to make errors (by the training instructions) in their performance phase. So the results of this study should be interpreted with care. Nevertheless the authors argue for a consideration of personal characteristics in training research. Heimbeck, Frese, Sonnentag, and Keith (2003) examined the interaction of goal orientation and training method on task performance and found a benefiting effect of error avoidant training for people with a high prove and avoidance goal orientation. The hypothesized positive effect of error-management-training on high learning goal oriented people could not be demonstrated. Furthermore Heimbeck et al. (2003) compared an error training group with error-management-instructions to an error training group without error-management-instructions and an error avoidant group. The results showed that the error-management-instructions are essential for error-management-training to work because the error training group with error-management-instructions showed better performance than the other two groups who had no difference between them.
We agree with Gully et al. (2002) and support the research of ATI for two reasons. First, the findings from those studies can give information about what kind of training fits for what kind of person (practical implication). Second, ATI could provide hints about how and why error-management-training works (theoretical implication). For example when we get to know that personal characteristics (e.g., habitual learning strategies) are beneficial for training success in the error-avoidant training but have no impact in the error-management-training, we can make inferences about mechanisms of the error-management-training assuming that this training method might induce such characteristics. The study at hand takes up the results of former studies and tries to broaden the knowledge about the work mechanisms of the error-management-training approach. First, we examine the effect of an error avoidant training group that is provided with error-management-instructions which was – to our knowledge – never done before. Second, we explore the impact of a large number of personality variables that potentially interact with the training method using a large sample size as proposed by Gully et al. (2002), because ATI are difficult to detect in small samples.
In the following sections we give a short summary of the characteristics of error-management-training and ATI. After introducing the examined personal variables we describe the method of this study followed by the results. In the discussion section we try to interpret the findings and take a look at the strengths and limitations of the study at hand.

1.1 Error-Management-Training
In every learning situation people make errors because of insufficient knowledge or skills, inappropriate goal setting and planning, interruptions during action or wrong interpretation of system feedback (Zapf, Brodbeck, Frese, Peters, & Prümper, 1992). In general, errors are linked with negative emotions and are said to lead to frustration, anger and despair (Brodbeck, Zapf, Prümper, & Frese, 1993). For a long time errors were seen as something that should be avoided to prevent the trainee from adopting wrong routines (Ausubel, 1968). Skinner (1953) equated errors with punishment that leads to emotional arousal and can therefore paralyse the learner and prevent him or her from further action. The result of this belief was the famous programmed learning machine which guides the trainee through the learning task and tries to minimize the possibility of errors to enhance learning by giving positive reinforcement (Skinner, 1968).
The error-management-training approach proposes just the opposite of these opinions. Errors are considered a functional part of the learning process and a very useful source of feedback (Frese & Altmann, 1989). This feedback is crucial to the learning process to evaluate the adequacy of action. So the principle of error-management-training is free exploration of the training task with the possibility of making a great number of errors. In practice the trainee gets a short introduction into the training material (e.g., a short description about the basic functions of a software program) and is then left to explore the training tasks on his or her own. Here the initiation of the ′Undo′-function is important because novices tend to go from one error to the next and therefore get deeper and deeper in the error state (Frese &Altmann, 1989). This is why a possibility to leave the error state is essential. Besides this, going backward in action to the point where the error occurred is the most used strategy to eliminate the error (Zapf, Lang, & Wittmann, 1991). In contrast to the concept of exploratory learning (Bruner, 1966), a concept very similar to error-management-training, the training tasks in error-management-training are rather difficult from the beginning, so there is the possibility of numerous errors. Trainees have to formulate and test hypotheses (by risking and making errors) concerning the training issue and can therefore develop a more sophisticated mental model of the software than trainees who just follow step by step instructions to the solution of the training task (Irmer, Pfeffer, & Frese, 1991).
At this point a very close linkage of error-management-training to action theory becomes clear. According to Hacker (1998) the operative image system (OIS) is such a mental model of all action relevant information a person has. It is the sum of internal long-term representations of condition-action-result interrelations and is therefore the cognitive basis of action regulation. It is to be seen as something like a cognitive model of all work situation-relevant issues. That means that an OIS is not a complete representation of the reality but a cost-optimizing system that contains only the action-relevant procedures. The difference to other cognitive systems is that an OIS is clearly action-oriented (which does not mean that it denies non-action-oriented knowledge). An OIS does not regulate actions but the better it is sophisticated and developed the better is the quality of the actions based on this specific OIS (Frese & Zapf, 1994).
In an error-management-training the construction of a well fitted OIS should be enhanced by the process of setting subgoals, developing hypotheses, testing them and using the feedback of occurring errors to further elaborate the existing OIS (Frese, 1995). During error-management-training the OIS should therefore become more appropriate than in an error-avoidant training, where participants only learn fixed routines and have no opportunity to test the adequateness of their OIS.
This process of elaboration of a mental model has an important consequence. In contrast to a guided, error avoidant form of training where you do not have to spend much cognitive resources following the instructions, error-management-training forces trainees to exert a great amount of effort into the solving of the training task and apply a deeper level of processing. According to Craik and Lockhart′s (1972) level of processing approach Marton and Säljö (1976a) mention two approaches to learning: a) surface approach like repeating the learning contents very often to remember them; b) deep approach that connects the learning task to existing knowledge structures for a better understanding. The authors demonstrated that the level of processing can be experimentally manipulated and that these manipulations have effects on learning outcomes (Marton & Säljö, 1976a, 1976b). Students seem to adopt one of these approaches depending on their expectations of what is required of them (Marton & Säljö, 1976b). In error training participants are explicitly told to explore the material and perform a self guided learning. They are instructed to get down with the material to understand it. This fact and the difficulty of the training tasks should suggest a deep level of processing approach to the participants. In consequence the usage of deep level processing should lead to a superiority of error-management-training concerning performance.
To demonstrate this superiority performance measures are assessed in a separate test phase after the training. Tasks in this phase are again rather difficult and exceed the material presented in the training. Participants of an error-management-training who are expected to have a better mental model of the training material due to deeper level processing and exploration showed a better performance than participants in an error-avoidant training (Greif & Janikowski, 1987; Frese & Altmann, 1989; Dormann & Frese, 1994; Frese, 1995; Ivancic & Hesketh, 2000, Heimbeck et al., 2003). In contrast to participants of error-avoidant training who did not encounter error-management-instructions and are affected by the negative effects of errors, error-management-trainees have acquired error-management-competences because they got used to deal with errors and see them as learning opportunities (Heimbeck et al., 2003).
By presenting tasks that were not introduced in the training the test phase resembles the later real life situation at work because there will always be issues in complex training material that could not be covered in the training. But then they have to be dealt with without the help of a trainer. So error-management-training prepares trainees best for transfer to real life work situations.

1.2 Error-Management-Instructions
The error-management-instructions are another fundamental part of error-management-training.

[...]