지식 추출
(in Cognitive task analysis) Knowledge elicitation is the set of methods used to obtain information about what people known and how they know it: the judgments, strategies, knowledge, and skills that underlie performance.
Categorization
Working minds에서는 데이터를 수집하는 방법이 무엇인지, 어떤 영역에 집중하는지에 따라 분류한다.
수집 방법에 따른 분류
Interviews:
The most common CTA method is a structured interview…. Interviews are efficient … . Interviews can also elicit information about issues that are easily missed by the other methods…. Many CTA practitioners view interview data as extremely rich, but best treated as exploratory data and as a source of hypotheses. —loc. 436, Chapter 2, Working minds
Self-reports:
These methods vary from highly structured formats, such as surveys and questionnaires, to open-ended formats such as diaries and logs…. The disadvantage is that structured questionnaires and rating scales do not allow for the elements of discovery and exploration that are available in more open-ended reporting formats…. data quality depends a lot on participants’ motivation and willingness to complete entries consistently. Finally, self-report methods assume that respondents are capable of “self CTA” and of reporting tacitly held knowledge, subtle cues and perceptions, and other cognitive elements on their own. That assumption is not backed up by research - in fact the evidence suggests quite the opposite: people have considerable difficulty reporting on their own cognitive processes (Nisbett and Wilson 1977; Wilson 2002). And as people gain experience and higher levels of skill, it becomes increasingly difficult for them to articulate the basis for their expertise and the judgments, decisions, and assessments they make so capably (Chi and Bjork 1991; Chi, Glaser, and Farr 1998; Feltovich, Ford, and Hoffman 1997; Klein and Hoffman 1993). —loc. 475, Chapter 2, Working minds
Observation:
If on-site observations are feasible (they often are not), we strongly advocate that the CTA researcher take advantage of the opportunity…. Observations provide opportunities for discovery and exploration of what the actual work demands are; what sort of strategies skilled workers have developed for coping; how work flows across the environment, the team, and the shift; and communication and coordination issues (Roth 2002)…. In our view, observation is best coupled with other forms of data collection such as interviews to find out how the participants were viewing the events. Merely recording the events and actions taken can result in a misleading or cognitively shallow account. —loc. 475, Chapter 2, Working minds
Automated capture:
The collection of CTA data can be handled by computers…. One example is the Situation Awareness Global Assessment Technique (SAGAT) developed by Endsley (1998b; Endsley and Garland 2000)…. In the midst of a computer-driven simulated mission, all of the instruments go blank and the pilots are asked to reconstruct the instrument values. SAGAT is a measure of situation awareness. According to Endsley, the better a person’s situation awareness, the more accurate the reconstruction. —loc. 475, Chapter 2, Working minds
집중하는 영역에 따른 분류
네 개의 측면을 고려:
- 시간time: 과거, 현재, 미래
- 현실성realism: 실제, 시뮬레이션/시나리오, 인공적 환경
- 난이도difficulty: 일상적 사건, 도전적인 사건, 드물게 발생하는 사건
- 일반성generality: 추상적/일반적, 과업, 사건/사고