Expertise: a long and winding road

The idea that experts tackle problems that increase their expertise seems supportive of the self-efficacy behind ACT-R:

  • Reinvestment – the  motivational aspect
    • conserving resources to have energy to put back into new problems
  • Progressive problem-solving – the cognitive aspect
    • tackling more difficult problems AND tackling more complex representations of recurrent problems
    • represents working at the edge of competence (ZPD)

Pattern learning, which occurs without extensive effort, involves choosing the right patterns and recognizing when no pattern fits. This seems to equate with imagaic memory which is efficient for spatial and temporal data.

Procedural learning starts as step-by-step problem solving which become “chunked” into a single procedure; while this automaticity frees resources, it becomes a handicap in the improvement of performance. However, automaticity does not inevitably lead to inflexibility if automated skills are building blocks to new skills that are not automated.

However, learning is also pattern and procedural learning; what distinguishes expertise is the seeking of complexity. Complexity is described as a matter of the number of constraints. Because most real-world problems are not reducible to a step-by-step economy, we use simplified representations (akin to simulations). The class of problems that are endlessly complex are the constitutive problems of a domain.

Experts are motivated by:

  • flow:  total absorption and feeling of control, loss of normal time; becomes addictive to the point that problems are invented
  • second-order environment: the expert sub-culture where your recognition as an expert matters to you (not useful in fostering early development of expertise)
  • heroism: effort disproportionate to rewards

Competitive environments foster expertise. However, so do expert sub-cultures which may not always involve competition but always involve recognition of success and help/cooperation leading to success. The expert environment constantly changes as the experts become more expert; the reason experts help each other is to help that environment conttinue to get more difficult (i.e., inventing problems). Expert teams are one example: everyone more or less knows how to do everything so the focus is on the goal, not on individual achievement.

Instance-based and rule-based learning

Citation

Taatgen, Niels A. and Wallach, Dieter. “Whether Skill Acquisition is Rule or Instance Based is determined by the Structure of the Task.” Cognitive Science Quarterly: 2.2 (2002). 163 -204.

Summary

The traditional view of skill acquisition as “a gradual transition from behavior based on declarative rules in the form of examples and instruction towards general knowledge represented by procedural rules” is challenged by instance theory. While the latter seems to explain the inability of experts to verbalize rules, research on the directional asymmetry of rules seems to support the traditional importance of rules.

ACT-R’s instance-based architecture is based on 2 key arguments:

  • the strategy to use or the memory to retrieve is based on which has the highest expected gain (optimization)
  • declarative memory is activated (and filtered) by environmental demands and past experience (which has been encoded as production rules by procedural memory)

At the symbolic level of ACT-R, procedural rules are applied to declarative chunks which store information in a proposition; chunks are either new (perceptions) or created internally by prior knowledge/experience. Each rule contains a condition- and an action-part, and declarative items are pattern-matched to the condition and applied in the action. The subsymbolic level of ACT-R deals with the choice of which rule to apply according to Bayes’ Theorem (increases base-level activation each time it is retrieved and decays over time).

The article seems to imply that if instance-based learning fails (because the problem is too time-consuming–no discernible pattern), learners will attempt to derive some sort of rule. The authors argue that instance-based learning works best when the relationships between variables is very difficult; rule-based learning (simplified cases) is more successful with a large number of cases with obvious relationships.  The authors then test this hypothesis with two detailed experiments.

The results:

  • Previous research showed no learning through observation without direct rules (explicit relationships)
  • Subsequent research, indicated that exploratory participants did better than observers, but that observers could better verbalize and construct a causal model.
  • This research shows evidence of learning by observing even without rules; and participants seem better able to answer questions about old systems than new. Both results support instance-based learning.

Response

ACT-R seems to offer tremendous explanatory power. However, instance theory seems related to the concept of expert (tacit?) knowledge:

  • know when to apply
  • gets better over time (more instances)
  • gets better with use
  • additive for community

Because production rules that propose new declarative rules are not accounted for in the ACT-R architecture, this “missing link” may be the elusive transfer element; the authors propose partial-matching in ACT-R retrieval as a solution.

In instance theory, encoding and retrieval seem more closely linked to temporal/spatial reality than to attention as the authors claim; however, the view of memory as evolving from algorithmic processing to memory-based processing succinctly describes expert knowledge.

The recommendations for design strategies seem profound:

  • Instance-based learning takes over from rule-based over time
  • Creating declarative rules is the most important (first) step
  • Analogy works best at the start but declines quickly
  • Declarative rule works well at start and persists
  • Instance continues to improve over time to become best

First Principles

Impact

I measure conceptual density by the number of highlights; I measure impact by the number of marginal annotations. My copy of Merrill’s, First Principles of Instruction is almost illegible on both counts.

Hypothesis

Merrill argues for the existence of basic methods (first principles) that are always true regardless of the variable method used (as a specific instructional activity/practice or a set of activities/program). The argument seems plausible although those principles are open to debate due to the hypothesis that learning is in direct proportion to the implementation of those principles (i.e., if a set of principles is indeed “first,” we’d need to prove the proportion and disprove any other possible principles). Clark’s four instructional architectures (receptive/lecture, directive/tutorial, guided discovery/simulation, and exploratory/collaborative problem-solving) is worth exploring later.

Merrill’s model (shown later) is compared with two similar models from Vanderbilt and from McCarthy. Merrill then proceeds to expand on characteristics of each of his five principles.

Problem

  • Show learners real-world problem rather than abstract learning objectives
  • 4 levels of instruction
    1. problems
    2. tasks to solve
    3. operations that comprise the tasks
    4. actions that comprise the operations
  • Elaboration theory advocates progression of successively more complex problems
  • If a simple version of a complex problem is difficult to locate, the coach must do some of the problem solving for the learner and do less and less with each successive problem (scaffold)

Activation

  • Learners recall prior knowledge
  • Learners are provided foundational knowledge
  • Learners demonstrate previous knowledge (pre-test as activation rather than assessment)

Demonstration

  • Demonstration must be consistent with goal
    • Examples and non-examples for concepts
    • Demonstrations for procedures
    • Visualizations for processes
    • Modeling for behavior
  • Multiple representations for demonstrations (i.e., Gardner)
  • Three classes of problems
    • Categorization
    • Design (plans and procedures)
    • Interpretation
  • Coaching involves information focusing which is gradually faded (scaffolded)
  • Presenting alternative representations is not sufficient; learners must compare

Application

  • Information-about: recall
  • Parts-of: locate/name
  • Kinds-of: new examples
  • How-to: do
  • What-happens: predict

Integration

  • Knowledge must be transferred to life beyond the instruction
    • Publicly demonstrate (could be high score)
    • Reflect, discuss, defend
    • Create new and personal ways to use
  • Multimedia has a temporary (attention-getting) effect on motivation

Comparison

Finally, Merrill compares his principles with components from other learning theorists:

  • Gardner – emphasis on problem and activation (entry points and analogies)
  • Nelson – emphasis on application via collaboration
  • Jonassen
    • Related cases can supplant memory by providing representations of experiences the learner has not had
    • Behavioral modeling demonstrates how to perform activities
    • Cognitive modeling articulates reasoning used while engaged in activities
    • Scaffold by
      • Adjusting difficulty
      • Restructuring task to supplant lack of prior knowledge
      • Providing alternatives
  • van Merrienboer
    • Multiple approaches to analysis
    • Recurrent skills – require consistent performance – supported by just-in-time information
    • Non-recurrent skills – require variable performance – supported by elaboration
    • Progression of demonstrations
      • Worked-out examples
      • Just-in-time information
      • Models of heuristic methods used by skilled performers
    • Demonstrations are subordinate to practice
    • Demonstration and application are integrated (and iterative?)
    • Product-oriented and process-oriented problems
  • Schank
    • Emphasis on application
    • Goal/mission mapped to story/role (environment)
    • Coaches scaffold
    • Experts tell stories

Merrill concludes by questioning collaboration as a first principle; however, the analysis of solitary activities may answer the question:

  • a learner alone with a book – the author is the collaborator
  • a learner who makes a discovery – perhaps learning does not exist if it can’t be replicated/shared (if a tree falls…)

A slight expansion of Merrill’s model to position the problem in an environment of learners and coaches may accommodate this concern:

Merrill's Problem Model Extended with Environment

Merrill's Problem Model Extended with Environment

Big misconceptions

The final chapter in Wiggins deals with the three most common qualms about backward design, all of which seem germane to any change. The first misgiving (“I need to teach to the test”) is effectively refuted by citing the research that shows challenging instruction produces long-term gains (and transfer); as a result, you CAN teach to the test by teaching authentically and without drill and kill on released test items. The third misgiving (“I don’t have time”) is countered with the recommendations to share good practices and collaborate; this seems more effective in K-12 than in higher education.

The second misgiving (“I have too much content to cover”) is an old and well-entrenched argument and is more complex. The notion that the textbook equals the course content is spurious; we know that it’s simply a tool, and if a student can pass the TAKS test without memorizing the entire textbook, a teacher should not attempt to cover the textbook in lockstep fashion. This section then lays out a three-part sequence that few teachers could argue with:

  1. Students come in with preconceptions based on individual histories; we must first engage them.
  2. Students must learn facts, place them in a conceptual framework, and then organize them for retrieval and application.
  3. Students must reflect on their own learning (metacognition) which provides learner control and self-monitoring

Overall, I found this a practical way to end the book; by focusing on misconceptions, Wiggins practices the big ideas of instructional design.

PBL design of Alien Rescue

Because students see no value in what they are asked to learn, they “tune out”and never own that knowledge. PBL seeks to address this value proposition by creating authentic situations that students care about. PBL develops skills in 3 areas:

  1. problem definition (critical) and problem solving (trial and error?)
  2. reflection (can this be done with blogs or social networks?)
  3. deep understanding

Alien Rescue implements 3 implementations of PBL:

  1. anchored instruction
  2. goal-based scenarios
  3. cognitive flexibility (multiple learning perspectives in ill-structured domains)

Alien Rescue uses cognitive tools to support the scaffolding that PBL requires (which I think is equivalent to leveling up in games); these tools:

  1. support cognitive and metacognitive processes;
  2. share cognitive load by supporting lower-level cognitive skills to free up resources for higher order thinking;
  3. allow learners to engage in activities that would be otherwise out of their reach; and
  4. allow learners to generate and test hypotheses in the context of problem-solving.

Alien Rescue incorporates PBL design features:

  • situating the problem
  • complex problems with tools
  • multimedia formats for different learning styles
  • expert guidance from multiple perspectives
  • interrelated knowledge through links

The lessons from the learners were enlightening:

  • The expert tool brought self-study inline with expert actions (expected) although another Alien Rescue article suggested that students did not like the loss of control that occurred when the expert tool was used.
  • A tool that supported activities otherwise denied to students proved too popular; students over-used the tool, requiring a design change that made the tool availability a reward.
  • The version that includes expert stories (which seem distinct from the expert tool) to scaffold learning produced significantly better near transfer and far transfer results.

The events you are about to see are real

Fools rush in… I see now that these posts should have been limited to 200 words and comments to 100. I’d heard of Gagne’s 9 events before but the interesting part of reading his original work is the bifurcation of the process: the first 5 steps are the learning, and the last 4 are the assessment which implies that every instructional event must include proof. The Presentation step was the most helpful, especially:

  • content must mirror the objective in delivery mode
  • variety is required so learners can generalize
  • present discrimination through finer- and finer-grained examples
  • present concepts through a variety of examples and non-examples
  • provide examples then a definition for concrete concepts with younger learners
  • provide a definition then examples for defined concepts with older learners

I had misunderstood that the Guidance step was practice when in fact it is a series of small activities that build and allow the learner to discover the big idea; constructivists will say that learners who need fewer activities (hints) before seeing the big idea bring a different social-cultural history to the event. I also misunderstood the last two steps; assessment is NOT a repeat of feedback but a drive toward reliability and validity while transfer is applying the concept to an entirely new problem.

For my ILM, I now suspect that assessment of an attitude may require human observation.