Review: Online Learning Based on Essential Concepts and Formative Assessment

At OEIT, two things we’re interested in are the notions of concepts/learning objectives/learning outcomes and embedded formative assessment. The Journal of Engineering Education published a relevant article in April 2012.

Lawton, D., Vye, N., Bransford, J., Sanders, E., Richey, M., French, D. & Stephens, R. (2012). Online Learning Based on Essential Concepts and Formative Assessment. Journal of Engineering Education, 101(2), 244-287.

This article provided a relevant overview of the literature, and it should be applicable to what we do, how we teach, and what we want to support at OEIT.

I think the title is a bit misleading, I think the article focused more on formative assessment plus “good” design rather on the impact of “essential concepts.” The authors don’t really spend as much time on why/how/wherefore essential concepts played a critical role in the course they designed and the assessment/evaluation they performed. Also, their discussion of the LMS is also a distraction, they don’t really focus on the community and communication aspects and their impact on the course, but rather it’s a placeholder (perhaps) for some future work.

I do believe it’s best to design learning experiences with clearly identified learning outcomes (in this case Big Ideas), assessments (tied to the Big Ideas), and learning activities and presentations (as based on Wiggins & McTighe, Understanding by Design). Nevertheless, I think the outcomes of the article fall into line with what I’ve been claiming about a possible approach to transition MIT courses online — pave the cowpath as a method of preparing courses for more extensive redesign based on concepts (but this is a different post) and embedded formative assessments. There are clearly multiple levels or layers in course design as highlighted in this article–from identifying the essential concepts (Big Ideas), to developing formative assessments that focus on those concepts and common misconceptions, to choosing which form(s) of feedback to provide and how customized (generic feedback or based on learner activity). I like it when they’re prescriptive of the good practices they used in the course design. They cite relevant literature, and they match the gut feel I’ve had so that’s good!

Two references in particular stand out for me:

1. Wiggins, G. & McTighe, J. (2005). Understanding by design, expanded 2nd edition. Alexandria, VA: ASCD. Available from Google Books or from ASCD at http://shop.ascd.org/Default.aspx?TabID=211&ProductId=406
2. Hattie, J. (2012). Visible learning for teachers. New York, NY: Routledge. (This is the practitioner focused version of Visible Learning, Hattie’s synthesis of over 800 meta-analysis of achievement, available from Google Books or at Amazon)

Other takeaways:

• There’s been quite a bit of research into formative assessment. Some highlights:
  

  • “Feedback without instruction can be confusing; instruction without embedded feedback can be disconnected. Excessive or too detailed feedback can be confusing while concepts are being introduced. Feedback is especially effective when ‘it addresses faulty interpretations, not a total lack of understanding.’ (Hattie, 2007). Feedback is most effective when instruction and feedback become intertwined until ‘the process itself takes on the forms of new instruction, rather than informing the student soley about correctness’ (Kulhavy, 1977).” (p. 248)
  • “Hinge questions directly differentiate and diagnose students’ conceptions. Their creation requires significant pedagogic and domain understanding. Striking examples of these for physics instruction can be found in Mazur’s Peer Instruction (Mazyr, 1997, 2011).” (p. 250)
  • “Nyquist [fusion_builder_container hundred_percent=”yes” overflow=”visible”][fusion_builder_row][fusion_builder_column type=”1_1″ background_position=”left top” background_color=”” border_size=”” border_color=”” border_style=”solid” spacing=”yes” background_image=”” background_repeat=”no-repeat” padding=”” margin_top=”0px” margin_bottom=”0px” class=”” id=”” animation_type=”” animation_speed=”0.3″ animation_direction=”left” hide_on_mobile=”no” center_content=”no” min_height=”none”][2003] classified the types of feedback into five ordered classes: weaker feedback (student is told whether their answer was correct or not); feedback only (the student is also provided with the correct answer); weak formative assessment (student also receives an explanation of the correct answer); moderate formative assessment (student is also given suggestions and hints for improvement on similar problems) and strong formative assessment (student is also given specific activities to undertake for improvement). The effect sizes for these types of formative assessments increase monotonically, with larger effects for stronger formative assessments.” (p. 251)

• They followed an instructional design paradigm based on Wiggins & McTighe’s (2005) Understanding by Design.
• Cognitive load:
  

  • “Cognitive load is important in online learning … because removing sources of learner confusion is critical when there is no teacher present to clarify things.” (p. 252)
  • “Cognitive load occurs when the number of active schemas exceeds limits on short term memory. Short term memory limits can be exceeded during learning, especially for novice learners. (p. 252)

• “Focused learner controlled video resources.” (p. 260):
  

  • “Each video was two to four minutes in length and students could replay them section by section…The videos were conceptual animations or involved a narrative over a set of screen captures doing a particular…activity… There was nothing to read in the treatment course outside of a course overview section and short introductory overviews of Big Ideas at the start of course modules. (p. 260)

• “Feedback and learner activity through formative and reflective assessments.” (p. 260)
  

  • “We designed the treatment course to make extensive use of formative assessments which were interleaved with the videos, so students could get feedback and correct misconceptions while they were learning. After each video there were one or more formative items related to concepts in the previous video or to motivate following videos. Minimally, the formative assessments would call attention to concepts in the video while others were hinge-type questions related to common misconceptions.” (p. 260)

[/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]