MIT Study of Online Course : Equal Results To On Campus?

Interesting - online course yield results for students nearly equal to those that attended on campus even with special extra help from instructors. Innovation is everything but more work is needed to refine the process.

MIT Study Finds Learning Gains for Students Who Took Free Online Course
Source:  Nick Anderson, "MIT Study Finds Learning Gains for Students Who Took Free Online Course," Washington Post, September 23, 2014.

September 29, 2014

Students in a free online physics course from the Massachusetts Institute of Technology demonstrated roughly equal learning gains if they stuck with the class, regardless of previous academic experience, researchers recently found.

Their progress also was comparable to what some MIT students showed when they were required to take the introductory course on campus as a remedial measure.

The findings offer evidence that "massive open online courses," or MOOCs, can be an effective way to teach a broad range of people from around the world who want to learn from top-flight universities without paying tuition.

David Pritchard, physics professor at MIT, led a team of academics from MIT, Tsinghua University in China and Harvard University who analyzed the results of an MIT course called Mechanics Review, which was offered in 2013 on the MOOC platform edX. Results of their study were published in the International Review of Research in Open and Distance Learning.

• About 17,000 people registered for Mechanics Review, a MOOC patterned after a course offered on campus to MIT students who had received a D or lower in a fall physics course. Most of those who registered were just window-shopping.
• After the second assignment, fewer than 10 percent were on track to complete the MOOC.
• Eventually, about 1,000 people earned certificates of completion.

The researchers gave students an identical test before and after they took the MOOC, comparing results to assess gains. They also analyzed in detail the performance of 1,080 students who attempted more than half of the MOOC questions, using a statistical method called item response theory. That meant the MOOC "dropouts" were effectively excluded from the study. What surprised Pritchard was that various types of MOOC students showed relatively equal progress.

• Those who had obtained a doctorate advanced about as much as those whose highest credential was a master's degree, a bachelor's degree or a high school diploma.
• Gains also were about equal for MOOC students regardless of their level of preparation in math and physics.

That did not mean all scored equally well. Some had superior results, and some failed. But the relative progress of those who persisted was roughly equal across groups.

The analysts also compared how the persistent MOOC students did with results for 35 MIT students who took a similar review course on campus in 2013 after struggling with the subject. Here, too, there was a surprise: Even though on-campus students had extra instruction, researchers found "no evidence of positive, weekly relative improvement of our on-campus students compared with our online students."

Pritchard said this finding was puzzling, given how much faculty attention the MIT students receive. "I had hoped that because they went to our classes, they would learn a lot more," he said.