Udacity, San Jose, and it is what you do, not the way that you do it? Kindof.



I’m not sure how to cite this one.


It was a series oc Udacity xMOOCs for remedial and preparatory college courses. The greatest predictor of student success was student effort, specifically, the amount of problems’homework submitted by students, the number of hours of lectures watched, and the number and eveness of sessions they logged into. These were the major predictors of outcome.

The highest passing course had mandatory weekly problems to be submitted. The more problems a student submitted, the more their likelihood of passing increased, and this was exponential.

In general, f2f courses had better results than wholly online. But particular groups were particularly vulnerable. Students who were already academically at risk did worse online than in f2f courses. Particularly if they studied together.  Other groups who fared worse were young men, those with no previous online educational experience, and African Americans.

Another predictor of success was whether students had accessed support services by week 5.

Many students had limited knowledge of where support services were, as did many faculty, and what was available to them. Students wanted more f2f time with faculty, but there may be a degree to which the most underprepared students will be unwilling to contact faculty.

The vast, vast majority of questions put to faculty were not about content, but were technical, with regard to accessing the course, and procedural.

It looks like better information about what support is available, and where, with a focus on early uptake is important. The provision of more f2f instances, perhaps mandatory, and initiated by the course might also be useful. Assumnptions that students will be able to navigate the technology without help may also be problematic. Mandatory homework may also help significantly, and, perhaps, analysing data to

The Report

AOLE = Augmented Online Learning environments. The report “presents highlights form a study of Augmented Online Learning Environements  (aole) delivered by San Jose University through a collaboration with Udacity”

The courses were remedial algebra, intro to college level algebra, into to college level statistics.

Research questions 
“1. Who engaged and who did not engage in a sustained way and who passed or failed in the
remedial and introductory AOLE courses?
2. What student background and characteristics and use of online material and support
services are associated with success and failure?
3. What do key stakeholders (students, faculty, online support services, coordinators,
leaders) tell us they have learned?”


Matriculated students (college enrollees) did better than non-matriculated ones. The students from the partner high school were less successful than others. Pass rates varied hugely, both between these groups, and across the courses. e.g. pass rated in Math6L for matric = 29.8, non = 17.6, in stat 95, 54.3% and 48.7.

The study found that persistence, which it quantified as the number of problem sets a student submitted, “trumped all other variables tested for their relationship to student success”, including demographics, subject matter, and use of support.

The positive effect was not linear. Once a baseline effort level has been breached, success increases dramatically.

Video time ( amount of time spent watching the instructional videos) was another strong predictor, particularly for the stat 95 students (intro to college level stats)

The findings re use of online support services are not clear. They do not appear to have had a predictive effect, but qualitative data (student and staff responses) indicates that students had difficulty accessing these – due to online unfamiliarity, lack of awareness of them, and interaction difficulties with the platform. So, they suggest further investigation here, and no meaningful conclusion is drawn.

Context is key. Low pass rates can in part be attributed to the fact that the project targeted “at risk populations” ( would not a comparison to a f2f intervention with these populations prove useful here?), and students who were already known to be less likely to succeed in online environments.

Lit review

Focuses on Jaggars and Bailey, 2010, and Xu and Jaggars, 2013, who found “at-risk student populations” tend to to do worse here, despite the general results indicating no difference in students using onloine versus trad instruction, and a slight increase with hybrid.

Xu and Jaggars find that all students do less well online, but groups were more negatively affected – males, younger students, students with lowe levels of academic skills, and African American students” an effect exacerbated when those who “adapt least well to online learning study together” though the study has limitataions.

There is however further lit. based evidence for this outcome ” (Kaupp, 2012; Xiu & Smith, 2011, Terenzini & Pascarella, 1998).

Student Characteristics and Background

High school students represent 20% of all participants, and 45% of non matric students, and are subgrouped as a result. 213 students comprise the sample. (98 matric, 113 non matric). All Matric students in 6l had failed previously. This is too detailed to summarise, and is best assessed in the linked report.

Findings in detail

Students tended to fare better in f2f courses than in wholly online courses. In terms of results, between 10 and 20% more matric students in f2f scored a c or better. Spending less time than average was likely to correlate with students being unlikely to pass. Spending above average increased pass rates significantly.

Problem completion had a marked effect, with 90% of students who submitted below the median number of problems failing.

Early engagement with support services also correlated with increased pass likelihood. Students who did not interact with support within 5 weeks .

Summarising, student effort is the biggest predictor of success, as evinced by the results correlation with problems submitted, and video lecture viewing, number of sessions logged in for. Completing the first few problems in a course has a negligible effect on pass rates, but every subsequent problem increases chances of a pass. to a greater extent than thee previous one did over it’s predecessor.  Viewing hours of lectures has a very strong effect. The most successful course had mandatory assignments each week.

Stakeholder Input

Faculty, students, Udacity and University leadership team, co-ordinators, Udacity staff.

Many students indicated a lack of experience with online education, and a lack of awareness of what support was available. Slightly less than half the students in week 5 partially understood the support services available. 51% of students said they would use support services more as the main change they would make if they were to repeat the experience.

Students expressed a desire to have all their courses and resources on one site/ Many students expressed a desire for more f2f engagement with instructors, though very few students took the opportunity to videoconference with instructors.

It may, partially, be the case that underprepared students particularly fear engaging with faculty. Those that need faculty support most may be the least likely to access it (is this more the case in online contexts, where they have to initiate it entirely, and less in f2f?)

Students argued that the course was to a degree unresponsive – some stated that though the course got more difficult, they were alloted the same amount of time for lessons.

“In response to the Survey 3 question about what they would like for Udacity and SJSU to do differently a large majority of students also pointed to “more information upfront about what kind of support services are available” (87% of matriculated students and 72% of non-matriculated students) and “more information about how to find and access support services” (79% of matriculated and 72% of non-matriculated students).”

Faculty noted that the overwhelming help request they got was not about content, but about tech and procedural issues. Several faculty menmbers stated they got just one content related question all semester. Faculty were also less aware of support services.

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s