The web-site is now in readonly mode. Login and registration are disabled. (28 June 2019)

Briar Jamieson: Measuring Outcomes/Learning Gains in Open Access Language Learning Environments

Cloud created by:

Briar Jamieson
15 January 2013

My Dream

I have come to the OLDSMOOC to understand how 21st Century learning designers are expressing and measuring learning outcomes in open educational environments.  My hope is to immerse myself in the OLDSMOOC activities and learn from participant experts/explorers by observing/questioning decisions about activities and the indicators/measures/assessment of learning gains.

It is my dream to find ways to better express, collect, measure, report the learning gains of self-directed language learners registered with my publicly funded organization.  

My situation (context)
  • not-for-profit organization, providing language learning resources/support to Self-directed learners, 1600 registered learners, 300-350 active every month; w/2.5 FTE instructors
  • funding received from government to provide language learning supports/resources to newcomers
  • access to learning resources at no cost to learners everything online
  • activities for learners include: drop-in online classes, access to modules from Literacy to Intermediate, additional lessons, assignments, tests, community discussion board, 4 week courses, weekly tips, learners encouraged to develop Personal Language Learning Environments, access to an
  • most of these activities are self-assessed, answer keys, community discussion, journal reflections
  • instructors feedback on baseline assessments, guide to further resources, deliver synchronous classes, weekly tips, moderate discussion board and feedback on journal reflections
  • learners can pick and choose the content and activities they want to engage in; there is no requirement to complete
  • work completed within an LMS
  • learners have access to a learning portfolio within the LMS to show evidence of work but not a requirement

Learner: focus on writing, completes only writing activities, receives feedback on writing and then leaves. We have examples of writing improvement, but each learner is different in their gain, collection is time consuming. Sometimes a testimonial comes; or feedback on survey (10% returns).

Learner: focus on listening, uses links from Twitter, wiki, etc. doesn’t login to LMS for 3 months.  Gets a reminder call, and they have been completing activities on recommended resources.  We have no data on language gains.

The change I would like to see (challenge)

  • Funders, understandably, want ‘proof’ of learning gains, reliable/valid measures of project impact on clients.
  • Our organization reports primarily on learner activity (number of hits, logins, hours in LMS, number of calls, journal reflections...) as measures of activity or impact.  Learning gains are reported in learner testimonials or feedback surveys that are ‘only’ self-reflections of learning.
  • As I move through this OLDSMOOC, I would like to understand how designers are assessing learners in open environments.
  • I would like to have more examples/ ways to collect data on learner achievement.
  • I would also like to be directed to articles, books, research on assessment of SDL

I have a list of new assessment activities for self-directed learners.

I have a bibliography of literature/research reflecting the good, bad and ugly of SDL learning gains.

I have innovative ideas to change the current learning options provided to learners to better meet their learning needs and at the same time the needs of funders!  (True dream?)

How you might go about bringing that change

Tackle the problem:
  • What do I do/know: reflect on current assessment practices (strengths/weaknesses)
  • What does group know: ask for feedback on current assessment; ask for new ideas; observe/question assessment choices in other projects.
  • Literature review (maybe not that extensive at this point but something to inform assessment choices...)

Community circle/cloudscape of assessment activities, models, approaches

Wiki of literature

Extra content

Embedded Content