Briar Jamieson: Measuring Outcomes/Learning Gains in Open Access Language Learning Environments

Cloud created by:

Briar Jamieson
15 January 2013

My Dream

I have come to the OLDSMOOC to understand how 21st Century learning designers are expressing and measuring learning outcomes in open educational environments.  My hope is to immerse myself in the OLDSMOOC activities and learn from participant experts/explorers by observing/questioning decisions about activities and the indicators/measures/assessment of learning gains.

It is my dream to find ways to better express, collect, measure, report the learning gains of self-directed language learners registered with my publicly funded organization.  

My situation (context)
Overview:
  • not-for-profit organization, providing language learning resources/support to Self-directed learners, 1600 registered learners, 300-350 active every month; w/2.5 FTE instructors
  • funding received from government to provide language learning supports/resources to newcomers
  • access to learning resources at no cost to learners everything online
  • activities for learners include: drop-in online classes, access to modules from Literacy to Intermediate, additional lessons, assignments, tests, community discussion board, 4 week courses, weekly tips, learners encouraged to develop Personal Language Learning Environments, access to an
  • most of these activities are self-assessed, answer keys, community discussion, journal reflections
  • instructors feedback on baseline assessments, guide to further resources, deliver synchronous classes, weekly tips, moderate discussion board and feedback on journal reflections
  • learners can pick and choose the content and activities they want to engage in; there is no requirement to complete
  • work completed within an LMS
  • learners have access to a learning portfolio within the LMS to show evidence of work but not a requirement
Example

Learner: focus on writing, completes only writing activities, receives feedback on writing and then leaves. We have examples of writing improvement, but each learner is different in their gain, collection is time consuming. Sometimes a testimonial comes; or feedback on survey (10% returns).

Learner: focus on listening, uses links from Twitter, wiki, etc. doesn’t login to LMS for 3 months.  Gets a reminder call, and they have been completing activities on recommended resources.  We have no data on language gains.


The change I would like to see (challenge)

Challenge:
  • Funders, understandably, want ‘proof’ of learning gains, reliable/valid measures of project impact on clients.
  • Our organization reports primarily on learner activity (number of hits, logins, hours in LMS, number of calls, journal reflections...) as measures of activity or impact.  Learning gains are reported in learner testimonials or feedback surveys that are ‘only’ self-reflections of learning.
Achieve:
  • As I move through this OLDSMOOC, I would like to understand how designers are assessing learners in open environments.
  • I would like to have more examples/ ways to collect data on learner achievement.
  • I would also like to be directed to articles, books, research on assessment of SDL
Example

I have a list of new assessment activities for self-directed learners.

I have a bibliography of literature/research reflecting the good, bad and ugly of SDL learning gains.

I have innovative ideas to change the current learning options provided to learners to better meet their learning needs and at the same time the needs of funders!  (True dream?)


How you might go about bringing that change

Tackle the problem:
  • What do I do/know: reflect on current assessment practices (strengths/weaknesses)
  • What does group know: ask for feedback on current assessment; ask for new ideas; observe/question assessment choices in other projects.
  • Literature review (maybe not that extensive at this point but something to inform assessment choices...)
Example

Community circle/cloudscape of assessment activities, models, approaches

Wiki of literature


Extra content

Embedded Content

Contribute

Karen Ferreira-Meyers
12:04pm 17 January 2013


I am very keen on developping this dream. As a language teacher I am trying to see what works best in the context I live and work in, namely Swaziland. I look forward to reading the resources people will post here and perhaps contributing (if time permits).

Briar Jamieson
2:40am 18 January 2013


Fantastic Karen,

I would appreciate any input for helping develop this project.  I have been feeling a bit daunted, but now feel that a litle application, focused thought and a little help from friends and we may have a product at the end.

Thank you!  

Briar

Sancha de Burca
11:29am 22 January 2013


Hi Briar,

Have you considered using blogs as evidence of/places to assess your learners? I've been working with these with face-to-face as well as distance (international) graphic design and televison production learners and have found them really useful. Learners could upload written work and images as well as audio and video, which might suit your learners. The blogs would provide evidence for those who want to track the results of your organsiation.

We've found that the blog users also raised their grades after starting to use blogs - tho I can't legitimately prove the link. We suspect it is that our learners are now working in a better, more productive process, especially posting things at the right time (rather than the night before submission!) which helped their thought proceesses.

Hope this is if use,

Sancha

Contribute to the discussion

Please log in to post a comment. Register here if you haven't signed up yet.