Evaluations and Outcomes: Organizing OER Challenges into Strategic Areas to Measure

Notes from the Evaluations and Outcomes Break-out Session

We asked participants to put one answer on a post-it note responding to this about their OER project:

• What are is your big challenge?
• What are you looking to know? 

People then had to move around the room and self-cluster with others whose response was similar.

Here are responses and clusters:

Learning Outcomes

Impact of OER in improving learning outcomes

How to measure extent OER offers competency, esp. for low income students, and developmental math

What do kids actually learn through participation, e.g., in interest-driven communities, and...

How to relate participation to school assignments?

How to meassure behavior changes of students, e.g., middle school students are safer and smarter in their internet behaviors, in becoming creators of digital media?

Example: PhET has measured student confidence and participation; asks student "what are you thinking?" when use OER? ; looks at peer2peer learning

Example: WGBH Teachers' Domain has video archive using open content from group of TV stations for students to make videos. What are the learning outcomes from using video? What frameworks or lessons should accompany the content? align to subject rubrics? does using video change reading scores? What are the impacts of students re-posting materials?

How are Users Using

Who, How, Usability

Does the level of openness make resources more valuable and effective? e.g., streaming vs. downloading

(continued in next comment) 

Amee Godwin
10:56pm 4 March 2009

How to prove our learners are learning?

Will my project make an impact so more learners wil want to use it over and over?

How to measure amount of reuse and remixing that's taking place?

How to meet the hugely diverse needs of different users?

OCWC: How are uses wanting to be employing our members' content and what can we do collectively to increase chances of success?

Teachers Roles

What motivates teachers to share their work with strangers?

How are participants in a project "developed" through the project period?

What is the impact of introducing OER to K-12 teachers who hold different teaching philosophies?

How are instructors adapting materials?

Changing role of Teaching

Collaboration. WHY?

Making Collaboration Easier

Grassroots collaboration - enhance problem-solving

Teacher reluctanct and lack of time

CCOTP: teachers who create additions to open material now more willing to post it

Students need to see teachers doing it - get students to post and fix content

Keep Toolkit, CMU - How is OER changing your teaching and learning outcomes?

Quality

Compare resources

Ownership

Control

How much openness is good, at what point does one decides the openness is right?

Accuracy, Vetting

Trust: How do you know what's good?

How do you know what it's good for?

How to use rich media to offer alternative options for different learning styles?

e.g. Phet: does do effectiveness testing through use

Big Questions: Research and how to measure

What sources to trust?

How to frame how OER is offered?

How to make reserach data transferable so that it can be pushed up and used by others?

Challenge: Matching research to others' questions

Test assumptions - develop baselines - Free, Remix, Collaboration

Amee Godwin
4:34pm 5 March 2009

Excellent way in which the sessions was managed...very interactive and I learned a lot whcih will be helpful for my own project.

Thanks..

frances ferreira
6:38pm 5 March 2009