C. Tuesday - Activity 2 - Finding and Using OER

Exploring ways of assessing OER usabiity

Cloud created by:

Andreia Inamorato Dos Santos
8 December 2009

Today our focus will be on finding and using OER. Section 4 of the UNESCO OER Toolkit presents a list of repositories to help practitioners find licensed content.

The toolkit also suggests that there is a set of requirements that are necessary for the resources to be of acceptable standards to use, but what are they?

"It can be a frustrating experience trying to find good quality, appropriate and clearly licensed resources in the mass of existing material".

                                                                                                   ( UNESCO OER Toolkit)

Let's discuss some possible criteria to assess  an OER. Can we come up with some examples of what to look for in a OER  within this mass of existing material? 

Contact details: Andreia Santos - a.i.santos@open.ac.uk

Extra content

Professor Andy Lane's video on Supported Open Teaching

Andreia Inamorato Dos Santos
09:37 on 8 December 2009

Here's the link to the replay of the live talk on FM this afternoon: 'Working with OER: the experience of UnisulVirtual as a user-provider'.

by Murilo Matos Mendonca of the UnisulVirtual, Brazil. 

Andreia Inamorato Dos Santos
17:11 on 8 December 2009 (Edited 17:13 on 8 December 2009)

PPT Slides: Working with OER: the experience of UnisulVirtual as a user-provider, Brazil

Talk by Murilo M. Mendonca to the OLnet team.

Andreia Inamorato Dos Santos
23:06 on 8 December 2009 (Edited 23:08 on 8 December 2009)

Embedded Content


Karen Cropper
9:41am 8 December 2009

I've added some links which I sent to a friend of mine who teaches A level maths and when she heard about OERs asked me how could she find good things to use for her teaching.

Giselle Ferreira
10:14am 8 December 2009 (Edited 10:16am 8 December 2009)

Difficult question, eh? ;-)

The toolkit suggests some 'quality assurance processes':

  • 'curation'
  • editorial control
  • 'peer-review' via reviews and comments in the style of Amazon

There's also the notion of 'quality-based-on-reputation' (e.g. MIT OCW and OpenLearn!), but, as you say, Andreia, not much on criteria as such.

I wonder how grounded in 'context' quality criteria actually are ... This occurred to me as I was watching Andy's video: in distance learning, 'text' (in a very broad sense) is everything, but in other modes that's not really the case. Perhaps what counts as an 'excellent' OER in a classroom (or online learning in smaller scales and different models than what we do at the OU, in situations where there is the presence and leadership of a teacher or team) is very different than what counts as 'excellent' in ODL.  And there are all those issues that Giota brought into yesterday's discussion (we could call them 'cultural' with emphasis on the scare quotes). I suspect that 'context' is very important here.

As I say: difficult question! ;-)

Andy Lane
10:40am 8 December 2009

Everybody goes on about QA for OER without specifying what it is they are actually woried about. In most cases it appears to be concern that it 'is not good enough' educationally, but I also think there are large elements of presentation and technological format that come into that. So quality seems to revolve around academic integrity (is it authorative and current), presentation (it is free of typos, coughs, hesitations and not wrong colour of typeface for print disabled students) and format (can people access this asset readily).

Now the big issue for QA generally is 'is this product/process fit for purpose' which begs what the purpose is. Obviously when educational resources are developed for a target group of students then it gives focus to what needs to be done to assure quality in all three elements I mention. But if it is open to anyone the purpose is open ended as well and so what are quality issues that remain for users. At one level there are none as users have to make of it what they will depending on their own circumstances.  However at another level there are big issues for the 'publisher' as they are putting their authority on the line with users and peers. So qulaity is more about reputation of the 'authors' than fitness for primary purpose - an educational resource to learn from.

The issue with small scale synchronous teaching is the fact that most teachers do not have their teaching exposed to their peers and a wider learner audience which places new demands on their teaching practices and confidence in their abilities.

Giselle Ferreira
11:09am 8 December 2009

I like your QA elements, Andy (you say they're 3, but perhaps 'authoritativeness' and 'topicality' or 'currency' could be split into separate categories? anyway, it's a very nice list to think about) - one observation I'd make here, though, is that, no matter how much people like and talk about this 'global village' notion, we don't really have a single, worldwide 'academic community', do we? Teachers and academics have different preferences on theories, practices, etc. even in a single country/language, let alone across borders. Another point is that things that are considered 'authoritative' in one language don't always get translated and, so, don't always become part of what is considered 'authoritative' elsewhere. 'Authoritativeness' looks like another context-bound category ...

I totally see the point you're making about 'reputation of the authors' (which leads me to start thinking about the lot in terms of politics) and the extra demands that 'opening up' places on them - but then I often wonder about the need for a bit of humility in our trade?

Andy Lane
11:41am 8 December 2009

Giselle, I was thinking about authority being bestowed by your peers so in a sense that takes in cultural differences but of course in an open arena then there will be much greater contestation of the reporting and interpretation of knowledge. And to a certain extent I think there is a lot of humility in our trade which is why so many are reluctant to publish.

To come back to the issue of both pre publication and post publication review, the review process is about gaining and/or confirming authority and reputation and the question becomes one of how those not involved in the process view the product on offer. This then becomes an issue of trust, with trust variously earned and embodied in named publishers, universities or individual authors/academics over many years or through a unnamed collective effort such as Wikipedia which gains trust or reputation partly through weight of users or through rating systems. Such trust probably put very different weightings on the issues that the author does in terms of quality.

Giselle Ferreira
12:02pm 8 December 2009

I'm not so convinced that 'humility' is always what puts people off from 'opening up', Andy - but then I'd be better not go into armchair psychologising  :-(  

Lots of things to think about here, but to pick up on your thought-provoking points in the second paragraph:

  1. are you suggesting that 'authority', 'reputation' and 'trust' are perhaps separate issues? I'm not sure I can see this separation, so could you say a bit more if you have the time?
  2. I find Wikipedia a fascinating example, partially because I wonder about what the role of anonimity. If I understood you properly, you're suggesting that Wikipedia provides an example of an alternative 'QA' system (consistent with one of the processes suggested in the Toolkit, 'wisdom of the crowds' type of thinking) - I'm still not clear about how this system can work productively when there is no anonimity, thinking about how things can spiral downwards.

Giota, are you around to pitch in?

As I say, lots to think about!

Giota Alevizou
1:32pm 8 December 2009 (Edited 8:39pm 8 December 2009)

Fascinating discussion...I am got up with a lot of work at the moment, but I will try and contribute as well summarise some interesting topics later on tonight. For the moment, I am putting a couple of links that I think capture some of the issues regarding quality, anonymity, expertise and authority. So the first is a recent special issue on the 'epistemology of mass collaboration' by the journal Episteme (see also right).  Yes, this is philosophical, it relates to the links with Foucault's Order of Things, genealogy and subjectivity that we were discussing yesterday Giselle, but I think it is relevant within the context of OERs (esp. if you consider WP a massive OER ... in a sense it is).

The second relates to QA. I think WIkieducator's  quality assurance and review offer very accessible and straightforward guidelines.

Giselle Ferreira
1:38pm 8 December 2009

Brilliant intervention, Giota, I thought you'd have something or other to add ;-) The special issue of the journal looks fascinating!

Time, oh time ... :-(( 

Andy Lane
2:55pm 8 December 2009

  • are you suggesting that 'authority', 'reputation' and 'trust' are perhaps separate issues? I'm not sure I can see this separation, so could you say a bit more if you have the time?

    They are both separate and intertwined.  In general for institutions such as Universities the authority is part determined by accrediting bodies, governments etc who may regulate the use of the word University and what it does. However even given such authority they have to build up their reputation as such a institution, Different 'clients' of that institution will trust the institution partly on the authority and partly on their own dealings with it as well as the reputation coming from their dealings of others. So many ODL instiutions have authority bestowed upon them but poor reputations and varaible trust levels with different clients or stakeholders.

    With other organisations they may not have any bestowed authority and so rely more on building reputation and trust. With Wikipedia it feels like weight of use generally came through trust that in turn drove up reputation since reputation could not be helped by the names of contributing authors but by the rigor of the early contributors and mode of contributor responsibilities. However Wikipedia is not a good case because it is so popular and weight of users squeezes out alternatives being equally used just as Google dominates search.

    Now it is entirely different if we were to consider the position with individuals since this creates another dynamic between authority, reputation and trust which may be influenced as much by views on the nature of the person as their 'products'. Authority and/or reputation and/or trust may be gained solely by being at University X or appearing on TV programme Y or having controversial views or being within a small academic community.  

    Andreia Inamorato Dos Santos
    5:50pm 8 December 2009 (Edited 5:50pm 8 December 2009)

    Hi all,

    Like Giota, I have also been finding this conversation fascinating. There's lots to think about and you raise interesting and important issues.

    It seems that in terms of 'OER assessment' one item high up the scale is 'quality', and quality means  a combination of factors such as context (e.g. OER  for F2F or OER for ODL?); authority (who the author is, e.g. a renowned academic); currency (how up to date the OER is); presentation (free of typos, coughs, hesitation) and format (easy access).

    There's also the element of 'fitness for purpose' (which Andy rightly argues to be open-ended when the audience is not pre-defined) and the element of trust (either granted by an institution (e.g. PhD, MScs etc) or achieved by mass use (e.g. Wikipedia). This is all very complex and indeed your discussion is exploring some of these complexities - thank you for that.

    Now I ask: if we think of a scenario in which the practitioner is not renowned (e.g. a teacher at a school/university who has the minimum necessary qualification to teach); the institution is not renowned either (e.g. a new institution in the market, so common in developing countries); the OER developed by the teacher is audience-targeted (e.g. within his/her community) and the element of trust is not built due to the reasons above - how can this OER be assessed?

    Besides taking 'context' into consideration, what is the role of what is actually 'being taught' - the 'knowledge' in the content? Are we not taking for granted the role of the tutor in being able to assess the suitability of the material in terms of what is says, what it brings to the learner?

    I agree that people like clean, well-produced content but then it means the OER world will always be depending upon a lot of institutional investment to support content production rather than growing organically through the genuine participation of educators within their own limitations and resources available.

    Am I too much of a dreamer?


    Giota Alevizou
    9:40pm 8 December 2009 (Edited 9:44pm 8 December 2009)

    Oh my gosh, even more interesting points you raise Andreia. I will try to pick a couple, that relate to my view of threating OERs not only as resources, but also as educational media genres, and hence looking at the processes of mediation (theoretically and methodologically) is important:

    a) practioner/instituional recognition or reputation: there's certainly an issue about that, which I think has to do with what is the purpose or the range of purposes around OER use. If a user is an autodidact and wants to learn through an accredited instituion and also get some some accrediation through this kind of informal learning, then maybe s/he will try to find resources that come from mainstream, or established instituion rather than wikiversity or p2pu. Then again if the 'learner', 'surfer', user is up to some ideas to brainstorm for the design of a course, granular/modular OERs, clips, activities maybe more important rather the actual recognition. I think there will be a lot variation if: 

    • the purpose were to to 'translate' and/or 'adapt' material in an other context (mediating the attribution and reuse);
    •  expand learning through OERs and course materials;
    • credit learning through OERs...just to mention a few co-relations... Interestingly as Dave Cormier said, 'OERs will be the dictionaries of our time' Can they be?

    There are inscriptions about use and interactive audiences here that have not been accounted fully I think in the OER literature...and certainly the purpose/use(r), and f2f or ODL context are very important to flesh out further.

    a) 'the OER developed by the teacher is audience-targeted (e.g. within his/her community)'

    Yes there're certainly this element, esp. in f2f conventional university contexts. But as universities (in the west) at least become increasingly more, global instituions. One thought then is, how to attach more audience/pedagogical context in OERs that are about a specific discipline and specific level and couple that with these teacher communities? can this work in similar ways that scholarly communities and associations work? will some disciplines more prone to adapt to this mentality than others?

    Your last two points Andreia, raise the issue of sustainablity against the notions of motivation, serendipitous participation, tensions between codified and tacit knowledge, knowledge as content against knowledge as a process. Lab space tries to mediate transparently the process of learning, I think..but again takes a lot of motivation and lot of media, skills literacies...

    Finally, are instituional strategies and/or visions (often top up) that drive OER development from traditional universities enough to mobalise a shared vision among academics to share their articulated knowledge for a pedagogical purpose enough?  Do 'alternative' OER communities differ in so far they have a lot of enthousiasts that seek to mobalise individuals towards a common goal? And in what ways? What sort of incentives are needed for anyone to contribute/share?


    Giselle Ferreira
    10:29pm 8 December 2009

    and yet more great stuff for thought: great questions, Andreia and Giota (I like the idea of looking at the 'processes of mediation'!)

    I don't think you're necessarily a dreamer, Andreia (though, without dreams, things are quite dull, aren't they?) ... I was thinking about the specific context you describe (QA of a particular 'type' or OER produced by a particular 'type' of contributor in a particular 'type' of institution) and issues of engagement and community development, and playing with words in my head, this occurred to me: in that particular case, would perhaps the 'assurance' part of QA more important than the 'quality'? I reckon Giota, in her mentioning 'motivation' and a good collection of quite profound questions related to engagement, followed a related road. Once again today I'm wishing I had more time to think and discuss ...

    I'll only add at this point (it's late but tomorrow my day is taken with 'stuff' and I'm not sure I'll be able to contribute, though I'll try) that, when I was talking about 'context' I was thinking of something broader than mode of model of education. I was thinking about 'context' as everything and anything that constitutes the 'background' against which things make sense.

    If I may, I'll rescue from my memory an example used by a teacher I had many years ago. He was trying to explain to a bunch of young and noisy music students the importance of 'silence' in music, and he used lace as an example. He said that the patterns on the lace are made exactly by the bits that are 'cut out', so to speak, which makes the fabric and the empty space inseparable in making the lace what it is. That's kind of the relationship I tend to see between 'ideas' ('values' etc) and 'context'.

    Hope tomorrow's discussions are even better than today's!


    Patrick McAndrew
    3:37pm 9 December 2009

    I was looking at an interesting report from a project called OERTN (http://oertn.eun.org/) that is linked to the SchoolNet project. In brief they were looking at what teachers in several countries found when they tried to use OER.

    They came up with a definition for when OER "travel well"  across country boundaries. In OLnet a sub-project led by CMU is looking at this in relation to reuse but across levels and different sectors of education in the US.

    They also took an approach of allowing teachers to select resources regardless of source demonstrated that the formal label of "OER" is less relevant to the user than it is to the provider. Both clear copyright release (CC) and the ability to remix element considered essential characteristics by some appear to matter less to the teachers.

    Seeing material as in general able to be reused across boundaries also resonates with findings and discussions and it is great to see this quantified.


    Contribute to the discussion

    Please log in to post a comment. Register here if you haven't signed up yet.