Assessment of Intercultural Competences
Cloud created by:
26 November 2009
Some questions to discuss after the CEFCult kick-off:
How can you assess intercultural competences? Which are the criteria you can base assessment on?
Is there something such as expert assessment of intercultural competence? Or is it always self-assessment and reflection by the learner? Can peer-assessment play a role - if yes, which form does this take?
There are several proposed frameworks (INCA, YOGA, LEA, CILT, LOLIPOP, etc. - listed below) but none of them have become mainstream (or are endorsed by the Council of Europe). Which one can we adopt for the CEFCult project?
What form will the feedback we give learners take? Should we be focussing on cultivating an intercultural mindset in the learner?
Comment 1 by Ana Beaven
12:04pm 2 December 2009
Well, let's start here, for instance:
I have added the link to the National Occupational Standards that Inma had in Leuven. These are not a framework as such, and would be a basis for building a framework and then a tool - very interesting, but no time I think. Still worth reading, though.
I'm afraid I don't know Lolipop, perhaps Antje can tell us more about it - what I do know is that it is within the European Language Portfolio, a line I would follow, as it seems a good idea for any assessment to be included in the learner's portfolio if so wished.
As for INCA, in the TOOLS section, there are a number of scenarios and role-plays used for assessment of intercultural competence that could be a starting point for our project.
Comment 2 by Jan Van Maele
Jan Van Maele
4:28pm 3 December 2009
I've been looking at INCA today as a potential counterpart of the CEFR for providing feedback on intercultural competence in CEFcult.
It struck me that their definition of 'communicative awareness' (in the assessor variant of the framework) is a very dynamic one. Together with 'behavioral flexibility', 'communicative awareness' constitutes 'adaptability'. Let's call that the Skills strand of intercultural competence in the assessee framework. The Knowledge and Attitude ('openness') strands complete that variant of the framework.
Turning to Kamakshi's question what sort of feedback could be given to learners in CEFcult, what would you think of such skills-oriented descriptors that refer to the manifestation of intercultural strategies in communicative encounters? Wouldn't the following INCA-descriptor complement the CEFR-descriptors in contents and form, you think?
"Uses a limited repertoire of strategies (metacommunication, clarification, simplification) to solve and prevent problems when interacting with a [...] speaker [from another cultural background]." (JVM: INCA writes 'non-native speaker')
Comment 3 by Lutgart
12:54pm 5 December 2009
Dear Jan, SEE also (with thanks to Maija Karjalainen for the reference)
Vulpe, Thomas, Daniel Kealey, David Protheroe and Doug MacDonald. 2000. A Profile of the Interculturally Effective Person. Ottawa: Centre for Intercultural Learning, Canadian Foreign Service Institute.
The Canadian Foreign Service Institute has developed a set of competencies to identify Interculturally Effective Persons (IEPs):
1. Adaptation skills. IEPs have the ability to cope personally, professionally and in their family context with the conditions and challenges of living and working in another culture.
2. An attitude of modesty and respect. IEPs demonstrate modesty about their own culture’s answer to problems and a respect for the ways of the local culture, are humble about their knowledge of the local context and are therefore willing to learn and consult with locals before making conclusions on certain issues.
3. An understanding of the concept of culture. IEPs have an understanding of the concept of culture and the pervasive influence it will have on their life and work abroad.
4. Knowledge of the host country and culture. IEPs possess knowledge of the host country and culture and continually try to expand that knowledge.
5. Relationship-building. IEPs possess good relationship-building skills, both social/personal and professional.
6. Self knowledge. IEPs have knowledge of their own background, motivations, strengths and weaknesses.
7. Intercultural communication. IEPs are effective intercultural communicators.
8. Organizational skills. IEPs strive to improve the quality of organizational structures, processes and staff morale and promote a positive atmosphere in the workplace.
9. Personal and professional commitment. IEPs have a high level of personal and professional commitment to the assignment and the life experience in another culture
(Vulpe, Kealey, et. al 2000). Selection of extended description of IEP-criteria for expatriates ....
Comment 4 by Ari Huhta
8:53am 11 December 2009
I just added a link to the LanQua project (www.lanqua.eu), which has an Intercultural Communication subgroup that has also done work on defining that area. See their Year 1 report. They will also soon publish more stuff as soon as they have finalised their new / final report.
It will be interesting to read more about the Lollipop project as they attempted to create a CEFR linked, 6-level descriptions of intercultural competence (IC). It was mentioned in the meeting that we do not necessarily aim to create a scale (or set of subscales possibly) for IC that is aligned with the CEFR scale on one-to-one basis, although if one could do that it would obviously be a huge bonus. I feel however that it will be very difficult to do that because there may not be enough in the CEFR about IC to help us validate such claims. (I haven't yet studied Lollipop so don't know if and how they've approached the linking / alignment question.)
Secondly, I've got a feeling the IC may be one of those competences, abilities, etc that cannot be divided into very many levels, i.e., people could not differentiate between six IC levels. If we could even define (and importantly, agree on) 3 or 4 levels, that would be great in my view.
At the end of the day, the number of levels you can split something into is a question that can be studied empirically, and I'd be most interested in participating in research into this at some point. Also, it may be possible to link a shorter, say a 3-level IC scale with the 6-level CEFR scale, but again this is difficult to say without actually studying it.
Comment 5 by Jan Van Maele
Jan Van Maele
9:27am 15 December 2009
Re-reading what I posted earlier, I think that I was on my way to proposing (but not quite getting there) that we focus on 'intercultural communicative competence' in the CEFcult assessment model.
Remember, in WEbCEF we didn't use the full CEF either. WebCEF (and CEFcult) do not claim to support the assessment of every aspect of language proficiency, but center on the assessment of spoken language. Hence, we selected a number of descriptors drawing from the scale for overall oral production and from the table for qualitative aspects of spoken language use.
Similarly, CEFcult is not about assessing intercultural compentence in general, but, as I see it, intercultural communicative competence. Accordingly, we will have to select a number of descriptors (from INCA or another framework) that stand a good chance to be manifested by the speakers in the performance of the speaking tasks that we will define for them.
Let's change the name of this cloud to Assessment of Intercultural Communicative Competence, and make our task (slightly) more feasible.
Comment 6 by Ilpo Halonen
4:37am 10 February 2010
I added 19 bookmarks for CEFcult in http://delicious.com/ilpohalonen. The seacrh words were intercultural competence + assessment. Have a look on them if you think there might be something of interest for you.
Do you think it would make sense to start a CEFcult network for collecting and sharing more bookmarks of CEFcult partners in Delicious.com?
Comment 7 by Pirjo M Harjanne
Pirjo M Harjanne
1:39pm 2 March 2010
we already sent this by email last week, because we didn't manage to send it through Cloudworks. Hope this comes through and doesn't fall through.
It has been very inspiring to follow all the comments you have made on CEFcult and its background rationale. Unfortunately, we have not been able to contribute to the discussions so far because of major university reforms here in Finland (been overpressed by the new structures being constructed). We have, however, discussed the topics between ourselves and also reflected on what would seem important for the project.
So once again—many thanks to all of you for comments that have opened up certain ideas and advanced the total process we are facing at the moment. Special thanks to Jan for concretizing the links between CEFR and a number of other intercultural assessment scales in general and the CEFR proficiency levels in particular.
We would like to raise the following issues:
a) scenarios? Discussion still open, we believe. In our context, it would certainly mean higher education. We might also like to link them to our KIELO project (FL teaching, studying and learning in Finnish language classrooms). In this framework, we plan to develop concrete scenarios. A further exchange of ideas highly welcomed.
b) intercultural communicative competence vs. oral language proficiency: why not clarify the starting point for everybody? Let’s adopt Jan’s idea of emphasizing intercultural communicative competence in lieu of intercultural competence as such (too large, too vague for us). Another matter is whether we need to distinguish between linguistic tasks and cultural tasks. To our way of thinking, they are inseparably intertwined. Still, we would like to see intercultural communication aspects more highlighted than linguistic ones in this project. Wouldn’t it make our lives easier and the project implementation more realistic and easier?
c) CEFR: communicative competence = intercultural communicative competence – we believe that communicative competence in CEFR in fact stands for intercultural communicative competence. If we accept this interpretation, links to CEFR might be easier to be tied. Cf. the following article:
Coperías Aguilar, M. J. (2009). Intercultural communicative competence on the context of the European higher education area. Language and Intercultural Communication, 9, 242–255.
d) “tasks that force speakers to apply certain communication styles”? This is in one of the texts. We are slightly uncertain whether this is wise and how it could be carried out. Wouldn’t it be better to take advantage of “natural, authentic, genuine, realistic” episodes to show, for instance, the high/low context intercultural features in communication and in the interlocutors’ behaviour?
e) INCA ok. Looks interesting and functional to us. (Seppo has asked a group of Japanese university students to analyse the different components of the project, just in order to get some more “intercultural” interpretations of that project.)
f) Multicultural Toolkit http://www.awesomelibrary.org/multiculturaltoolkit.html
This is a US-based kit of different scales and background classifications. Even if biased towards US, it might give us a few insights of what to ask and what not.
g) Vulpe et al. A Profile of the Interculturally Effective Person -- This article, together with a few others, invites us to share with you some articles that also deal with similar classifications and criteria for intercultural communication from different perspectives, including the European Higher Education Area and CEFR. Please have a look! (cf. Trompenaars, Hofstede, Byram et al. >> not all aspects >> focus on intercultural communication >> which criteria relevant and possible to evaluate?)
Let us remind all of us of Byram, M., Gribkova, B. & Starkey, H. (2002):
The components of intercultural competence / Intercultural attitudes
- Curiosity and openness, readiness to suspend disbelief about other cultures and belief about one’s own.
- > a willingness to relativise one's own values, beliefs and behaviours, not to assume that they are the only possible and naturally correct ones
- > to be able to see how they might look from an outsider's perspective who has a different set of values, beliefs and behaviours.
The components of intercultural competence / Knowledge
- of social groups and their products and practices in one’s own and in one’s interlocutor’s country - includes knowledge about how other people are likely to perceive you
- of social and individual interaction
- No teacher can have or anticipate all the knowledge which learners might at some point need > teachers can acquire information about other countries together with their learners
The components of intercultural competence / Skills of comparing
- as important as attitudes and knowledge
- Intercultural speakers need to be able to see how misunderstandings can arise, and how they might be able to resolve them
- > they need the attitudes of decentring but also the skills of comparing: by putting ideas, events, documents from two or more cultures side by side and seeing how each might look from the other perspective, they can see how people might misunderstand what is said or written or done by someone with a different social identity
The components of intercultural competence / Skills of discovery and interaction
- ability to acquire new knowledge of a culture and cultural practices and
- = to know how under real-time communication and interaction to ask people from other cultures about their beliefs, values and behaviours, which because they are often unconscious, those people cannot easily explain
The components of intercultural competence / Critical cultural awareness
- an ability to evaluate, critically and on the basis of explicit criteria, perspectives, practices and products in one’s own and other cultures and countries
- to be aware of one’s own values and how these influence the views of other people's values
- It is not the purpose of teaching to try to change learners values, but to make them explicit and conscious in any evaluative response to others.
These are just some of the reflections we have had over CEFcult and intercultural communication assessment, but we’d like to air these before you discuss INCA in a couple of days’ time.
Looking forward to your comments and reactions… With warm intercultural regards from Helsinki, Pirjo and Seppo
Comment 8 by Pirjo M Harjanne
Pirjo M Harjanne
1:48pm 2 March 2010
Dear all again,
we added a number of academic references separated with spaces, but unfortunately the system only allows one reference at a time. Sorry about that.