|
 |
Ideas for a purely ling. CEFR-like scale? | ||
Tags: Placement Test | ||
Share with:
| ||
 Language Learning Forum : General discussion |  | |
| Medulin Tetraglot Senior Member Croatia Joined 4669 days ago 1199 posts - 2192 votes  Speaks: Croatian*, English, Spanish, Portuguese Studies: Norwegian, Hindi, Nepali | Message 9 of 35 25 June 2013 at 2:06am | IP Logged | ||||||||||||||
|
CERF scale is a result of EU-bureaucracy, that's all.
Thank God it's not used outside of Europe. So, one cannot say: My level in Brazilian Portuguese is C1, or my level of Argentine Spanish is about B2. The methodology is different, these two are more like real language tests and not cognitive/IQ tests advanced CERF tests can be: Brazilian Portuguese certificate (CELPE-Bras): http://en.wikipedia.org/wiki/CELPE-Bras ''CELPE-Bras (Certificado de Proficiência em Língua Portuguesa para Estrangeiros or Certificate of Proficiency in Portuguese for Foreigners) is the only certificate of proficiency in Brazilian Portuguese as a Second language officially recognized and developed by the Brazilian Ministry of Education (MEC). '' Argentine Spanish certificate (CELU): http://es.wikipedia.org/wiki/Certificado_de_Espa%C3%B1ol:_Le ngua_y_Uso ''The CELU certificate has been officially recognized and approved by the Ministries of Education and of Foreign Affairs of Argentina. At the international level, it has been recognized by the Brazilian and Chinese governments. '' Edited by Medulin on 25 June 2013 at 2:25am 2 persons have voted this message useful | |||||||||||||||
| mrwarper Diglot Winner TAC 2012 Senior Member Spain forum_posts.asp?TID=Registered users can see my Skype Name Joined 5227 days ago 1493 posts - 2500 votes Speaks: Spanish*, EnglishC2 Studies: German, Russian, Japanese | Message 10 of 35 27 June 2013 at 7:52pm | IP Logged | ||||||||||||||
While any scale / level system eventually develops an examination system around it, I hoped we could ignore that for a while. Anyway, a quote from the only link above that has some information: "[...] Posibilita estudiar en universidades y trabajar en puestos que requieran el uso efectivo de la lengua, ya que entre otras cosas mide la capacidad de desempeño en este tipo de situaciones. [...] Se acreditan dos niveles de desempeño: intermedio y avanzado. Ambos dan constancia, en distinto grado, de la capacidad lingüística efectiva del hablante extranjero para actuar en situaciones cotidianas, laborales o académicas [...]". This suggests the CELU-derived levels are as heavily cognitive / academically based as the CEFR ones, so you may want to elaborate on how/why this is not the case.
For the benefit of others who may not be so aware of shortcomings in scales and examination systems. For my own benefit in saving efforts -- the more discussion takes place and the more arguments are given here, the more I have to base my reasoning on, and the more I have in my arsenal, i.e. the less energy I have to spend in coming up with stuff the next time I find myself involved in any debate regarding the subject. I love recycling ;)
That, as most numbers in SLA, is ridiculous, because it would depend on language at the very least. Regarding the addition of "can do" statements, it seemed like a good idea the first time I read it, but now I feel it would make the scale both less academic AND less precise. We'd have to refine that quite a lot. I also liked first what you proposed in the 'Average Joe' thread about higher-dimension scales, but where's the limit? 4 axis? 5? It's clear if we want to devise something better than the linear CEFR scale, we must lose some linearity (sense of absoluteness of levels regarding if people are better/worse than others), but at the same time, we probably want to make it as simple as possible so it makes sense for as many people wanting to give it a go.
Thank you, Juan, for such kind words. Since you actually went on to comment, I'll take them as being a bit tongue-in-cheek :)
I agree with you, if only partially, but I think you're mixing up stuff on certain points so let's see how well I can express this... So, first things first, let's get this out of the way: of course being a native X or not is irrelevant to how proficient you are in Xish, except in terms of being exposed to the language for (way) longer and how much you could (or didn't) benefit from it. Do lofty thought and cultivated language really go hand in hand, as the Roman proverb goes, or as you say, give voice and tangible form to each other? For foreigners, it is more often than not the case where people may think at whatever high levels and still not be able to express themselves properly. I am C2 certified, thus I obviously can think at that level. Can I express myself at that level? Depending on language, the answer ranges from 'of course' to 'no way in hell'. Does the fact I flunked a B2 German test mean I cannot function at C2 level? Thank God that's only true in German. The thing is, there are people who, as natives, can't function beyond some point of the CEFR scale (presumably in any language, does anyone think you could get TL C2 certified and flunk an NL C2 test? Open a new thread!), and still do better in their native X language than people who theoretically operate at higher levels but have all sorts of problems with X. Thus this must be covered by any language level scale that aspires to be truly universal, unbiased, etc. In real life there are simpler solutions to assess what people *could* do -- if a guy has a serious degree and is A2 in a foreign language, he surely has the potential to go all the way up to C2... someday. Still, this is bringing in again non-linguistic stuff, which was something I wanted to avoid even if only as an experiment. I'll post some stuff I'm thinking about when/if it matures a bit more. Please feel free to keep posting here meanwhile. Edited by mrwarper on 27 June 2013 at 7:57pm 1 person has voted this message useful | |||||||||||||||
| s_allard Triglot Senior Member Canada Joined 5431 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 11 of 35 28 June 2013 at 3:32pm | IP Logged | ||||||||||||||
I disagree with the idea that the academic bias of the CEFR C levels is a "flaw." Quite the contrary, it was designed exactly for an academically biased audience; this is its strength. If you do not aspire to attend university in the EU or apply for membership in certain professional societies, you have no need for the CEFR C levels. I wonder if there are actually any situations where a C2 is a requirement. But to come back to the theme of this thread, I think that most observers would agree that all language proficiency evaluation systems are based on a profile or model of ideal speakers. Let's take for example the speech of the campesinos mentioned in the original quote of @nonebi. Let's take a sample of good speakers and we use their speech as the reference for developing our system of evaluation. Since these campesinos are probably illeterate, we do not test for reading or writing ability. The test is purely oral We have to do some descriptive linguistic work because some of the grammar and vocabulary is probably non- standard and contains many dialectal features. At our highest level, let's say Campesino C2, what are we testing for? With reference to the needs of @nonebi, we are looking for fluency, grammatical accuracy and vocabulary sophistication. How can you do this without regard to content? Our campesinos may be illiterate but they have culture. There is certainly a whole world of knowledge of agriculture, nature, weather, medicinal plants, folklore, etc. that are expressed in their language. Some campesinos are more knowledgeable than others and can speak in a more "sophisticated" manner, i.e. render fine distinctions. How can one speak a language without knowing any of this? Can one speak a language perfectly and have nothing to say? Can you have a wide vocabulary about nothing? When @nonebi sits before the two campesino examiners, what is he going to talk about in order to demonstrate his Campesino C2 language skills? Are the examiners going to ask him to conjugate verbs, complete sentences with the right words or give synonyms of certain words? I doubt it. Edited by s_allard on 28 June 2013 at 4:40pm 2 persons have voted this message useful | |||||||||||||||
| Cavesa Triglot Senior Member Czech Republic Joined 5010 days ago 3277 posts - 6779 votes Speaks: Czech*, FrenchC2, EnglishC1 Studies: Spanish, German, Italian | Message 12 of 35 28 June 2013 at 7:30pm | IP Logged | ||||||||||||||
|
mrwarper, I totally agree that it might be much better to have scales tied to each language. That system was already in place and had some shortcomings (the CV with English exam CAE, French B3 or A4 or whatever and another for another language was probably quite messy) but every centralization comes at a cost.
I think the line of thought of dividing both parts of the language skills and knowledge (pure linguistic as you say) might be the bug here. This discussion is aimed at one extreme, CEFR sometimes at the other. What if all that is needed to make a good exam is to semi-separately grade both? On the previous thread, I mentioned one thing that caught my attention about the language tests. The scale is always 1D so the exam tries to put all the various criteria into one result, one grade and sort the candidates on one line accordingly. This creates many nonsense comparisons and damages value of the exams. There are even employers (not few), who take a certificate just as a way to throw away a pile of applications without any certificate but test their applicants for the job themselves. The CEFR is too vague for them, despite being meant for employers, as many positions require some skills above all. Have you seen those 2D personality graphs? What if one axis would be that "situational competence" or how to name that cefr approach. The other could be "preciseness" or something like that and would take into account the mistakes and such things. And there could be third one "fluency" focused on how well you sound and how non cruciating it would be for a native to speak with you and to read from you. Such a result would tell much more about the skills, in my opinion. And therefore it could better serve to the purposes that are already covered in CEFR and could be used for others as well. Just a wild idea. 4 persons have voted this message useful | |||||||||||||||
| Serpent Octoglot Senior Member Russian Federation serpent-849.livejour Joined 6598 days ago 9753 posts - 15779 votes 4 sounds Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish | Message 13 of 35 28 June 2013 at 8:32pm | IP Logged | ||||||||||||||
|
Great idea, Cavesa! Then you should be able to pass C2 despite failing the cognitive part. Your employer will just know that you may have trouble with summarizing or presentations - which is no big deal for many jobs. s_allard, it's a flaw because if it's not needed for your job, your employer will have an inaccurate idea of your qualification. They see that there are six levels, and you didn't pass the highest one although your language knowledge is much better than that of most B2 learners.
Back to the separate grading system, if your employer speaks the language, they can also let you prove that you've learned a lot since you took your CEFR exam - you don't need to be a language teacher in order to rate this part accurately. Same with university - they can test your ability to summarize a scientific article without involving language profs (who could get it wrong anyway). Edited by Serpent on 28 June 2013 at 8:50pm 1 person has voted this message useful | |||||||||||||||
| nonneb Pentaglot Groupie SpainRegistered users can see my Skype Name Joined 4752 days ago 80 posts - 173 votes Speaks: English*, Ancient Greek, Latin, German, Spanish Studies: Mandarin, Hungarian, French | Message 14 of 35 29 June 2013 at 2:27am | IP Logged | ||||||||||||||
|
Even though it doesn't directly develop a scale as described in mrwarper's OP, I really
like Cavesa's idea and think it is both feasible and would be a huge step in the right direction. If I'm understanding it correctly, the test could reflect that my speaking section was fluid and relatively error free, but poorly structured and somewhat off- topic (for example, I reflected when I should have summarized or something).
s_allard, that image made me giggle quite a bit. I think, though, that your idea of needing more specialized vocab for a higher level is a bit burdened by the idea that a higher level needs to be like a C2 level, which demonstrates skill of some kind, whether it be cognitive skills or knowledge of native wildlife. Although they were campesinos in my example, it could just have easily been less educated people from the city, and there is plenty that those two groups have in common to talk about: Soccer, food, what they did on the weekend, their kids, their significant others, other people in general, jokes, their jobs, etc. Now, a person at a B2 level should be able to handle these subjects. But how well can they handle them? Is there still room to talk about and understand these things more naturally? That's more what I'm looking for. Edited by nonneb on 29 June 2013 at 2:28am 2 persons have voted this message useful | |||||||||||||||
| s_allard Triglot Senior Member Canada Joined 5431 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 15 of 35 29 June 2013 at 6:19am | IP Logged | ||||||||||||||
I think I'm getting a better idea of what @nonneb is looking for. It reminds of a common observation among language learners. People often have a relatively good grasp of the mechanics of the language but have great difficulty sustaining native-like speaking informal interaction especially with less educated native speakers. This would include things like being able to use slang, make jokes and small talk Am I correct? If that is what we are talking about, I want to take issue with the assumption that "a person at a B2 level should be able to handle these subjects." The mistake here is the assumption that B2 corresponds to a less educated group of speakers and C-level is for the more educated. So we would require a C2 for a discussion in a university classroom but a B2 for sitting around a table drinking beer and shooting the breeze about mundane things. I disagree totally. As anybody who has had to work with the people that @nonneb describes can attest, this kind of speaking interaction is actually often more challenging than chatting with university professors. Let's take the example of telling jokes. This is one of the hardest skills to master in a foreign language. Jokes or humour depend on a combination of three elements: 1) some kind of play on words and often some slang 2) cultural, social or historical references and 3) comedic delivery or the art of telling jokes, which may include the imitation of different accents. I strongly doubt that someone even at the B2 level can fully understand most jokes, let alone tell them. What has been correctly pointed out is that the CEFR C2 scale does not address this kind of linguistic skill. It was not designed for this. This is why I had suggested scrapping the CEFR model entirely while keeping just the taxonomy. My original attempt was clumsy I admit but the basic idea was to acknowledge that the language skills required to interact with "common" folk are sophisticated in their own way. So we now have a model profile that includes lots of slang, dialect, the vocabulary of popular topics like sports, family, jokes, drinking, maybe some illicit activities and other elements of popular culture. For lack of a better term, we call it the Popular Speaking Model in which we can have our A1-C2 scales. It's a purely oral model based on a sample of popular speakers. It's a suggestion and an alternative to trying to cobble together something out of the CEFR which was not intended for this situation. I should also point out that my approach is also predicated on the idea that one should not separate form and content (or the so-called cognitive skills) but that's another debate. Edited by s_allard on 29 June 2013 at 6:20am 1 person has voted this message useful | |||||||||||||||
| s_allard Triglot Senior Member Canada Joined 5431 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 16 of 35 29 June 2013 at 7:03am | IP Logged | ||||||||||||||
|
While I think that the idea of trying to assess purely linguistic skills is interesting and deserves some exploration,
there are some major methodological issues that I will not attempt to explore right now and bore everybody to death. But I do think that we may perhaps be going backwards here to the days of the grammar-translation model of foreign language teaching where the emphasis was on studying the language per se and not actually using it. In my mind, the most fundamental innovation of the CEFR model is the use of the "can do" or task-based model to establishing a somewhat realistic approach to how language is used by the target audience. What I find striking in all the CEFR technical materials is the absence of reference to specific language features. There is no attempt to link the levels with anything like pronunciation, vocabulary size, mastery of certain tenses, etc. It's not that these are not important. It's just that the concern of the CEFR designers is really how users are able to use these resources to accomplish certain tasks and not what resources they use. Now, if we attempt to focus purely on linguistic skills, we run the risk of no longer looking at how language is used but rather how much is recognized or known. For example, are we to systematically look at size of vocabulary, the mastery of verb forms and preposition usage, among other things. So it becomes how much language you know and not what you can do with the language. This takes us back 50 years. I may be wrong. There were just some thoughts. 1 person has voted this message useful |
|  | 
Printable version
|
| Copyright 2024 FX Micheloud - All rights reserved No part of this website may be copied by any means without my written authorization. |
