|
 |
Ideas for a purely ling. CEFR-like scale? | ||
Tags: Placement Test | ||
Share with:
| ||
 Language Learning Forum : General discussion |  | |
| hrhenry Octoglot Senior Member United States languagehopper.blogs Joined 5131 days ago 1871 posts - 3642 votes  Speaks: English*, SpanishC2, ItalianC2, Norwegian, Catalan, Galician, Turkish, Portuguese Studies: Polish, Indonesian, Ojibwe | Message 25 of 35 30 June 2013 at 4:12am | IP Logged | |||||
If we're talking about jokes that adults tell each other as opposed to children's jokes and riddles, I think I would disagree with this. So, so many jokes rely on cultural context (especially pop culture) that a B2 learner still wouldn't be aware of. R. == 1 person has voted this message useful | ||||||
| Juаn Senior Member Colombia Joined 5346 days ago 727 posts - 1830 votes Speaks: Spanish* | Message 26 of 35 30 June 2013 at 4:54am | IP Logged | |||||
|
What I meant was, language develops and is learnt through meaningful processes. You come into contact with and master the language employed in philosophy for instance by doing philosophy. How would you learn or even encounter forms of a language used in contexts of which you take no part? One has no word for something one has never encountered or thought. Thus it makes no sense to examine for the kind of proficiency achieved by extensive reading and engaging in particular topics while explicitly denying acquaintance with them.
Incidentally, this view of language explains why comprehensible input and the methods championed by Krashen work so well, especially reading, whereas explicit grammar study produces such dismal results. 1 person has voted this message useful | ||||||
| Serpent Octoglot Senior Member Russian Federation serpent-849.livejour Joined 6598 days ago 9753 posts - 15779 votes 4 sounds Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish | Message 27 of 35 30 June 2013 at 5:09am | IP Logged | |||||
1 person has voted this message useful | ||||||
| s_allard Triglot Senior Member Canada Joined 5431 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 28 of 35 30 June 2013 at 4:48pm | IP Logged | |||||
|
I agree totally with @Juan on this. This is of course why I proposed developing an alternative content-rich CEFR
exam rather than a purely linguistic exam. This is not to say by any means that the idea of a purely linguistic evaluation is a waste of time or useless. It's just that I think that there are fundamental methodological issues, namely testing for form without content, that make the project unworkable. 1 person has voted this message useful | ||||||
| mrwarper Diglot Winner TAC 2012 Senior Member Spain forum_posts.asp?TID=Registered users can see my Skype Name Joined 5227 days ago 1493 posts - 2500 votes Speaks: Spanish*, EnglishC2 Studies: German, Russian, Japanese | Message 29 of 35 30 June 2013 at 7:50pm | IP Logged | |||||
|
Edit: 10-paragraph text wall ahead. Sorry again...
I think we're dangerously bordering the analogue of what incapable psychologists do so often that they often make people forget about the ones who actually do some valuable work -- they all go "oh, the human mind is so complex we'll never really understand it. So let's forget about that (and let's have a look at this vision of how it should work)" and then prevent anything of worth from being done around them. Please let's try to avoid that. First, the false dichotomy. CEFR "can do" statements leaves out specific details about language use that are [in]correct. That doesn't mean use of "incorrect language" is deemed as unimportant by those who made the CEFR. Such details are language-specific and thus necessarily out of scope for any "universal" scale. However, they must be somehow taken into account by any language-specific level assessment procedure, for the simple reason that there's no such thing as "can do ... using incorrect language". No good scale can measure what one "can do" forgetting about how one uses language to do it, nor dwell on how good one's language is forgetting [s]he cannot do what they're supposed to, because it is the language used what makes the task [im]possible to perform. Correctness, thus, must be examined twice: first, to deem any task [im]possible, and afterwards, to see how well it can be performed. Looking at correctness isn't any kind of backwards step from looking at "can do"s -- it's simply something that must be superficially examined first, and optionally examined in depth afterwards. That's why the more I read about it, the more I agree with the take that the CEFR is actually pretty good as it is, and it is particular examination systems built around it who are to blame when we see how the level assessment fails or exhibits glitches. However, I feel there's still a lot that can be said about how linguistic competence can and cannot be evaluated without entering language-specific land. Anything we try to devise to stay minimally real can't go with "declensions in the vacuum", or "talking perfectly about nothing", as the above clearly implies, and I'm certainly not proposing anything like that (please take this just as a reminder to avoid repeating myself --yet even more-- in the future). They can't be separated, so that's not the way to go. But what's wrong about particular CEFR level assessment procedures I've experienced as they purported to incarnate the higher CEFR levels? After some thinking I must conclude it is the contents I've seen used all too often as base for the tests. It is true, as Juan says, that cognitively demanding content practically imposes the use of cognitively demanding language. So are we onto something here? I think so, and a key point while at that, for the trap I've mostly seen examination systems fall into is thinking in the opposite direction -- that anything that is somehow demanding is fit for consumption of all learners at some level. And that couldn't be further away from being true. The problem with "academic" materials goes through tree stages: they look CEFR-C1/C2-"kosher", which builds up expectations; then I suppose they must be readily available in heaps for the academic types like test makers, so they're tempting to use; and finally, they fail the reality check: they're hardly apt to test everybody. Why? I'll get to that. A lot of "literary" stuff (not the majority, I must hope) and "philosophy" suffers from similar problems and we still have see these crop up in 'language ability' tests all around. Juan asks "How would you learn or even encounter forms of a language used in contexts of which you take no part?" and I answer: except for the necessary neologisms such as those you bump into when reading about medicine, you should not encounter "forms of a language" that are particular to contexts you don't partake in... in a test. Think what's the real relevance of how well a candidate does handling something they had never bumped into before, and they'll probably never see again for exactly the same reasons, no matter how ignote? If in doubt, ask them candidates about it ;) On top of that, I firmly believe that if a not too specialized text requires from an educated and intelligent reader/speaker anything more than the occasional trip to the dictionary (because it's about something he doesn't know well) to understand / make heads or tails of it, then it is, plainly and simply said, rubbish. But that's another story. We all know how some people are hell-bent on looking cleverer than they really are, and one of the most prominent tell-tale clues is their use of language in convoluted and purportedly obscure forms that hide just for long enough how little worth there is in what they have to say (politicians or post-modernists, anyone? -- and don't think for a split second I'm leaving out many academic types here :) It is a free corner of the world, so to each his own, but I think letting such crap slip into examination systems aimed at the general population (within the expected level) is a most dangerous mistake that must be avoided very carefully. It is of course OK, and perhaps even advisable, to base tests on legal materials and texts if candidates are expected to speak 'legalese' instead of the 'regular form of the language' in the contexts where their level placement is relevant. Same goes for any specialized use of the language. So the important question remains, 'how to weed out stuff that is [demographically or otherwise] irrelevant from tests aimed at the general population at level X'? I think this is where we should mostly explore to come up with productive ideas regarding language level placement. As for my own, I have a somewhat simple idea, but I think it may be effective as a test for tests: Materials are always about something. Pick up somebody who is a suitable candidate for the level, and who knows *nothing* about the subject, and give him/her the test and a dictionary. If they can't do it, your test may be better off in a niche ;) Edited by mrwarper on 01 July 2013 at 10:58am 3 persons have voted this message useful | ||||||
| s_allard Triglot Senior Member Canada Joined 5431 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 30 of 35 30 June 2013 at 10:18pm | IP Logged | |||||
|
As I follow the thread, I wonder how big this problem really is. If I return to the original comment by @nonneb, it
is clear that the CEFR exam system in Spanish does not meet their needs. But what about the needs of the vast majority of people who do take the Spanish DELE exams? When we say that there is an academic bent, are we saying that the documents used are so specialized as to exclude large segments of the population? In French for example, there are around a dozen tests of proficiency in French. There are so many tests because they are used for different purposes.e,g, to obtain French citizenship, for admittance in a French university, for business or legal needs, for immigration to Canada, etc. I haven't seen all the tests; I am more familiar with the most common, the TEF and the TEFAQ that are used for immigration purposes to Canada. Interestingly, they give the results in terms of the A1-C2 scale. The format is typical: timed tests for lstening, reading, writing and speaking with an examiner. Since the same test is used for everybody, the various test items are not particularly academic or even difficult. The testing strategy for the reading and oral comprehension taks is to ask a very large number of questions in a given lapse of time. The number of right answers determines the level of proficiency. Obviously, these exams do not test for certain kinds of French, especially things like slang and popular speech. Nor is the language very specialized about French institutions, In fact, the material is rather bland, as to be generic as possible. This makes me think that our problem here is more one of test content and materials than anything else. Edited by s_allard on 01 July 2013 at 3:32pm 1 person has voted this message useful | ||||||
 Iversen Super Polyglot Moderator Denmark berejst.dk Joined 6704 days ago 9078 posts - 16473 votes Speaks: Danish*, French, English, German, Italian, Spanish, Portuguese, Dutch, Swedish, Esperanto, Romanian, Catalan Studies: Afrikaans, Greek, Norwegian, Russian, Serbian, Icelandic, Latin, Irish, Lowland Scots, Indonesian, Polish, Croatian Personal Language Map | Message 31 of 35 01 July 2013 at 11:38am | IP Logged | |||||
|
I don't have a particularly high opinion about philosophy, but let's take it as an example of academical fields in general . For me science and culture and history are the meaningful fields, and I learn languages from comprehensible sources about these themes rather than from literature about non-existing people who flirt or quarrel or kill each other according to the whims of the author. Besides my comprehensible sources are mostly written rather than spoken.
One relevant point is that I also deal with such themes in Danish, English and other languages so when Juan mentions "meaningful processes" I immediately think of linguistics, nuclear physics and the nomenclature of vertebrates rather than messy and irrelevant things like interpersonal relationships, fashion, sports or party politics. Sport magazines and details from the lives of American popstars or rap artists or sportspeopleis also low on my rating lists. To be sure, there are themes which are interesting in spite of being non-academic even for me – like the weather at my travel destinations, shopping for basic food items and getting through airports – so it's OK with me if knowledge about such themes is assumed by the authors of a test. And I cannot possibly have a problem with an academical bias at the higher levels of suchs tests because that would just make it easier, but I do see that other language learners might have a problem with it if their preferred meaningful processes take place in other spheres. In practice this is all an academical discussion for me because I won't ever have to take any language test again in my life (hurray hurray!) but you wouldn't solve all problems just be removing any vaguely academical from the tests. Just as you would do some boys in school a grave disservice by telling them to get out in the school yard and start fighting a'cause that's what all real boys are supposed to do. Any language has a common core shared by all native speakers, and any placement level test must test for skills within that area. The problem is that this area doesn't have clear boundaries, and any kind of specialized knowledge may be seen as relevant for some learners and not relevant at all for others. Even the choice of speech or writing as the principial vehicle for a test can be a problem. So let's just say that the CEFR scales have a certain bias, and you are tested according to that bias. You could propose another set of tests with a different profile, but then you would just be unfair to somebody else. Btw. I like Krashen's idea about comprehensible texts (written as well as spoken), but contrary to him I don't see grammar studies as something evil and superfluous. Edited by Iversen on 01 July 2013 at 11:51am 3 persons have voted this message useful | ||||||
| s_allard Triglot Senior Member Canada Joined 5431 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 32 of 35 02 July 2013 at 4:01pm | IP Logged | |||||
A small but important point here. The presence of a certain orientation or bias (I don't really like the term) in the current CEFR exams is not necessarily unfair to certain candidates. This orientation reflects the purpose or market of the test. Proposing another set of tests with a different profile woult not be unfair to other people. It's all a matter of the purpose of the test. In the case at hand, it is a matter of designing a test that would reflect the needs of users like @nonneb who are poorly served by the current CEFR exams. 1 person has voted this message useful |
|  | 
Printable version
|
| Copyright 2024 FX Micheloud - All rights reserved No part of this website may be copied by any means without my written authorization. |
