|
 |
Average Joe/Jose takes a level test | ||
Share with:
| ||
 Language Learning Forum : General discussion | ( Topic Closed) |  |
| mrwarper Diglot Winner TAC 2012 Senior Member Spain forum_posts.asp?TID=Registered users can see my Skype Name Joined 5035 days ago 1493 posts - 2500 votes  Speaks: Spanish*, EnglishC2 Studies: German, Russian, Japanese | Message 49 of 80 24 June 2013 at 12:00pm | IP Logged | |||||||||||
|
This is getting a bit tiresome. Because of the nature of CEFR tests, you're right one can't cheat for some parts of the test, but you can clearly cheat for others, period.
I've been a candidate, a tutor and an examiner in different disciplines for years, and *I* have seen it all. Test-making involves more recycling than I'd like and many if not most candidates know this. Many prospective CEFRers prepare monologues and stock phrases for ALL subjects seen in 'past papers' collections and memorize them to use whenever they see an opportunity. While this is of course legal, it is clearly against that bit in the CEFR definition of levels that says 'spontaneously', at the very least. And if a candidate is actually asked about one of the subjects he memorized stuff about, his chances are considerably better. And then there's all the other stuff we've been telling about. Regarding how cheating doesn't always work... what does, except studying enough? Two examples of how just like exams can suck, so can candidates (and that's especially true about people who try to cheat): I've seen my parents present their students with the *exact same exam* they flunked two months earlier, and have them all flunk again, even using cheat sheets both times (which was spotted but ignored). Seems like recycling cheat sheets implies a lesser effort than memorizing the answers for the last exam. Turns out it would have been a better strategy :) The only exams where it was virtually impossible to cheat, I sat in my first year of Physics. You could bring in any materials *except* books compiling problems and solutions (to allow recycling in test-making and alleviate the probability of finding the exact answers for everything in books). People would enter the exams with loads of books, it was incredible. A few others decided to study for real and would enter the exam with just a handful of formulas written on a paper sheet, if anything. Guess what group did better and why. The higher the level the less sense cheating one way or another makes. But people still try. Edited by mrwarper on 24 June 2013 at 5:19pm 3 persons have voted this message useful | ||||||||||||
| Cavesa Triglot Senior Member Czech Republic Joined 4818 days ago 3277 posts - 6779 votes Speaks: Czech*, FrenchC2, EnglishC1 Studies: Spanish, German, Italian | Message 50 of 80 24 June 2013 at 2:40pm | IP Logged | |||||||||||
|
Let's keep to that candidate A and candidate B (parrot) example:
Reading comprehension exercises: Candidate A:Reads a lot in the language. Can read anything even though he/she isn't too excited about one or two areas. Has tried only a few exam exercises to see the exam structure. Candidate B:Doesn't read on his own except for the past exam papers. Does know all the usually used topics, has done so many of the exercises that he/she can see the test patterns like: "After this kind of sentence always comes one beginning with this word" or "in the multiple choice, there is always one question with all answers correct", "there are never more than half of the questions with B as correct", "there are always at least two or three idioms from this list" "one third of the questions focuses on this and none on that" and so on. Writing exercises: Candidate A:Writes in the language daily, is creative (as that is required for most of out of the test situations),can write anything in a clear, understandable and his own way. Has seen a few of the past assignments and found out there is nothing specifically difficult for him/her. Candidate B: Doesn't write the language at all in his/her life but went through all the available assignments, memorized which opening and closing structures get more points, which vocab was highlighted as awesome in the examples. And wrote many exercises with only exam points in mind, no matter the fact that some of the criteria go against the logic. He/she knows well not to try any kind of creativity. They learn "how to compose" the test exercises and are often unable to write anything else. Speaking: The least cheatable one. But still. Candidate A: Can speak about quite anything even though heshe does have more and less favourite topics. Candidate B: Struggles with anything but the test assignments. Was taught good looking expressions, how long approximately to speak before asking the other student (Cambridge exams pair up people for speaking), what opinions to tell when speaking about some issues! (Yes, it does happen because everyone knows the examiners are still people). Listening: Candidate A: Can understand quite everything. Radio, movies, drunk friends. Looked at a few past papers. Candidate B: Never watches even a movie on his own and has trouble understanding many things that weren't in the common exam preparation audio files. May even have trouble understanding natives unless they speak of the common test topics. Again, knows a lot about the multiple choice "rules". Can easily spot what kind of information will be in the questions but often would be unable to just plainly tell you what was it all about. What I was trying to show. sallard, you keep turning it into comparison of a careless person who just one day says "I'm gonna sit an exam" and someone working for it. What mrwarper and I tried to show are differences between a person who is on the level but looks at just a few past papers or perhaps none and someone who is much worse at the language but chooses "the smart way" as you see it. It is hard to tell which one will get better grades but it is far too often candidate B. No matter his real skills. That is a flaw in most test. Any test can be cheated on or as you take it "one can prepare themselves in a smart way". So, you can tell whether the person before you just memorized a sentence and doesn't understand? Good. But can you tell whether the person is able to correctly use the sentence anywhere it is appropriate or just when they know "I am in the exam and must use it"? Preparing for the spoken part with a tutor, sure it is a good way to know the skills as they are required in the test. But are their other speaking skills at the same level?
Sure, you are advocating the approach "learn for the test, that's all that matters. the system is always right". And it is just wrong. Of course you need to somehow prepare for the test which is more or less torn from the real life, ideally by learning the real skills. But once your real skills matter much less than your test skills, the test is just wrong, the system failed and the "smart, diligent students" as you see them are often just dumb morons who know how to exploit it at the expense of better quality candidates. Once you, as a examiner as you said, don't care whether the test is realistic, something is very wrong. 2 persons have voted this message useful | ||||||||||||
| s_allard Triglot Senior Member Canada Joined 5239 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 51 of 80 24 June 2013 at 4:40pm | IP Logged | |||||||||||
|
What this running argument between myself and both @cavesa and @mwraper boils down to is: Can you beat the
CEFR exam system by systemetically preparing for the test regardless for of one's "real" level of proficiency? I say no, they say yes. Two minor points to begin with. I think this is the same with all the CEFR tests. 1) You have to pass all the parts and 2) The results do not tell your level, i.e. you are B2, but rather the score solely for the test you took. This is totally unlike certain tests like the IELTS where it is important to score as high as possible. This also means that when you register for a CEFR, you have to choose the level carefully because it is all or nothing. But to come back to the fundamental argument. As I have pointed out, for many tests today you can find all kinds of preparatory material, especially past tests. For example for the French baccalaureat exam you can get all the previous questions and perfect answers. Does this worry the people who make the exams? Couldn't a student just memorize the answers from the last 5 years and ace the 2014 exams? Why doesn't everybody get top marks? The answer is pretty simple. Memorizing past test answers does not replace knowing the matter. @cavesa is saying that with efficient memorization and test preparation, one can get better results than one's true knowledge deserves. I quote: "Any test can be cheated on or as you take it "one can prepare themselves in a smart way."" But how can smart preparation replace knowing the matter? This is what I find so intriguing. We're not talking about intelligence tests or aptitude tests. We're talking about language tests where you are required let's say to fill in some blanks with the right form of a verb. Or to listen to a scratchy phone conversation and answer some questions. And then have a conversation with an examiner. How can you memorize and pass a test without knowing the material? How can you fill in the blanks correctly if you don't know the material? Should you memorize five telephone conversations and pray that the right one comes up? And you think you can fool the C-level examiner with your memorized answers? No way. 1 person has voted this message useful | ||||||||||||
| beano Diglot Senior Member United KingdomRegistered users can see my Skype Name Joined 4431 days ago 1049 posts - 2152 votes Speaks: English*, German Studies: Russian, Serbian, Hungarian | Message 52 of 80 24 June 2013 at 5:01pm | IP Logged | |||||||||||
|
Nothing wrong with rote learning. A lot of that goes on in language learning. Conjugating the verb "to be" for example, which is nearly always highly irregular.
2 persons have voted this message useful | ||||||||||||
| mrwarper Diglot Winner TAC 2012 Senior Member Spain forum_posts.asp?TID=Registered users can see my Skype Name Joined 5035 days ago 1493 posts - 2500 votes Speaks: Spanish*, EnglishC2 Studies: German, Russian, Japanese | Message 53 of 80 24 June 2013 at 5:05pm | IP Logged | |||||||||||
Excuse me, I'm not saying you can beat the system (I sincerely hope not), I'm saying you can cheat (so to say) to artificially get better results than you deserve, just like in any other examination system. Whether that alone makes the difference between passing or not and for how many candidates you pass is another matter. Many people take their chances, and that's a fact. Edit: I consider it immensely amusing that most 'cheaters' I have met didn't use such artificial edges to actually get good marks, but to try and spare themselves some of the effort real learning takes. I guess honest candidates, once they learn how far their effort gets them, mostly end up spending more energy on study if necessary and they could afford it.
Then one minor point down to start with. The C2 I sat (CPE) didn't require to pass all the parts.
Again, it depends on the test, at a minimum. Unless it was a total flunk (I hope) if you failed the CPE (C2) you could still get a C1 certification.
It can't. For this to work, you need some luck. But you can help your luck by doing a lot of stuff that's easier than real study, and that's enough for many people to try. It works sometimes, and that's all that's needed for people to keep doing it. Edited by mrwarper on 24 June 2013 at 5:16pm 2 persons have voted this message useful | ||||||||||||
| s_allard Triglot Senior Member Canada Joined 5239 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 54 of 80 24 June 2013 at 8:01pm | IP Logged | |||||||||||
Maybe we just don't agree on the use of the word "cheat." For me, cheating on an exam is obtaining the questions or the answers fraudulently. For other people here, "cheating" is to obtain results that are higher than what one deserves. If I understand this correctly, for example, a person may only be at the CEFR B2 level but is able to game the system and obtain a C1 by studying systematically for the C1 test. Or that person obtains an IELTS 9 when they are really an 8. Is this really possible, "like in any other examination system?" The fundamental question is one of test design. It's up to the designers of the tests to make them fair, accurate and foolproof. The designers are fully aware that some people are out to game the system by memorizing material from previous tests. And there are tons of guidebooks to passing these tests. Is this a problem? Not really because the tests are designed to verify what you can do not what you can memorize. I note in passing that the following website of Cambridge English Language Assessment, the people who run the CPE, has a whole section devoted to preparing for the test. For each part of the test there are tips, FAQs, sample tests and recommended materials. Cambridge Test Preparation Website Why does Cambridge provide this material? Are they encouraging cheating? Aren't they afraid the people will use this material to obtain scores higher than they deserve? This is not a problem for Cambridge. They provide the materials because they want the candidates to be prepared, i.e. to know what to expect and how to best prepare in order to get the best results. The CPE has been around a long time and the English language doesn't change radically from year to year. So couldn't you just memorize last year's exam and be sure to pass this year's CPE? Test designers are pretty smart people. Whether it's the CPE or the Spanish DELE C2, the designers will ask you to accomplish certain tasks. You are asked to listen to recordings and answer some questions. You will be asked to write a composition of a certain length on a certain topic. You may be asked to fill in the blanks on vocabulary and grammar questions. And you will be asked to have a conversation (in the DELE) with an examiner. Some people believe that if you memorize enough material, you can trick this system. But the people who design these tests know that candidates can't memorize all the possible questions and all the possible answers. How can you memorize for the composition test? Should you memorize some key phrases that you can plug in anywhere? Or maybe three entire 250-word texts and then try to cobble together an answer using one of these memorized documents? None of this works very well. This is why the test designers are happy to give people all kinds of sample test materials. They are confident that the design of the test and the constraints of time are such that the test results will truly reflect the ability of the person "to pass the test." I add this little phrase to emphasize the fact that these tests reflect the ability to perfom in a test environment. They do not claim to reflect your true ability to perform in real-life situations. As I have stated repeatedly, test preparation does not replace mastering the material. You prepare for the test by learning the material expected of you and how to perform in a test situation. You practice with sample tests because you want to get a feel for the material, your level and how the whole thing works. That's not cheating. 2 persons have voted this message useful | ||||||||||||
| Cavesa Triglot Senior Member Czech Republic Joined 4818 days ago 3277 posts - 6779 votes Speaks: Czech*, FrenchC2, EnglishC1 Studies: Spanish, German, Italian | Message 55 of 80 24 June 2013 at 8:27pm | IP Logged | |||||||||||
Why? Because it earns them a lot of money. Of course people will buy not only the exam but as well two preparatory coursebooks, perhaps classes, mocktests, past papers, guides etc. when the test is made to measure test skills not real skills CEFR, which is currently the common base for all those exams (it used to be different but nowadays it should be so) is a scale defining skills, so mastering the material while not really having the skills is getting higher grades than the learner deserves and therefore cheating. As I tried to explain: Mastering the exam material is NOT necessarily the same as mastering the skills. Not having the skill and just drilling exam exercises is not what smart students do, contrary to your beliefs. Intelligent student primarily learns the skills and suplements that with exam material. A moron masters the material, gets unfairly good grades no matter that his/her skills are worse compared to the intelligent student. 3 persons have voted this message useful | ||||||||||||
| s_allard Triglot Senior Member Canada Joined 5239 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 56 of 80 24 June 2013 at 9:23pm | IP Logged | |||||||||||
If there is one area where we agree it's that these tests measure the ability to perform on a test. The reasonable expectation is that test results are correlated with real-life skills. But how do we measure real-life skills short of observing the individual in real situtions? Since we can't do this on a reasonable scale, we are stuck with testing in an artificial environment. This is why it is important to keep in mind that the CEFR is biased towards the academic and professional world. It is not designed to reflect all the everyday needs of individuals. I find intriguing this observation that a moron could drill exam exercises and do better on the test than the ntelligent student who has the real skills. This is very unlikely. As I have tried to show, the very design of these language tests makes it very difficult if not impossible for a moron to perform well just by drilling exam exercises. Frankly, if that moron passes the C-level or the CPE exam, I'd say that the moron is not so moron after all. Who cares about the so-called real skills? What counts is the test results. The point of all this is that the most language testing systems measure relatively well the ability to perform on a test. We hope and expect that the test results correlate generally with overall proficiency. If a moron can do better on a C1 or C2 test than an intelligent person, I am in awe. Am I to question that person's credentials and say that that C was obtained fraudulently? 1 person has voted this message useful |
|  | 
Printable version
|
| Copyright 2024 FX Micheloud - All rights reserved No part of this website may be copied by any means without my written authorization. |
