|
 |
Is comprehension measurable? | ||
Share with:
| ||
 Language Learning Forum : General discussion |  | |
| smallwhite Pentaglot Senior Member Australia Joined 5105 days ago 537 posts - 1045 votes  Speaks: Cantonese*, English, Mandarin, French, Spanish | Message 33 of 211 08 August 2014 at 5:16pm | IP Logged | ||
I thought we were talking about measuring comprehension instead of honesty of students or ease of examining students. 4 persons have voted this message useful | |||
| s_allard Triglot Senior Member Canada Joined 5227 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 34 of 211 08 August 2014 at 5:19pm | IP Logged | ||
|
Continuing with this idea of honest self-assessment, we hear someone say: "I watched a great film last night and I
understood 80% of it." What does this mean? Around here, that means that the person understood four words out of every five of the film. How did the person measure that while watching the film? How about attending a conference and understanding 70%? Or maybe having a conversation and understanding 90%? Most of the time it's pure guesswork with no basis in reality. And even if one can count, the significance is really murky. 1 person has voted this message useful | |||
| montmorency Diglot Senior Member United Kingdom Joined 4625 days ago 2371 posts - 3676 votes Speaks: English*, German Studies: Danish, Welsh | Message 35 of 211 08 August 2014 at 6:00pm | IP Logged | ||
|
Accepting s_allard's distinction between comprehension and vocabulary size assessment,
I imagine many language learners and HTLAL members would very often simply be content with the latter as a rough measure of their progress (unless they were needing to pass examinations or tests which tried to measure comprehension). But on the subject of vocabulary size assessment, people will perhaps remember the 2-part video lecture by Professor Arguelles in which he compares the vocabulary required to read "Moby Dick" with that required to read "Pinnocchio". He is using English, because at the time, it was the only language in which he could find a sufficiently large word-frequency-list-containing corpus (he did actually appeal for viewers to send in information about sources for other languages; the video was made a few years ago, so things might have moved on since). He used a specialist piece of (freeware) software to compare the word-families in the books with the WFLs in the corpus, and come up with an idea of the level of vocabulary required to read each of the books at various comprehension levels (percentage of word- families known). It occurred to me that if corpuses were available for your particular target language(s), you could do the above for a book you want to read,, and then when reading it, use something like emk's method, where he counts the number of words (or word families) he has to look up over a certain number of pages, and compares that with the number of words over those pages, to get an estimate of your comprehension level. So, if the software says you need 17,000 word-families to read this book at 98% comprehension level, and you can reach or exceed 98% comprehension in your test-read, you know you have at least 17,000 word-families under your belt. If you were really keen, you could count up the actual total of word-families you had to look up over the whole book (and emk probably has already written himself a program to do that. :-) ). part 1 part 2 1 person has voted this message useful | |||
| Bao Diglot Senior Member Germany tinyurl.com/pe4kqe5 Joined 5563 days ago 2256 posts - 4046 votes Speaks: German*, English Studies: French, Spanish, Japanese, Mandarin | Message 36 of 211 08 August 2014 at 6:24pm | IP Logged | ||
|
In 90% of all cases I don't put a number on how well I understand something.
Though I use qualifiers. For example: 'almost all of it': I believe I understood all the important details and connections, and even if I can't explain all of them in the language I will happily do so in German or English 'most of it' I think I got the important points, though I am a bit hazy on some details 'more or less' I think I got the gist and some of the details 'the gist' 'nothing' (single words or phrases) 'almost all of it' includes being able to learn completely new information at that level, 'most of it' includes new details about things I already know a bit about. The last example in French I'd put at "I can answer this kind of question correctly with a 'more or less' comprehension level." That is probably 30-50% automatic, the rest mostly deciperable with a few bits opaque. That can even mean being able to give a correct answer that includes opaque words because I understand the question and the rest of the text and know it has to be the opaque word, and I simply copy the sentence pattern for the answer. I personally wouldn't call that comprehension, though. Edited by Bao on 08 August 2014 at 6:28pm 4 persons have voted this message useful | |||
| s_allard Triglot Senior Member Canada Joined 5227 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 37 of 211 08 August 2014 at 8:39pm | IP Logged | ||
|
Although I believe that comprehension, generally speaking, is difficult or impossible to measure accurately, I
acknowledge that there are degrees of understanding of the target language. I like a simple non-numeric scale, somewhat like Bao's. I use a very simple scale: Nothing Some All I don't bother with fine statistics. If I don't have pretty good idea of what is going in the text, my comprehension is Nothing. If I feel that I can appreciate and follow the text without worrying about the language, it's All. Everything else is Some. Let's say that because of French and Spanish, I have a headstart when it comes to understanding Portuguese. I've never studied Portuguese. Could I pick up a Brazilian newspaper and claim I understand at least 30% ? Who knows?. I just prefer to say that my Portuguese reading comprehension is Nothing. 2 persons have voted this message useful | |||
| s_allard Triglot Senior Member Canada Joined 5227 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 38 of 211 08 August 2014 at 9:06pm | IP Logged | ||
|
A lot of people like the idea of word counting as a tool for assessing vocabulary size and comprehension. Fair
enough, but how many people have actually done any word counting? Most people are talking through their hats. The only people I know around here who have done something in this area are Iversen and emk. A cursory search on the Internet will reveal a whole academic subdiscipline devoted to the subject and with much disagreement between researchers. Just the definition of what is a word is subject of much debate. Part of the problem of the word counting method is that it relies totally on self-assessment. emk's solution to this problem is to have the student do a translation of the text without the help of a dictionary. We can see how well a person has understood the text by the translation into the student' s native language. This is a good idea but fraught with a major problem. In order to assess the learner's comprehension of a foreign language, we are forced to assess the translation into the learner's native language. And what kind of translation do we want: a more literal translation that stays close to the original or something more idiomatic and interpretive that reflects a more deeper understanding of the original? Could one not understand something perfectly in the target language but be unable to translate it properly into one's language? In a different thread, I differed with emk over a translation from the French. The problem wasn't with understanding the French as much as how to best convey the idea in English. 2 persons have voted this message useful | |||
| Bao Diglot Senior Member Germany tinyurl.com/pe4kqe5 Joined 5563 days ago 2256 posts - 4046 votes Speaks: German*, English Studies: French, Spanish, Japanese, Mandarin | Message 39 of 211 08 August 2014 at 11:05pm | IP Logged | ||
|
Well, to me the two steps in-between make sense, because I can use almost all/most comfortably, but don't completely trust myself with details in the 'most' category. I can use 'more or less' material, but it takes some effort. 'gist' material means I need to work with the material intensively if I want to actually use it, and 'nothing' material means I need to return to the basics before attempting this again.
That means that the two levels I would put as 'some' vastly differ in comfort of use, and the two levels I would put as 'nothing really' differ in whether I can deal at all with available material at that level of comprehension. I think translations can always be used as a starting point. Some people are just not that good at finding a good equivalent in the other language, but when you ask them to explain in more detail they'll probably come up with a useful description or remember the correct term. But the important point is to not give other cues than 'this sounds awkward, what do you mean with it' - I knew students that were pretty good at guessing their way through such situations. But I guess it's impossible to look at pure comprehension without coping strategies like that or the one I mentioned for multiple choice or cued comprehension questions. Though maybe multiple choice that forces a right/wrong/doesn't say decision and where there could be several, all or no options correct for a given question do help. Edited by Bao on 08 August 2014 at 11:14pm 1 person has voted this message useful | |||
| luke Diglot Senior Member United States Joined 7002 days ago 3133 posts - 4351 votes Speaks: English*, Spanish Studies: Esperanto, French | Message 40 of 211 08 August 2014 at 11:35pm | IP Logged | ||
|
76.3% of all statistics are made up.
3 persons have voted this message useful |
|  | 
Printable version
|
| Copyright 2024 FX Micheloud - All rights reserved No part of this website may be copied by any means without my written authorization. |
