|
 |
Is comprehension measurable? | ||
Share with:
| ||
 Language Learning Forum : General discussion |  | |
| s_allard Triglot Senior Member Canada Joined 5241 days ago 2704 posts - 5425 votes  Speaks: French*, English, Spanish Studies: Polish | Message 113 of 211 20 August 2014 at 2:40pm | IP Logged | ||||||||
|
I'll ignore the hissy-fit and the rambling musings and get straight to the point. Iversen's point of methodology is
well taken. When I said that my approach was crude this is what I was alluding to. It was nothing more than a quick-and-dirty attempt to demonstrate what 75% word coverage could look like. Using Hu and Nations method, the text would be less informative and allow for less guessing and deduction by the advanced learner, as Iversen rightly points out. The key point remains --sorry to keep repeating myself here like a broken CD player -- that this so-called 75% comprehension, whether metaphoric or real, is, in my opinion, actually the equivalent of nothing or 0%. Well, not exactly. I'll concede that the reader knows that something is happening. What percentage comprehension is there here. What is the difference between 25% and 75% comprehension? How many people would read an entire book like this? Most people of course would make a beeline for the dictionary with the expectation, rightly so, that a big investment in time and effort at the beginning would make the text gradually easier and more enjoyable to read. As those who have actually read what I wrote will have noted, I'm in favour of very simple scale: All, Some and Nothing. No lengthy descriptions here. No numbers. Edit: I wrote this post before seeing emk's post that I'll address promptly. Edited by s_allard on 20 August 2014 at 2:50pm 1 person has voted this message useful | |||||||||
| s_allard Triglot Senior Member Canada Joined 5241 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 114 of 211 20 August 2014 at 3:51pm | IP Logged | ||||||||
I fully recognize the limitations of the "silly" example that I gave, as I have said in my previous post. I was trying to give an example of what it was like to not understand 25% of the words. That's why we speak of word coverage. Working at the sentence level is a completely different problem. Here is the Harry Potter text with two sentences removed: "Mr and Mrs Dursley, of number four, Privet Drive, were proud to say that they were perfectly normal, thank you very much. They were the last people you'd expect to be involved in anything strange or mysterious, because they just didn't hold with such nonsense. ... Mrs Dursley was thin and blonde and had nearly twice the usual amount of neck, which came in very useful as she spent so much of her time craning over garden fences, spying on the neighbours. ..." Of course, the text is much more understandable. We're not talking about word coverage here as used by Hu and Nation. These authors do not talk about 98% sentence coverage for enjoyment of fiction. There's an interesting methodological issue here. What makes an entire sentence incomprehensible? Is it one or more words? Or maybe some unusual construction? Or even some kind of cultural or historical reference. How could a reader understand 50% of a Jules Verne book at the sentence level? Could one systematically understand one sentence and not the next? For works of fiction, not understanding entire sentences is not a problem. If I have no doubt that if you removed every tenth sentence in a Jules Verne book, most native readers would probably not notice the difference. But take a legal document such as a letter from the tax department and a computer user guide and remove 25% of the sentences. Technically, one understands 75% of the text but the entire document probably becomes incomprehensible. So far the thread has focused at the word level. And much of the recent discussion has been about how people arrive at these percentages. If someone like emk actually counts or estimates the number of unknown words per page I have no problem acknowledging that this measure of word coverage is probably indicative of comprehension. But how many people actually do this? These issues come to a head when one as has to test for comprehension. Do you count words or do you test for content? I have never heard of any testing strategy that relies on trying to determine how many words a user knows. All the tests use some try to determine what the student can do based on the content of the text, How to do this with fiction is very tricky. Edited by s_allard on 20 August 2014 at 3:52pm 1 person has voted this message useful | |||||||||
| s_allard Triglot Senior Member Canada Joined 5241 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 115 of 211 20 August 2014 at 5:37pm | IP Logged | ||||||||
|
If someone says to me, "I understood 75% of Harry Potter and the Philosophical Stone", I can interpret this in
different ways: 1. The person did not understand every fourth word in the text. (my crude example). 2. The person did not understand 25% of the words of the text in random order. 3. The person did not understand every fourth sentence. 4. The person did not understand 25% of the sentences in random order. 5. The person did not understand every fourth paragraph. 6. The person did not understand 25% of the paragraphs in random order. What is highly improbable is that the person did not understand an entire page out of four or did not understand the last 25% of the book. I find the idea of not understanding entire sentences intriguing. I'll have to admit that I've never really looked at the question. What I find interesting is that the rest of the text is perfectly readable. There is something in 25% of the sentences that make them impossible to understand. Could one hypothesize that in certain kinds of texts, e.g. fiction, up to 25% of the sentences can be dispensed with. In which case, that 75% comprehension is in essence much higher. In other words one could totally enjoy or understand Jules Verne or Harry Potter with just 75% of the sentences. This is an interesting question and warrants some research. 1 person has voted this message useful | |||||||||
| Cavesa Triglot Senior Member Czech Republic Joined 4820 days ago 3277 posts - 6779 votes Speaks: Czech*, FrenchC2, EnglishC1 Studies: Spanish, German, Italian | Message 116 of 211 20 August 2014 at 7:09pm | IP Logged | ||||||||
|
I'll ignore the childish offences from someone who just can't stand people disagreeing with him and get to the matter.
Well, there is actually a world of difference between what people mean by the 25% and what by the 75% and it is impossible to not see it unless you are determined not to see. Your statement has the same logic as saying there are only three levels of knowing a language. Knowing nothing, knowing some of it and being native-like. Since you throw around the cefr all the time, you may probably realize this isn't the case. I think we have an interesting link here to the intensive vs. extensive reading: -I'd say intensive readers would be more likely to agree that either you understand nothing or everything. Or perhaps it is the other way around: They want to understand everything as soon as they encounter it for the first time because understanding less, even temporarily, equals to understanding nothing in their eyes, and that's why they read intensively. -On the other hand, extensive reading is based on gradual improvements in comprehension. An extensive reading learner would necessarily give up as soon as he'd get convinced he understands nothing and cannot enjoy less then full comprehension. Any extensive reading learner can notice even quite small improvements in comprehension. In amount of known vocabulary, in amount of clear structures, in amount of details they understand, in enjoyment. A three statements scale (nothing,something,everything) is totally useless and actually harmful for progress of an extensive reader. I believe most people using our % metaphor do not speak about % of sentences not understood. It is rather a measure of the amount of nuancies and details that are being lost. Whether we follow just the skeleton of the plot or whether we enjoy even details from sceneries descriptions etc. The sentence counting is even trickier than the vocabulary counting. Some sentences are long, some are short. Some are essencial for understanding the plot while others just add flavour. In a long sentence, it may be only a part of it that is essencial. Therefore I'd say counting sentences understood is totally useless. 2 persons have voted this message useful | |||||||||
| Cavesa Triglot Senior Member Czech Republic Joined 4820 days ago 3277 posts - 6779 votes Speaks: Czech*, FrenchC2, EnglishC1 Studies: Spanish, German, Italian | Message 117 of 211 20 August 2014 at 7:18pm | IP Logged | ||||||||
|
http://ebcl.eu.com/wp-content/uploads/2011/11/CEFR-all-scale s-and-all-skills.pdf
The detailed description of reading skills, beginning at page 10, proves that it is nonsense to belive there is no difference between a learner who can understand a little and one who is a little bit under the level of complete comprehension. No counting of sentences needed. 3 persons have voted this message useful | |||||||||
| Serpent Octoglot Senior Member Russian Federation serpent-849.livejour Joined 6408 days ago 9753 posts - 15779 votes 4 sounds Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish | Message 118 of 211 20 August 2014 at 9:20pm | IP Logged | ||||||||
I mentioned this on the very first page. Sometimes I wonder if there's a tool for auto-hiding posts by this or that member. edit: on the second page actually But other than the part about sentences, I agree with Cavesa. (the length difference can be eliminated by counting clauses) My guesstimate is generally based on 1) the individual words I missed; 2) those of them that made an entire clause/sentence incomprehensible; 3) the sentences/clauses where individual words weren't the problem. Edited by Serpent on 20 August 2014 at 10:10pm 1 person has voted this message useful | |||||||||
| s_allard Triglot Senior Member Canada Joined 5241 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 119 of 211 20 August 2014 at 10:48pm | IP Logged | ||||||||
|
I can certainly tell the difference between 25% and 75%. It was 50% when I last counted. But that isn't the
question. The question is what does 25% or 75% reading comprehension mean? One answer is you feel it when you know it. Another answer is that if you keep track of the words you don't know as you are reading you will see that you don't understand 25% of the words. A third answer is that you don't understand about a quarter of the sentences of the text. ı think I understand all these approaches. Now, how can they give us an answer to the question: How can one understand 75% of Harry Potter and the Philosophical Stone? We'll discard the feely-I-know-it-when-I-see-it- approach. The sentence approach seems to provoke a lot of ire as well. This leaves the word counting approach. It seems then that a 75% comprehension has something to do with the number of unknown words in the text. In a first attempt to approximate what this might mean, I mechanically omitted every fourth word in the excerpt. This was roundly criticized. I agree. Here is another attempt where I massaged the excerpt to keep proper nouns and common words. Readers are encouraged to provide a better version. "Mr and Mrs Dursley, of ___ four, Privet Drive, were ____ to say that they were perfectly ___, ____ you very much. They were the last ___ you'd expect to be ____ in anything ___ or mysterious, ____ they just didn't ___ with such ___. Mr Dursley was the ___ of a firm ___ Grunnings, which made ___. He was a big, ___ man with hardly any ___, ____ he did have a very large ___. Mrs Dursley was ___ and blonde and had ___ twice the ___ amount of ___, which came in very ___ as she ____ so much of her ___ craning over garden ___, spying on the ___. The Dursleys had a small ___ called Dudley and in their ___ there was no finer ___ anywhere." I fail to see how this corresponds to 75% understanding from the point of view of a learner of English. All I see is a bunch of recognizable words. I don't see a big difference between 75% and 0% understanding of the text. I see a huge difference in the number of recognized words but not in understanding. Edited by s_allard on 21 August 2014 at 12:10am 1 person has voted this message useful | |||||||||
| Serpent Octoglot Senior Member Russian Federation serpent-849.livejour Joined 6408 days ago 9753 posts - 15779 votes 4 sounds Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish | Message 120 of 211 21 August 2014 at 12:07am | IP Logged | ||||||||
|
The world isn't split into good people and Death Eaters. Words aren't split into well-known and opaque.
Also, opaque or unclear words can reappear a lot. One of the most useful tips I've seen is to finish the paragraph/page/even chapter before looking up (depending on how crucial the word is). Very often you can figure out the meaning by comparing several contexts where the word appears. Edited by Serpent on 21 August 2014 at 12:11am 3 persons have voted this message useful |
|  | 
Printable version
|
| Copyright 2024 FX Micheloud - All rights reserved No part of this website may be copied by any means without my written authorization. |
