|
 |
Is comprehension measurable? | ||
Share with:
| ||
 Language Learning Forum : General discussion |  | |
| Serpent Octoglot Senior Member Russian Federation serpent-849.livejour Joined 6599 days ago 9753 posts - 15779 votes  4 sounds Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish | Message 129 of 211 21 August 2014 at 11:50am | IP Logged | ||||||||||||||
|
naapuri should've also been transparent, and possibly hölynpöly.
1 person has voted this message useful | |||||||||||||||
 emk Diglot Moderator United States Joined 5534 days ago 2615 posts - 8806 votes Speaks: English*, FrenchB2 Studies: Spanish, Ancient Egyptian Personal Language Map | Message 130 of 211 21 August 2014 at 1:36pm | IP Logged | ||||||||||||||
Emphasis added. I think this will make more sense if you re-read what I wrote. I understood 60% of the text with minimal difficulty, and I could figure out another 30%, either by thinking about it for a while or using the popup dictionary. But despite my best efforts and heavy use of the popup dictionary, about 10% of the text remained largely indecipherable. I found that a lot of the indecipherable portions of the text stemmed from three problems: 1. Weak understanding of certain corners of French grammar. I had a wobbly grasp of clitic pronouns, and I struggled greatly with en. And a lot of the time, I glossed over word inflections, especially those that are silent in spoken French. 2. Weak grasp of word of how particular words were normally used in context, of common phrasal "chunks", and of common collocations. Normally this wasn't an overwhelming obstacle, because I could resolve ambiguities using context, but sometimes the comprehension process broken down entirely, and the sentence was hopelessly lost. 3. Too many unknown words in one sentence. If I know the words, I can often infer the grammar. And if I know the grammar, I can sometimes infer the words. But if I know neither, it's hopeless. Seriously, with the right kind of support, it's possible to "read" and enjoy with very low comprehension. I can't demonstrate this with French anymore, but Egyptian can work nicely. Here's a section of the Westcar papyrus from the St Andrews Corpus: 
Here's a rough recreating of how I "read" this the first time (using MdC notation since I don't have special characters): iwms. Doesn't that mean "surely?" wn. This is some sort of "to be" verb. Sspt. Pavillion! I know this one! Thank you Assimil! m. "In." pA. Huh. Supposedly "this/that", but on this page, it's behaving like "the." Weird. S. Garden! Thank you again, Assimil. wbA-inr. The name Ubainer, obviously. mT. Sentence-initial particle meaning "Look." n. "Us". ir=n. "Do", first person plural. At. I have no idea, but from context, I'd guess ir At means "spend time." im=s. "In her", and Sspt is feminine, so "in [the pavillion]." You'll notice that in 13 words, I was rather confused by both of the verb constructions, iwms wn "Surely there is" and ir At "spend time." In this case, I was able to follow along thanks to prior knowledge of common words and the support of the loose English translation. Looking at this text, if I use the English translation, I can decipher maybe 4 out of every 5 lines about this well. Other lines are more challenging, even now: 
Hna rdi.t di.tw. If I stop to untangle this, I get "And let there be given." Sns 1. This must be "one cake". Hnqt Dwiw 1. One of these words is "beer" and the other is "jug." iwf. Some particle? It looks like iw=f, but that wouldn't make sense here. wry. "Very"? snTr pAd 1. Huh? n xry-Hbt Hry-tp. "To chief <job title>." Here, I'm missing enough pieces that everything eventually falls apart. And as I suggested above, maybe 10% or 20% of the lines in text are like this, even using the English to help untangle the Egyptian. But then again, we can safely translate this second line as "Let's give a bunch of stuff to Imhotep; he's cool," and then move on. The details are neither critical to the story, nor especially useful to me at this stage of my learning. So even with only ~80% of the text being either transparent or decipherable, I can still read along and enjoy myself tremendously. When all else fails, I can just use the English text (or context, or memory of a translation, or pictures, or whatever) to fill in the blanks. Edited by emk on 21 August 2014 at 1:46pm 2 persons have voted this message useful | |||||||||||||||
| Cavesa Triglot Senior Member Czech Republic Joined 5011 days ago 3277 posts - 6779 votes Speaks: Czech*, FrenchC2, EnglishC1 Studies: Spanish, German, Italian | Message 131 of 211 21 August 2014 at 2:10pm | IP Logged | ||||||||||||||
|
Yes, that is exactly one of the problem with counting. There is more than two cathegories for either words or sentences.
Some people may take for the % guess or count only the things they understood at first sight and without hesitation. Such people will basically avoid overestimating their skill at the cost of underestimating it. They will get much higher "score" if they count in the things they understand after a moment of thinking or after getting a little bit more context in the next paragraph or so. Truth be told, I have found good grasp (at least passive) of the common grammar to be much more crucial to the beginnings of reading in a new language than having wide vocabulary. For vocabulary, you can basically apply the 80/20 rule, begin enjoying the books quite early and add new words one by one either by repeated encounters (extensive readers) or with a dictionary (intensive readers). But without the grammatic structure to anchor those words in, you are more likely to get lost and misunderstand chunks of the plot. And the point about various kinds of text in a book was a good one too (I think raised by Serpent). Exemple: About a month ago, I finished reading a French historical fantasy La trilogie de Wieldstadt by Pevel. I understood 90-95%. In this case, it means I understood everything of the plot including details with just a rare word missing occassionaly (and that usually got more or less clarified by the context) but my comprehension got much worse when it came to pages with descriptions of medieval houses or clothes. I only skimmed through a paragraph or two every hundred or so pages. I had only basic comprehension without much of a detail there but it had no impact at all on the plot understanding or enjoyment. And there was no point in memorising those tricky words as they were trully rare and I am not likely to need them much in future. So, the theories about people who understand 75% not understanding every fourth sentence etc. are totally different from reality. Words, sentences or whole paragraphs have various difficulty and various importance for overall comprehension of the book as a whole. Therefore you can somehow analyse a text and assess comprehension of the one text following methodologies desribed by Iversen, if I remember correctly, but it is very unlikely the reference values for one book could be applied on any other and give reliable results. 3 persons have voted this message useful | |||||||||||||||
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 132 of 211 21 August 2014 at 2:55pm | IP Logged | ||||||||||||||
I have made the correction to state that emk could understand only 2/3 of the sentences and not less than 2/3 and still enjoy the work. My question is how can this work, given that all the vocabulary studies point to at least a 95% word coverage -- not sentences -- in order to fully appreciate a work of fiction. emk speaks of 60% easy reading, 30% decipherable and 10% indecipherable in the case of Jules Verne. Let's say 80% transparent reading to simplify things. To understand how this is possible, it is important to realize that we are working at the sentence level. 20% of the sentences are somewhat unreadable or incomprehensible for the reasons given above. Despite this, one can follow the story and enjoy the work. This is not surprising because of the nature of the writing of Jules Verne and of fiction in general, especially 19th century fiction. As I pointed out in my previous post concerning Jules Verne, one could easily omit a quarter or a third of all the sentences and not see a big difference in the plot because so much of the writing is redundant and wordy. We saw something similar with the Harry Potter excerpt that I gave above. One can drop entire sentences and the reader will not notice a thing. This is because the existing sentences provide enough content to make the story coherent. I think that even native speakers do a certain amount of skimming or speed reading when undertaking those weighty Russian classics. Then there is the issue of enjoying a text of which one understands only two thirds. Depending on the nature of the writing, does two thirds understanding lead to two thirds or reduced enjoyment? It makes me think of my high school days and those condensed versions of literary classics where a 400-page book is reduced to 80 pages. Is there the same enjoyment? Another very complex issue is of course how well does one really understand a sentence. When a person says that they understand something, especially in another language, how do we know what was really understood. We don't. This is why comprehension tests use content questions. But that is a whole other issue. With non-fiction such as newspaper articles, legal and technical writing, it is very doubtful that one can get by or enjoy the text with 80% sentence level comprehension. For certain forms such as legal texts, every single word is very important. All of this is very different from not understanding at the level of individual words spread throughout the text. Not understanding 20% of all the words is very different from not understanding 20% of all the sentences. 1 person has voted this message useful | |||||||||||||||
| emk Diglot Moderator United States Joined 5534 days ago 2615 posts - 8806 votes Speaks: English*, FrenchB2 Studies: Spanish, Ancient Egyptian Personal Language Map | Message 133 of 211 21 August 2014 at 5:38pm | IP Logged | ||||||||||||||
Yeah, this is why I keep using a three-way breakdown of opaque/decipherable/transparent. Here, "decipherable" corresponds to Krashen's "i+1 input"—things you can understand either thanks to the context, or because you stared at them for a few seconds before they clicked. Krashen has spent his entire academic career insisting on the importance of "i+1 input", but by definition, this input is a little above the learner's level. If people divide everything into "comprehensible" and "incomprehensible", it's easy to overlook what Krashen claimed was the most important part.
Krashen talks a little bit about this:
In another paper that I can't find right now, Krashen talks about how some language learners can cope with much lower comprehension. The closet I've found is this note:
Anyway, my personal experience is that comprehension can be surprisingly low as long as the input is enjoyable. As long as I'm having fun, an I can match some meaning up with words, things seem to work out. How low can comprehension go before enjoyment is lost? It depends a lot on the media. For example, this rather silly and occasionally tasteless video about superhero hangovers would still be pretty enjoyable without any knowledge whatsoever of French. So if a student can puzzle out 25% of this video, then 75% which they can't get poses no particular problem. 
(Not recommended for use in class, but I assume that's obvious.)
And this is why A2 French students should not attempt to negotiate complicated business deals in France without a qualified bilingual lawyer. Fortunately, this does not seem to be a common problem. Le Tour du monde is far more forgiving than a high-stakes business contract. As you've observed, even if you throw away entire clauses and paragraphs, it's still a pretty fun story (for the right reader). I mean, it's not like anything bad will happen if you miss parts of the story, or misinterpret the occasional phrase. 3 persons have voted this message useful | |||||||||||||||
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 134 of 211 21 August 2014 at 5:49pm | IP Logged | ||||||||||||||
|
As I think about this idea of not understanding at the sentence level, the more I realize the complexity of the
issue. Again I'm curious as to how this can be measured. What I find most intriguing is how someone can say they understand 75% of, let's say Harry Potter, because they fully understand only three out of four sentences. I can see how one can follow the story, at least in fiction, with only these sentences, but I have difficulty figuring out how a learner could perfectly understand three sentences and totally not a fourth. For example, here are the first eight sentences in the Harry Potter book: "Mr and Mrs Dursley, of number four, Privet Drive, were proud to say that they were perfectly normal, thank you very much. They were the last people you'd expect to be involved in anything strange or mysterious, because they just didn't hold with such nonsense. Mr Dursley was the director of a firm called Grunnings, which made drills. He was a big, beefy man with hardly any neck, although he did have a very large moustache. Mrs Dursley was thin and blonde and had nearly twice the usual amount of neck, which came in very useful as she spent so much of her time craning over garden fences, spying on the neighbours. The Dursleys had a small son called Dudley and in their opinion there was no finer boy anywhere. The Dursleys had everything they wanted, but they also had a secret, and their greatest fear was that somebody would discover it. They didn't think they could bear it if anyone found out about the Potters." We'll assume that our learner cannot understand two of those sentences. Which ones and why? I'm at a real loss to choose two sentences that would be unintelligible. Let's take: He was a big, beefy man with hardly any neck, although he did have a very large moustache. Now what would make this sentence incomprehensible? Perhaps it's the word "beefy"? But that's just one word and hardly makes the entire sentence incomprehensible to a person who can understand the other sentences. Let's take a second sentence: " They didn't think they could bear it if anyone found out about the Potters." Maybe our learner has a problem with the phrasal verb "bear it". How much is the entire sentence incomprehensible without "bear it" is debatable. The problem in this exercise is how to accept that a person can understand perfectly well the idiomatic English of all the other sentences and suddenly not understand these two sentences, What I suspect is that the two sentences are not really totally incomprehensible. It's just certain words. These may be key words, of course, and indispensable for the total meaning but it's not as if all the words of the sentence are unknown. In essence then it's just two or three words that are not understood here and not two sentences. Now we are back to the word level and instead of not understanding two sentences on a page, the person in reality does not understand two words. This corresponds to our intuition. How can a person understand an entire paragraph of complex language and then not understand a sentence that is no more complex but for a word or two? 1 person has voted this message useful | |||||||||||||||
| Cavesa Triglot Senior Member Czech Republic Joined 5011 days ago 3277 posts - 6779 votes Speaks: Czech*, FrenchC2, EnglishC1 Studies: Spanish, German, Italian | Message 135 of 211 21 August 2014 at 6:38pm | IP Logged | ||||||||||||||
It was already explained, and I think emk did an excellent job, that: 1.There are not only totally understood and totally not understood sentences, so you are seeking an answer where there is no question. 2.It is not about every fourth sentence not being understood. It is about passages understood better and passages understood worse while the passage understood worse (or even not at all) may be even significantly longer than a sentence but still not that important for overall comprehension of the book
The problem is that you are assuming something that is just very unlikely to happen. A person who understands 3/4 of the book is quite likely to understand all those sentences well, perhaps with one or a few opaque words, depending on their vocabulary, but instead have a longer passage of lower comprehension elsewhere in the book.
The problem in this exercise happens to be that it has very little to do with reality as I have already explained above.
Now were are back to the point that has already been adressed a thousand times. Many sentences are neither perfectly and automatically understood nor totally undecipherable. As emk and serpent have been pointing out, there is a lot of context using and it depends a lot on which words you are missing, whether you have any idea what they might mean and how important they are in the text. I think Serpent's experiment with those Portuguese and Finnish words was a nice demonstration.
And here comes the artificial and unrealistic construct again. The real situation is much more likely to be the direct opposite. The person understands 100% of sentences in 3/4 or more of the book, even though with some less familiar words and context guessing. But they don't understand some complex and difficult paragraphs at all or nearly at all. And if those paragraphs aren't too imporant for the plot (they might be purely descriptive and flavour adding), it doesn't matter much, at least not for the enjoyment or story comprehension. 3 persons have voted this message useful | |||||||||||||||
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 136 of 211 21 August 2014 at 7:16pm | IP Logged | ||||||||||||||
|
I'm still trying to figure out how one can understand 75% of Harry Potter. I'm trying my best with concrete examples.
I hear and read all kinds of explanations but few concrete examples. I have put excerpts of Harry Potter here and even two versions of Harry Potter with blanks. I have put an excerpt of Jules Verne. I wasn't the one who brought up the idea of understanding 2/3 of the sentences of Jules Verne. As a matter of fact, I didn't bring up the idea of understanding 75% of Harry Potter. All I ask is for people to show me how it is done. I don't need explanations. I need demonstrations. 1 person has voted this message useful |
|  | 
Printable version
|
| Copyright 2024 FX Micheloud - All rights reserved No part of this website may be copied by any means without my written authorization. |
