|
 |
Is comprehension measurable? | ||
Share with:
| ||
 Language Learning Forum : General discussion |  | |
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes  Speaks: French*, English, Spanish Studies: Polish | Message 1 of 211 06 August 2014 at 8:56pm | IP Logged | ||||||||
|
In many threads, one reads of learners experiencing a certain percentage comprehension of a written or spoken
text, e.g. 50%, 70%, 95% comprehension, etc. When i ask how people arrive at these figures, I'm told that it's just a question of counting the words you don't understand vs the words you do know and doing the math. I disagree. I believe that many people think they understand more than they really do and that it is impossible to measure comprehension by counting words. One viewpoint that I think is useful is the dual comprehension principle associated with the German academic Wolfgang Butzkamm. The basic idea is that there are two levels of understanding. One is the fundamental meaning per se. So if I ask what does the simple French sentence Elle n'est pas venue hier mean, the answer would be She didn't come yesterday. This is level one. The other level of comprehension is that of the underlying grammatical structures of the target language. For example, in the sample above it's important to understand how French expresses the past tense with the auxiliary être with the verb venir and how verbal negation is expressed. In a sense, the learner has to develop a feel for how the language functions if they ever want to actively use the language correctly. Some observers speak of a kind of metalinguistic awareness the even many native speakers feel when they use their own language, especially at sophisticated levels. The example above is very simple. Things obviously do get very complicated. But the point is for the language learner comprehension is much more than knowing all the words and having a gist or murky idea of what is being said. I wonder what other people think. 1 person has voted this message useful | |||||||||
 emk Diglot Moderator United States Joined 5534 days ago 2615 posts - 8806 votes Speaks: English*, FrenchB2 Studies: Spanish, Ancient Egyptian Personal Language Map | Message 2 of 211 06 August 2014 at 9:51pm | IP Logged | ||||||||
Well, there's no one right way to measure comprehension. All you can do is explain your methods: "I took this chunk of text, and I measured comprehension as follows, and here are my results." But there will always be other, equally-valid measuring techniques that given different results. For example, somebody might measure their vocabulary size as follows:
Sure, it's a highly subjective process, and any two given people would get different answers. You could easily define other ways to measure comprehension: 1. "I could follow 40% of that TV series" might mean "I could understand the topic of conversation 40% of the time, and get about 40% of the jokes." 2. "I could understand 90% of that TV series" might mean "I could transcribe 90% of the sentences accurately, including all the fiddly details, explain to you how the grammar worked, and translate it accurately. Well, except for the stuff which I'm sure I understand, but I'm actually wrong about. The other 10% of the sentences I might miss anything from a pronoun to the entire sentence." Either of these is a perfectly sensible way to measure listening comprehension. But if you want people to understand clearly, just explain how you counted. I find this kind of approximate measure very useful when saying things like: - "If you can't follow at least 40% of a TV series, extensive methods may not help you much." (A general rule of thumb, where accuracy is of little importance.) - "My third book was 60% known, 30% decipherable and 10% opaque. My fortieth book was 95% known, >4% transparent, and <1% opaque." (A relative measurement of a single learner's progress, with no comparison to anybody else's self measurement. With a consistent methodology, this might be vaguely useful.)
As a learner, I care about things like: 1. Can I understand "Elle n'est pas venue hier" just by looking at it? 2. Can I produce a sentence like "Elle n'est pas venue hier" automatically, just like I would in English, without ever mentally conjugating anything? 3. Can I produce variants like "Elle n'est pas venue voir ma fiche", "Elle n'est pas descendue", and "Elle a descendu l'escalier" without thinking about it? 4. Do I use variants like "Je ne suis pas venu", "J'suis pas venu" and "Chuis pas venu" in appropriate contexts? If the answers to these questions are "Yes", then I don't actually care whether or not I can identify an auxiliary verb correctly, or explain when to use which. Seriously, if I find that I'm actually thinking about about the difference between avoir and être when I'm trying to explain an idea to somebody in French, I already know it's going to be a rough conversation. At higher levels, this stuff should be automatic, just like it is for most adults in their native language. Maybe a little bit of quick conscious patch-up here and there is OK, but not so much that it slows down speaking or derails a train of thought. This isn't to say that grammar is useless or anything—it can definitely help puzzle things out at lower levels, or when writing, or when speaking with a patient native speaker. Edited by emk on 06 August 2014 at 9:57pm 10 persons have voted this message useful | |||||||||
| Arekkusu Hexaglot Senior Member Canada bit.ly/qc_10_lec Joined 5383 days ago 3971 posts - 7747 votes Speaks: English, French*, GermanC1, Spanish, Japanese, Esperanto Studies: Italian, Norwegian, Mandarin, Romanian, Estonian | Message 3 of 211 06 August 2014 at 10:06pm | IP Logged | ||||||||
|
On the one hand, you are right, even if a person recognizes every word they come across, they are unable to evaluate the extent of their own ignorance: how could the learner know that the meaning they attribute to a word covers all of its semantic fields, or to what degree their understanding of the range of the language's available grammatical devices and their passive role in the attribution of nuances is incomplete.
If L2 comprehension is to encompass both these levels, then no impromptu self-evaluation can ever be an representation -- albeit approximate -- of the learner's true understanding. However, the level 2 you mention plays a lesser role in comprehension than it does in active production. We either accept its minor role in comprehension or else we entirely negate ourselves the ability to evaluate our own sense of understanding and progression. Interestingly, I can't remember anyone ever attributing a percentage value to their production ability in a second language... 2 persons have voted this message useful | |||||||||
| YnEoS Senior Member United States Joined 4256 days ago 472 posts - 893 votes Speaks: English* Studies: German, Russian, Cantonese, Japanese, French, Hungarian, Czech, Swedish, Mandarin, Italian, Spanish | Message 4 of 211 06 August 2014 at 10:11pm | IP Logged | ||||||||
|
A big part of communication is understanding context and intention, and I think most users of the forum understand implicitly when someone says they understood __% of a text, they're not making some objective measure of their understanding through some scientifically valid process. Rather, they're giving their impression of how much they understood, and most readers will successfully understand they're giving their impression with all the cognitive biases that come along with it.
Whether or nor their impression is accurate, isn't necessary or useful in casual discussion. It might be useful in other contexts though. 3 persons have voted this message useful | |||||||||
| luke Diglot Senior Member United States Joined 7207 days ago 3133 posts - 4351 votes Speaks: English*, Spanish Studies: Esperanto, French | Message 5 of 211 06 August 2014 at 11:27pm | IP Logged | ||||||||
This is an area where I think FSI styled drills are helpful. Continuing with the French example. I noticed my comprehension improved noticeably when I started FSI Basic French. It opens of course with some of the most common language elements such as gender. Although I had a grammatical and intuitive understanding of gender, I didn't have a lot of practice. Developing some automaticity with this in a spoken context helped my comprehension. Similarly, many of the grammar points in the second quarter of the course have to do with pronouns. They're another big part of the comprehension picture, although the pronoun word count is small. As I look on to other parts of the FSI course, it does appear they front loaded many key grammatical points. As I complete each unit, the general comprehension fog clears a bit more. I know it's not the vocabulary, because each unit doesn't cover much that's new. That's not to discount the importance of vocabulary, but words are just half the picture. The FSI Basic Spanish course is very good as well. So far, I think it's better than the French course. 1 person has voted this message useful | |||||||||
| iguanamon Pentaglot Senior Member Virgin Islands Speaks: Ladino Joined 5264 days ago 2241 posts - 6731 votes Speaks: English*, Spanish, Portuguese, Haitian Creole, Creole (French) | Message 6 of 211 07 August 2014 at 12:09am | IP Logged | ||||||||
Not being a linguist and never having studied linguistics, but having learned two languages to a high level I'm reminded of the famous quote by US Supreme Court Justice Potter Stewart when asked to decide if the movie "Les Amants" was pornographic-
When I can zip through a page without having to strain or look up more than a word or at most two in a few pages, when I can follow the speech or audio without problems, when I can keep a conversation going and add to it, that's comprehension. I know it when I see it. Edited by iguanamon on 07 August 2014 at 12:24am 5 persons have voted this message useful | |||||||||
| Serpent Octoglot Senior Member Russian Federation serpent-849.livejour Joined 6599 days ago 9753 posts - 15779 votes 4 sounds Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish | Message 7 of 211 07 August 2014 at 1:01am | IP Logged | ||||||||
And does your experience with Finnish and Estonian confirm it? I can't imagine understanding them without being able to identify the case and function. With the many forms, at first it was actually much easier for me to produce them than to understand. As for the original question, it's just too theoretical or philosophical. HTLAL is generally a very practical forum. The following questions/requests are relevant to many people, and this kind of stats is useful for that: -how many words do i know? please recommend a good source for learning more at my level. -I understand only 50% of the words in the book I'm reading. a) how can I learn to understand more? b) what easier books can I read? -I understand most words but they don't fit together, what to do? Edited by Serpent on 07 August 2014 at 1:49am 4 persons have voted this message useful | |||||||||
| Arekkusu Hexaglot Senior Member Canada bit.ly/qc_10_lec Joined 5383 days ago 3971 posts - 7747 votes Speaks: English, French*, GermanC1, Spanish, Japanese, Esperanto Studies: Italian, Norwegian, Mandarin, Romanian, Estonian | Message 8 of 211 07 August 2014 at 1:57am | IP Logged | ||||||||
I would probably have placed the understanding of a word and its affixes in the level 1 category, but I suppose it's borderline. The learner would still have an immediate impression that he does or doesn't understand it. 1 person has voted this message useful |
|  | 
Printable version
|
| Copyright 2024 FX Micheloud - All rights reserved No part of this website may be copied by any means without my written authorization. |
