|
 |
Is comprehension measurable? | ||
Share with:
| ||
 Language Learning Forum : General discussion |  | |
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes  Speaks: French*, English, Spanish Studies: Polish | Message 9 of 211 07 August 2014 at 2:32pm | IP Logged | |||||
The idea of this grammatical level of comprehension is not one of being able to make a grammatical analysis of the phrase. It's having an implicit understanding of the grammar that determines the ability to produce all those variants that emk has pointed out. Why do we make mistakes in our target language? Why does an English-speaking learner of French say Elle n'a pas venu hier, as I hear often? This morning in my weekly Spanish tutoring class I said Del momento no tuve...instead of Por el momento no tuve... In all these cases, the mistakes stem from the fact that the user does not have an implicit understanding of the underlying structure. At some point the learner of French has to learn and interiorize that certain verbs take the auxiliary être and negation is indicated by ne...pas. Afterwards, they can completely forget about it but not forget it. Edited by s_allard on 07 August 2014 at 4:47pm 2 persons have voted this message useful | ||||||
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 10 of 211 07 August 2014 at 2:49pm | IP Logged | |||||
I agree. We know comprehension when we see it. That's not the question. The question is how do you measure it. Right now I'm reading García Marquéz's 12 cuentos peregrinos and thoroughly enjoying it. From time to time I have to look up some words. What is my level of comprehension? Is it 90% or maybe 98%? I could just say who cares but I'm sort of curious because a lot of people talk about these percentages. 1 person has voted this message useful | ||||||
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 11 of 211 07 August 2014 at 3:52pm | IP Logged | |||||
It is certainly true that one can use any subjective method to define comprehension. But that does not make that method right. I am questioning the validity of the method. I am not questioning the right to use any method that one feels like using. Let's take the example of vocabulary size. It seems pretty straightforward. I count the words I know on a page of a dictionary relative to the number of words I don't know and then multiply it by the number of pages in the dictionary. Sure, there are other methods of measuring vocabulary size but this is the one I use. it's rough but it's simple and it works. Thus we arrive at a given number that we can use in all sorts of statements such as "The average educated speaker of English knows 36,000 words." That's fine but there are people who object. There's a question of what exactly is a word. There are maybe function words and meaning words. What about prefixes and suffixes? Maybe we should be counting word families. What about the size of the dictionary? The bigger the dictionary, the bigger the vocabulary. What about phrasal verbs such as get over it, get by, get off, etc? What about idiomatic expressions? For some people, the dictionary method is not an accurate method for measuring vocabulary size. The fact that it gives a precise figure does not make it accurate; it has false air of accuracy. Other methods are proposed using various forms of sampling. Now, one can say who cares what other methods people use, I use my method because it suits my purpose. That's fine just as long as we all understand that when I say vocabulary size I'm saying vocabulary size according the s_allard's dictionary method. And the discussion ends there. Therein lies the problem. How can we compare results if we use different methodologies? Which one better reflects reality? As many people know, I question the use of words. When I read, "I didn't understand a thing", "I got the gist" and "I totally understood everything just as if it were in my native language" I don't have a problem. When I read, "I can understand this television program with 40% comprehension," my question is: how is that calculated? Does one take a transcript of the audio and count the known versus the unknown words? And what about the visual and audio component? Edited by s_allard on 07 August 2014 at 8:23pm 1 person has voted this message useful | ||||||
| Serpent Octoglot Senior Member Russian Federation serpent-849.livejour Joined 6599 days ago 9753 posts - 15779 votes 4 sounds Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish | Message 12 of 211 07 August 2014 at 4:04pm | IP Logged | |||||
|
It doesn't have to be scientifically accurate to be effective. HTLAL'ers generally have no interest in claiming they understand more than they really do because the more honest you are about your weak points, the better advice you get.
Also one more option is counting the number of sentences/clauses and measuring the percentage you understand. This can be done with video/audio too, although generally we just estimate and not literally count it. And yes, sometimes it's one individual word that makes you miss a whole sentence, and sometimes you understand all the words but not the meaning. All of this stuff is balanced out if you use a chunk that is large enough. 2 persons have voted this message useful | ||||||
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 13 of 211 07 August 2014 at 4:44pm | IP Logged | |||||
To illustrate some of these issues of dual-level comprehension, I'll look at a line from a popular French variétés song from 1924. I know that not everyone here knows French but that shouldn't be a problem je m'aperçus tout aussitôt qu'elle s'était fait couper les cheveux. Some readers here don't understand a thing. Some can make out something; others a bit more. Arekkusu, emk and advanced speakers understand the whole phrase. The level 1 meaning would be something like: I saw right away that she had had her hair cut. This is not really the meaning, it's a translation. As Arekkusu (and others) would point out, there are other and possibly better translations. Interestingly, one could say "I saw right away that she had cut her hair" even though she didn't actually cut it herself. There are around 13 words in this sentence. All of these are very common words in French. This is pretty easy French. Let's suppose you didn't understand cheveux. That leaves 12 words or 93% of the phrase that you did understand. Not bad. But cheveux is a key word here; if you don't understand it much of the sentence is meaningless. On the other hand, suppose you don't understand tout aussitôt, leaving you with 84% understanding. Is that correct? Statistically it is, but in fact the phrase it perfectly understandable. At the level 2, there is a number of things going. I won't go into the details here but there are a couple of quite complex issues of pronominal verbs in French grammar that you have to decode if this is to make sense. There's even a nice passé simple at the beginning. One could say that no listener has the time to do a grammatical analysis of this phrase. Most people don't even know what a pronominal verb is. You just understand the words and continue listening to the song. That's true but every native speaker has an awareness that this is correct French and could instantly detect an error. They have an implicit comprehension of the underlying workings of the phrase. Finally, Arekkusu makes an interesting remark: we don't talk about percentages when referring to our active abilities. Could I say: I speak with 75% ability? intriguing. Edited by s_allard on 07 August 2014 at 4:46pm 1 person has voted this message useful | ||||||
| YnEoS Senior Member United States Joined 4256 days ago 472 posts - 893 votes Speaks: English* Studies: German, Russian, Cantonese, Japanese, French, Hungarian, Czech, Swedish, Mandarin, Italian, Spanish | Message 14 of 211 07 August 2014 at 5:54pm | IP Logged | |||||
|
The problem with defining how much of a sentence you understand, is that you can keep adding clauses to make a statement more accurately infinitely, you never arrive at 100% accuracy and usefulness quickly diminishes. Can you define all the words in a sentence? Do you know their historical usages? Their local usages? Do you understand situational context that might determine how the word is being used in this instance? Most of the time we don't need all this information to understand someone, we make a reasonable assumption about what something means, and if our assumption is wrong we ask for clarification. To quote an example used in some philosophical discussions
"Hugo, engrossed in the paper, says, 'I need some milk for my coffee'. Odile replies, 'You know where the milk is'. Suddenly defensive, Hugo replies: 'Well I don't really know that, do I? Perhaps the cat broke into the refrigerator, or there was just now a very stealthy milk thief, or it evaporated or suddenly congealed'." The breakdown of communication here is not that Odile has spoken inaccurately or hasn't supplied enough information, it's that Hugo is ignoring the social context that most humans intuitively understand from experience. Even science isn't using the right method, it's using language specific enough to be useful so that others can reproduce the results. If you don't understand what someone means when they give a % of comprehension, you ask for clarification or additional information. If you want to conduct a scientific study you set a standard of language specificity to make sure your results are useful and reproducible and omit information that are is not particularly useful even though they may be related. Theoretical discussions in absence of a clear goal don't go anywhere. And ironically the problem here is actually that there are many factors are involved in how people use the word "comprehension" and its used to express several different things that aren't easily quantifiable. I can comprehend the meaning of a sentence by making an assumption about the speaker's intention without necessarily comprehending all the vocabulary and grammar involved. To follow this line of questioning is to illuminate the many different things people might mean by saying "comprehension", but it won't arrive at a scientifically sound method, nor improve communication between humans which seems to be functioning just fine otherwise. 7 persons have voted this message useful | ||||||
| Julie Heptaglot Senior Member PolandRegistered users can see my Skype Name Joined 6905 days ago 1251 posts - 1733 votes 5 sounds Speaks: Polish*, EnglishB2, GermanC2, SpanishB2, Dutch, Swedish, French | Message 15 of 211 07 August 2014 at 5:57pm | IP Logged | |||||
|
Sometimes I try to estimate my reading or listening comprehension. Apart from my most advanced languages, I rarely think about it in terms of the number of unknown words (per page, if it's about reading): I rather focus on how much of the message I understand. That's obviously by no means an objective measure (as pointed out above, we may not be aware of double/hidden meanings we just don't get) but it's better than nothing as it allows me to assess (albeit with an admittedly high degree of inaccuracy) how I progress.
I never estimate the level of comprehension for one sentence or a couple of sentences, I rather do it for some kind of language input (a given novel/podcast/TV show/newspaper/some kind of websites). Such an estimation might be helpful if I'm trying to decide what kind of language input I need or if I want to describe my skills in order to get some advice from other learners. Why don't we describe our active skills by means of percentage estimations? I think that, apart from the initial stages of learning, we are in most cases able to get the whole message across in our target language(s), we just tend to be more or less elegant (or correct) in our choice of words, grammatical constructions etc., which is why it doesn't make a lot of sense to talk about our active skills in percentages. Furthermore, if we wanted to give a percentage estimation of our active skills, we would need some kind of a reference point, some idea of what 100% would be. Native competence? If we are not native speakers, we cannot really estimate how far we are from there. I think that people generally tend to be much less aware of the language mistakes they make in speaking/writing than of their comprehension mistakes. With some language learning practice, you know when you're just guessing, you do notice if the story doesn't add up, you get suspicious, thinking that what you've just read might include an unknown idiom, a rare grammatical construction, a wordplay or a joke. Even though self-evaluation is a tricky thing, it's usually still much easier to self-evaluate our passive skills than the active ones. 2 persons have voted this message useful | ||||||
 emk Diglot Moderator United States Joined 5534 days ago 2615 posts - 8806 votes Speaks: English*, FrenchB2 Studies: Spanish, Ancient Egyptian Personal Language Map | Message 16 of 211 07 August 2014 at 6:05pm | IP Logged | |||||
Well, just to amuse myself, I took an entire page of a French book, and translated a page. I marked "opaque" portions in red, and "decipherable" (or "i+1") portions in blue. Anything which I could just read with no particular effort was left in black. Here's a short excerpt; the rest is available in the other thread.
A word like toiture "roofing, roof" is pretty obvious if you know toit "roof" and the suffix -ure, which often means "the trace left by an action." And I still had a little lingering interference between écrouler "collapse, cave in" and s'écouler "flow, pass". The only words which stumped my outright were contreforts "foothills" and some river-related terminology that I guessed incorrectly. Anyway, thanks to my problems with French geography, I scored the page as 94.5% "automatic", 3.5% "decipherable" and "2% opaque." I find a three-way distinction like this to be a bit more useful than dividing everything into "known" and "unknown", because there are always words like toiture "roofing" that are technically unknown but pose little challenge in context. And yes, this process is totally unscientific. I could have scored various words differently, and as my notes for each word show, the "decipherable" category includes a lot of fuzzy subcategories, ranging from "decipherable thanks to roots and context" to "I actually know this word, but since I'm translating it in public, I looked it up anyway to be safe" and even "That makes sense, but it's an unusual turn of phrase; is it slightly archaic?" And I didn't bother inventing a counting system for grammar comprehension, because it wasn't needed here. But even though this process is unscientific, it produces some ballpark numbers that can be useful, given appropriate explanations and caveats. If somebody says, "Hey, I tried doing extensive reading, and the text was 10% decipherable and 90% opaque, and it didn't work", then HTLALers will offer different advice than they would to somebody who says, "I'm having trouble learning new vocabulary from extensive reading, and typical texts are around 95% automatic, 4.5% decipherable and 0.5% opaque." In other words, we don't necessarily need scientifically validated and externally reproducible measuring techniques to compare notes, measure our personal progress, or share advice. 5 persons have voted this message useful |
|  | 
Printable version
|
| Copyright 2024 FX Micheloud - All rights reserved No part of this website may be copied by any means without my written authorization. |
