|
 |
Is comprehension measurable? | ||
Share with:
| ||
 Language Learning Forum : General discussion |  | |
| Serpent Octoglot Senior Member Russian Federation serpent-849.livejour Joined 6407 days ago 9753 posts - 15779 votes  4 sounds Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish | Message 89 of 211 17 August 2014 at 5:36pm | IP Logged | |||||
What is this based on?
I've already mentioned one possibility, which is estimating/counting the number of sentences and applying the emk scale or any other scale. Sometimes a single word can account for not understanding a whole sentence, but well, that's just the reality of language learning. If you watch with L1 subtitles, you can count the whole content as 100% and then subtract what you didn't understand. With a bit less precision you can also consider the amount of content in a typical movie/episode as 100%, and see whether it's more likely that you missed some things or there's less content than expected. Or you can count/estimate the time you spent being clueless. There are many possibilities, really. 1 person has voted this message useful | ||||||
| Cavesa Triglot Senior Member Czech Republic Joined 4819 days ago 3277 posts - 6779 votes Speaks: Czech*, FrenchC2, EnglishC1 Studies: Spanish, German, Italian | Message 90 of 211 17 August 2014 at 6:28pm | IP Logged | |||||
I think there lies the key. "If we compare..." seems to be the trouble. But why would we do such a thing? It is just self assessment, it is not meant for comparison with anyone else than yourself. If we want to compare ourselves with anyone else, we should go and sit an exam, that's true. Or we could translate the same passage of a text without using a dictionary and compare or whatever. But noone has tried so far, at least to my knowledge, to compare person A's 75% with person B's 80 or whatever. So, why should we search for remedy to a non existent problem? :-D 1 person has voted this message useful | ||||||
| s_allard Triglot Senior Member Canada Joined 5240 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 91 of 211 17 August 2014 at 8:17pm | IP Logged | |||||
|
Again, if I must repeat myself, no-one prevents anybody from using any system of self-assessment. And one can
assign any values and give them any meaning one desires. But why do we have measurement standards in general, and in languages in particular? It's to give values that everyone can understand, The problem isn't really the comparison between speakers. It's some of that, but the fundamental problem is what does the observer, the reader, understand. When I read that someone like Cavesa understands 85% of a tv series, I inevitably refer to my understanding of 85% because that is what I read. Now when Cavesa tells me that 85% really means something else, I must decide whether I agree with that figure 85%. If someone tells me that the outside temperature is 20 Celsius according to a thermometer, I know what this temperature means and dress appropriately. I don't ask what does 20 C means. I don't see anybody saying that according to them 20 C means what I would call 25 C. There is no controversy. But here we have people saying I understand 50% of a movie and, if I ask how did they arrive at that figure, the answer is I can define 50% any which way because it is for my personal use. So Cavesa's 85% doesn't correspond to my understanding of 85%. I have to ask myself what does Cavesa really mean. There is no problem of people questioning other people's self-assessment here at HTLAL because we are polite and there is no reason to question other people's judgment. But that does not mean that we don't have questions. When I see a list of eight "speaks" in a person's profile, I don't jump to the conclusion that that person is a fake. But I am curious as to how well that persons speaks eight languages. Similarly, when I read these percentages for comprehension, I interpret these figures to the best of my ability. Someone says to me that they could understand 50% of Spanish television. I try to interpret this. The image in my mind is that the person doesn't understand half the time. I also question how one can follow or enjoy something where you understand half the time. We saw that the authors Hu and Nation, like many others, arrived at the observation that you have to understand 98% of the words of a piece of fiction to be able to read with ease and enjoyment. It doesn't mean you can't read with less. It means that it won't be an enjoyable experience. That's the reason why I use a very simple three value non-numeric scale for comprehension. I don't see the value or enjoyment of reading or listening to something where I'm regularly in the dark because I don't know the meanings of the words being used. 60% and 80% mean exactly the same to me in this context because a) there's no objective method for arriving at these figures and b) in terms of enjoyment, pleasure or satisfaction of the text it makes no difference. Either I can enjoy the work fully or I have some partial understanding that is unsatisfactory. 1 person has voted this message useful | ||||||
| Cavesa Triglot Senior Member Czech Republic Joined 4819 days ago 3277 posts - 6779 votes Speaks: Czech*, FrenchC2, EnglishC1 Studies: Spanish, German, Italian | Message 92 of 211 17 August 2014 at 10:46pm | IP Logged | |||||
|
Again, I can see your point right from the beginning but I believe you are trying to repair something that isn't broken because noone minds the % are just approximate guesses. So again:
1.the % are used because some of us just are not satisfied with a three or four value scale as such a scale goes directly against some of our needs. Especially the need to see even little progress and to set small goals for ourselves. Your "I enjoy fully or not at all" approach is just not that widely shared. I believe being able to enjoy small pieces of progress is the key to burnout prevention. 2.non-numeric scales are just as bad or even worse. We've seen many of non-numeric scales already in many contexts. And "I understand just the main ideas" or "I am an advanced beginner" or "I understand quite a lot" are causing even worse confusion and can mean just as many things as there are learners. In the end, they are the same as the %s when it comes to being vague and totally uncentralized. 3.if the % sign is what makes those guesses appear too arogant or whatever, than take it as if some of us had a scale similar to yours, just with 100 values closer to each other. 2 persons have voted this message useful | ||||||
| Serpent Octoglot Senior Member Russian Federation serpent-849.livejour Joined 6407 days ago 9753 posts - 15779 votes 4 sounds Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish | Message 93 of 211 17 August 2014 at 11:34pm | IP Logged | |||||
|
Prof Argüelles also counts the words, btw. He also learns vocabulary through extensive reading with 80% known words.
I disagree that there's no difference in the pleasure from 60% and 80%. Even if we don't take into account the way your lack of knowledge can add more mystery to a boring book... As for 20C, just look at a more detailed forecast. They often state how many degrees the weather "feels like", taking into account the wind, sun etc. And haven't you ever said something like "well, my car says it's +10, but it doesn't feel quite so cold"? Edited by Serpent on 17 August 2014 at 11:39pm 3 persons have voted this message useful | ||||||
 emk Diglot Moderator United States Joined 5342 days ago 2615 posts - 8806 votes Speaks: English*, FrenchB2 Studies: Spanish, Ancient Egyptian Personal Language Map | Message 94 of 211 17 August 2014 at 11:53pm | IP Logged | |||||
You probably can't enjoy The Game of Thrones or Engrenages with 50% comprehension, because these are complicated shows designed for very involved viewers. Many times, viewers are expected to infer what's going by piecing together scattered hints. On the other hand, a show like Scooby Doo is almost entirely predictable: some villains dress up as monsters, everybody assumes it's a supernatural problem, the heroes arrive and blunder around like idiots, and eventually the heroes pull a rubber mask off the villain, who says, "And I would have gotten away with it, too, if it weren't for you meddling kids!" A good example in French might be Code Lyoko, which is harder to figure out at first glance, but which ultimately turns out to be almost equally predictable. Shows like this are much easier to watch with partial comprehension, because it's harder to lose track of the plot. When I started watching Buffy, I would say that I "followed" about 40% of the dialog. Or, if numbers bother you, "A bit less than half the time, I knew what the characters were talking about, and I could understand the major points of the conversation. The rest of the time, I was mostly lost." I actually enjoyed this quite a bit, though I occasionally had to pause and ask my wife what on earth was going on. After five seasons, there were episodes where I could have done a dictée and written out a large fraction of the dialog with minor errors. Even Krashen has pointed out that a minority of students can happily read or watch with far less than 98% comprehension. He says it's more a matter of temperament than anything else. 2 persons have voted this message useful | ||||||
| YnEoS Senior Member United States Joined 4064 days ago 472 posts - 893 votes Speaks: English* Studies: German, Russian, Cantonese, Japanese, French, Hungarian, Czech, Swedish, Mandarin, Italian, Spanish | Message 95 of 211 17 August 2014 at 11:53pm | IP Logged | |||||
The disagreement I keep having isn't that I think you're implying that people shouldn't be able to communicate with their system of choice. Its just that it always comes across that you think there's a better way of communicating. I'm not trying to argue that this is just a fun community and we don't need to be so serious about percentages because we know we're just estimating. I'm saying that when context is understood, people are communicating exactly what they're trying to communicate as efficiently as possible, and this cannot be improved upon. When someone uses percentages, its incredibly useful for quantifying their impressions of their comprehension, and there's no better way of communicating that. I would never assume it was an accurate measure of their actual comprehension, nor that it would match up to my own system. But when someone one month says "I understood 75% of Harry Potter 1" then a few months later "I understood 85% of Harry Potter 2" then a few months later "I understood 95% of Harry Potter 3" that makes it much easier perceived sense of progress than "I understood quite a bit of Harry Potter 1, quite a bit more of Harry Potter 2, and almost all of Harry Potter 3". If I wanted to compare those %s to my own %s, or get a more accurate sense of their actual abilities, I would ask for clarification. But this clarification wouldn't be more efficient or more accurate communication, it would simply be communicating different information towards a new and different purpose. Edited by YnEoS on 17 August 2014 at 11:55pm 3 persons have voted this message useful | ||||||
| s_allard Triglot Senior Member Canada Joined 5240 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 96 of 211 18 August 2014 at 2:12am | IP Logged | |||||
First a minor point. I nor Paul Nation is talking about 98% comprehension. The figure was used with reference to word coverage. Let's not confuse word coverage and comprehension, which is at the heart of the debate. Here is what Hu and Nation said: ""This conclusion must not be interpreted as saying that with 98% coverage of the vocabulary no other skills or knowledge are needed to gain adequate comprehension. All of the subjects in this study were readers in their first language, had considerable knowledge of English grammar, were experienced in reading English, and brought considerable background knowledge to their reading. These all contribute to their skill in comprehending text and account for some learners reading the 95% and 90% versions getting high scores. However, as readability studies show, vocabulary knowledge is a critical component in reading." When we are talking about comprehension of objects or products where the text coexists with music, symbolism, visual and audio-visual components, the issue of what is comprehension becomes even more complicated. How does one understand poetry? Or advertising where there is often all kinds of cultural references and plays on words? Or songs where the music is often more important than the words themselves? Take for example cartoons on television that are aimed at a young audience. When we claim to understand a cartoon we are referring to more than the words of course. We also have sound and images. I would argue that the words are not totally essential to understand and even enjoy watching these programs. Most of the story is in the images. I even think that young children often don't understand the dialogs and still enjoy watching. You could probably turn the dialogs off and still appreciate the show could see some learner not understanding half the words and still enjoying such a program. Obviously, there are many movies and television programs where dialog is very important. For example, the recent well-known series Sex And The City is known for its very witty and clever dialogs. If you don't understand the dialogs, you are missing two-thirds of the show. I should also point out that in its heyday the show was watched by many women for the fashions. Another very important component of the show was its New York Cityness. The City was a silent and omnipresent character in the film. If you had never been to New York City or were not from there, certain things in the program didn't make sense, Can you enjoy the show and not understand all the dialogs? Of course, you can. The non-verbal elements are part of your appreciation, but in a show like this the words are extremely important. When a learner of English says to me that they understand 50% of Sex And The City, I am not sure what that means. Is it because half the words are unknown to them? Or is it all the complex idiomatic usage and plays on words that make the dialogs so witty? And why 50%? Why not 40% or 60%? I know some people will say that it doesn't matter. The important thing is that a month later the person understands 70% and then 80%. That 50% is not really 50%. It's a purely place marker, a starting point. It could just as well have been 40%. The important thing is that the learner will make progress relative to this point. I understand this, and I don't know how many times I have to repeat that I totally agree that for pedagogical purposes and personal benchmarking, it can be useful for individuals to gauge their progress in terms of percentages. But the fact remains these figures are not measurements of comprehension; they are estimates or guesstimates at best. Who has here has actually measured their comprehension of anything? I see figures flying around without the slightest statistical foundation. As I have said, emk is the only one who has come remotely close using a method that I may dispute but respectfully acknowledge. 1 person has voted this message useful |
|  | 
Printable version
|
| Copyright 2024 FX Micheloud - All rights reserved No part of this website may be copied by any means without my written authorization. |
