|
 |
Is comprehension measurable? | ||
Share with:
| ||
 Language Learning Forum : General discussion |  | |
| James29 Diglot Senior Member United States Joined 5377 days ago 1265 posts - 2113 votes  Speaks: English*, Spanish Studies: French | Message 49 of 211 09 August 2014 at 1:44pm | IP Logged | ||
|
To the original point, great question! Humans seem to have a very strong desire to measure things that are simply not measurable. I think it makes people feel good to remove some sort of unknown. I think this is one of the examples.
I don't think the concept of "understanding" can be reduced to a mathematical calculation, period. In terms of measurement, it is a concept like desire. You can desire one car more than another, but you cannot desire it 90% more or twice as much. Similarly, you can understand one book more than another, but you cannot understand it 90% more or twice as well. Math just does not work that way with these types of concepts. Who so clearly explained the problems with defining what you know you know? Does this quote come to mind? "as we know, there are known knowns; there are things that we know that we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns, the ones we don't know we don't know." Donald Rumsfeld. 2 persons have voted this message useful | |||
| Bao Diglot Senior Member Germany tinyurl.com/pe4kqe5 Joined 5768 days ago 2256 posts - 4046 votes Speaks: German*, English Studies: French, Spanish, Japanese, Mandarin | Message 50 of 211 09 August 2014 at 1:55pm | IP Logged | ||
|
As I mentioned before, it should be possible to use well-made multiple choice quizzes for reading and listening comprehension. You would have calibrate the quiz using native/proficient speakers and learners at different stages, and then it should be possible to come up with a relatively objective result in percentage.
I don't know how useful that would be though. 2 persons have voted this message useful | |||
| Serpent Octoglot Senior Member Russian Federation serpent-849.livejour Joined 6599 days ago 9753 posts - 15779 votes 4 sounds Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish | Message 51 of 211 09 August 2014 at 2:24pm | IP Logged | ||
So this is where Prof Argüelles got the term (known unknowns) for words that are at the point where you should look them up. There are also unknown knowns, though. In their purest form they are words that you can understand but have never seen. Decipherable and not automatic. Cognates and loan words, of course. Edited by Serpent on 09 August 2014 at 2:24pm 2 persons have voted this message useful | |||
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 52 of 211 09 August 2014 at 3:16pm | IP Logged | ||
This is exactly the approach taken by all testing agencies. I gave an example a Canadian government test where you read a short text and answer a question based on the content. I would still argue that those tests do not really measure understanding and work only for certain kinds of texts - how to you measure the understanding of fiction - but those tests work well for their intended purpose. However, much of the debate here is on counting words as a method of measuring comprehension. Edited by s_allard on 09 August 2014 at 4:36pm 1 person has voted this message useful | |||
| Serpent Octoglot Senior Member Russian Federation serpent-849.livejour Joined 6599 days ago 9753 posts - 15779 votes 4 sounds Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish | Message 53 of 211 09 August 2014 at 3:44pm | IP Logged | ||
|
The problem is that the questions tend to be either too easy or too difficult/interpretation-based. For example I recently did the Dialang tests in Spanish, and in some cases I understood very well but couldn't choose between similar options. Like, a dialogue between people speaking of the post-war situation in Spain. Nobody said anything positive about it, so it seemed logical to choose that they were complaining and not merely remembering (the remaining two options were clearly wrong). Since you basically need to learn to second-guess the test makers, that's not the best option for individual learners asking for advice on HTLAL. I trust someone's self-assessment more if they actually went through the checklists, rather than just took some sort of placement test.
3 persons have voted this message useful | |||
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 54 of 211 09 August 2014 at 5:11pm | IP Logged | ||
|
In a previous post, I made a mistake: I wrote "test" instead of "text." This was a major mistake because it
changed the sense of the entire sentence. It made me think of a fundamental problem of the word counting method for assessing comprehension: all the words are given the same value. What we all know is that in a given phrase or text certain words are more important than others. They tend to be content words but could be a function word such as a preposition or a verb form. I've experienced this many times when reading articles or listening to television where I understood everything except for some detail, but this detail was central to understanding the whole passage. The only solution was to consult a dictionary or a native speaker. A related problem of course is simply misunderstanding. You think you understand something but you don't really until it is pointed out to you, if ever it is. This happens to me all the time when working with my tutor. Actually, it happens more when speaking and writing but sometimes when reading. Earlier I pointed out that in a French sentence, the translator had mistakenly translated "clocher écroulé" as fallen bell rather than collapsed steeple. But what was really important here is that this person had understood fallen bell and only realized the mistake when it was pointed out. The only way this mistake came to light was because it was written down here. The fundamental question is: how do I know that I'm understanding something correctly? This is a particular problem with cognates and false friends. A word may resemble one in your native language but take on a different meaning in the other language. Edited by s_allard on 09 August 2014 at 5:14pm 2 persons have voted this message useful | |||
| YnEoS Senior Member United States Joined 4256 days ago 472 posts - 893 votes Speaks: English* Studies: German, Russian, Cantonese, Japanese, French, Hungarian, Czech, Swedish, Mandarin, Italian, Spanish | Message 55 of 211 09 August 2014 at 6:50pm | IP Logged | ||
|
I'm sorry to keep harping on this issue, so I'll try not to be too long and not to repeat myself. And I'll shut up after this, if this line of discussion isn't appreciated or helpful.
But there's a reason a lot of philosophers stopped doing pure philosophy in isolation and started making more specific goals like the philosophy of science or the philosophy of ethics. Because when you ask big questions like "what is the meaning of life" you end up spending thousands of years debating what we mean by "meaning" which has many different usages and often people end up arguing over different things because they're using fuzzy unspecific words. I think there's mistaken assumption in this discussion is that "comprehension" and "understanding" refer to real things which can be measured, and aren't just fuzzy words that are used to refer to different things in different situations. So if someone says "comprehension" and they talk about the % of words they recognized, they're using "comprehension" to mean the % of words they recognized, and to discuss the other situations where "comprehension" might refer to grasping the intention of the author, is to misinterpret the person and impede conversation. If you just flip things around and instead of asking "what does this fuzzy word "comprehension" really mean?" and start asking "how is this fuzzy word "comprehension" being used in this context, is there a more specific word or sentence that can help clarify what we're really trying to get at in this discussion?" suddenly things can progress forward. So I could say something like understanding the words and understanding the grammar helps you understand the sentence better. But this is vague and each time I say "understanding" I could be using more specific terminology. It might be better to say that being familiar with all the words and the grammar being used improves your likelihood of grasping speaker's intention. Two people might have different levels of knowledge of grammar and vocabulary and still make the same assumption about the intention of the speaker because the context makes it so obvious. This doesn't get clarified by asking "what is understanding?" like you can break it apart and look at the different levels and parts, rather the word "understanding" is referring to different things. People use understanding of vocabulary, grammar and context, to understand the meaning of a sentence, but these aren't all layers of the same concept, they're separate skills used for accomplishing a task. Apologies if it sounds like I'm trying to say certain discussions should or shouldn't be had. I just think perhaps a lot of different issues are being lumped together with the idea that one single answer exists for all of them. I think things like "what skills should a learner be aware of to make sure they are actually understanding things they read", "what are some useful ways a learner can convey their level of comprehension to other learners", and "how should test makers format their test to create useful measures of different skill levels" are fundamentally different questions that require different sorts of answers. 7 persons have voted this message useful | |||
| Bao Diglot Senior Member Germany tinyurl.com/pe4kqe5 Joined 5768 days ago 2256 posts - 4046 votes Speaks: German*, English Studies: French, Spanish, Japanese, Mandarin | Message 56 of 211 09 August 2014 at 7:02pm | IP Logged | ||
As I mentioned, multiple choice quizzes where you know that one of the given options is the 'correct' one make it easy to cheat. Err, I mean, use test-taking skills instead of real comprehension. And I mentioned that this can be counteracted by offering a list of statements about the text and letting the examinee choose between yes - no - doesn't say with no predictable distribution for every single one. If, let's say, a learner can make five correct statements about a paragraph like the one in your example, chances are that they 'understand' the paragraph. The same can be done with fiction. It could also include statements about whether a certain phrase is meant metaphorically, about the narrator in a passage etc. Of course, that doesn't mean that they necessarily understand it exactly the same way the author intended, but then again, who does? And that is where using native speakers to calibrate the quiz would make a difference, because that would compare the examinees to native speakers' results, and not just to what the test makers think should be understood in a given paragraph. (And it'd make it much more expensive to create the test.) Edited by Bao on 09 August 2014 at 10:43pm 3 persons have voted this message useful |
|  | 
Printable version
|
| Copyright 2024 FX Micheloud - All rights reserved No part of this website may be copied by any means without my written authorization. |
