|
 |
Is comprehension measurable? | ||
Share with:
| ||
 Language Learning Forum : General discussion |  | |
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes  Speaks: French*, English, Spanish Studies: Polish | Message 65 of 211 13 August 2014 at 5:43pm | IP Logged | |||||
I like this post because it demonstrates well the utility of word counting. "..it's excellent motivation to see the number improve". As a learning tool, word counting is useful. I have said as much. I am not against word counting. . I just don't believe that it is an accurate measure of comprehension, for all the reasons that I've already given. 1 person has voted this message useful | ||||||
| rdearman Senior Member United Kingdom rdearman.orgRegistered users can see my Skype Name Joined 5238 days ago 881 posts - 1812 votes Speaks: English* Studies: Italian, French, Mandarin | Message 66 of 211 13 August 2014 at 5:55pm | IP Logged | |||||
Language proficiency isn't directly related to comprehension. The fact someone can use "various formulaic expressions, often composed of high-frequency words" gives no indication of ability to comprehend difficult text or native speech. Look at heritage speakers with 100% comprehension, but who might struggle to form sentences. In the heritage speaker example you have the opposite of what the expert says; a large passive vocabulary with high comprehension and a low proficiency. 2 persons have voted this message useful | ||||||
| luke Diglot Senior Member United States Joined 7207 days ago 3133 posts - 4351 votes Speaks: English*, Spanish Studies: Esperanto, French | Message 67 of 211 13 August 2014 at 6:16pm | IP Logged | |||||
The modifiers Nation uses are important in representing his position accurately. Edited by luke on 13 August 2014 at 6:17pm 2 persons have voted this message useful | ||||||
 emk Diglot Moderator United States Joined 5534 days ago 2615 posts - 8806 votes Speaks: English*, FrenchB2 Studies: Spanish, Ancient Egyptian Personal Language Map | Message 68 of 211 13 August 2014 at 7:18pm | IP Logged | |||||
I like this kind of "functional comprehension" test, because it focuses on what I consider important: Can a learner use the language in the real world? But there are some problems. For example, a B2 student with good test-taking skills should be able to answer concrete questions about most real-world texts, and do so with a high degree of accuracy. A C1 student should be able to do all that, and do it quickly. This means that test makers need to use very sophisticated texts, and ask students tricky questions that require careful analysis of the text. The usual problem is that, past a certain point, the texts become so dense, and the questions become so subtle, that even native speakers have to think long and hard before choosing the correct answer. Even when preparing for the DELF B2, there were sample questions where both my tutor (a university-educated native speaker who knew the exam format very well) and I would agree on an answer, but the official answer key would require a different one. Given these problems—and given that I'm assessing my own comprehension—I find it a lot easier to read through a sample text and categorize my comprehension of each piece of the text: Do I understand something easily? Do I need to puzzle it out? Do I need to guess outright? As s_allard kindly showed, this process is not without its pitfalls: I may read too quickly, and thereby mistake a belltower for a bell. But overall, it provides a way to estimate personal comprehension of a text without using murky texts and tricky questions. And it gives better resolution at B2+: There's a real difference between four opaque words on one page, and four pages with a single opaque word. These finer degrees of comprehension are to measure using comprehension exams, and it's often more efficient to go through a sample text in fine detail. Again, this won't produce "exact" numbers, because no such numbers exist. The best that can be done is to state a measuring procedure, demonstrate how it used, and see whether is generates reproducible results within some margin of error. And even imperfect numbers can be very useful for measuring personal improvements at the upper levels. Edited by emk on 13 August 2014 at 7:36pm 4 persons have voted this message useful | ||||||
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 69 of 211 13 August 2014 at 7:20pm | IP Logged | |||||
Let me see if I understand this post. If language proficiency - I suppose speaking proficiency - isn't directly related to comprehension, what is it related to? Comprehension is a component of proficiency. Can one be a proficient speaker and have low comprehension? In other words, can one speak better than one can understand? I think not. It's usually the other way around. Let's now look at heritage speakers. Can we assume they have 100% (sic) comprehension? Of course not. And just what is this 100% comprehension? If we mean by heritage speakers those individuals who have learned a language at home but not in the country of origin and are not schooled in the language, it's probably safe to say that their level of comprehension of the language, especially the written language, is not the same as that of an individual raised and educated in the country of origin. Describing heritage speakers as having "a large passive vocabulary with high comprehension and a low proficiency" is totally wrong. Compared to native speakers, heritage speakers are probably lower on every metric except pronunciation. 1 person has voted this message useful | ||||||
| emk Diglot Moderator United States Joined 5534 days ago 2615 posts - 8806 votes Speaks: English*, FrenchB2 Studies: Spanish, Ancient Egyptian Personal Language Map | Message 70 of 211 13 August 2014 at 7:45pm | IP Logged | |||||
Two things seem to be worth noting about heritage speakers: 1. Their receptive skills are often several CEFR levels better than their productive skills. This happens because they've often received years of input in the minority language, but they've either never made an effort to speak, or they abandoned that effort early in childhood. 2. Heritage speakers vary enormously. Some have A2 listening comprehension and no output. Others can mostly understand television but struggle with anything but the simplest conversations. Others are effectively native speakers with the vocabulary of a 5-year-old. If you've spent the first twenty years of your life listening to relatives speak a minority language, watching television in it, and reading your relative's Facebook posts, it's perfectly plausible that your listening comprehension is excellent. And yet I know people like that who are genuinely distraught by their inability to actually speak the language, and who wish they could speak it as well as I speak French (which is far below native levels). Mind you, a heritage learner with excellent comprehension and poor speaking skills can probably learn to speak very quickly. I only think that this imbalance can exist so long as the heritage learners can get away with listening in the minority language and replying in the community language. 4 persons have voted this message useful | ||||||
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 71 of 211 13 August 2014 at 7:58pm | IP Logged | |||||
|
As emk has rightly pointed out when assessing one's own comprehension, it can be very helpful to look at it in
terms of categories of words on a page. It goes without saying that comprehension is certainly related to the number of individual words one can decipher and understand on a page. That is not the issue. The issue is whether we can measure directly comprehension by such a method, especially in a testing environment. How do I know that you understand something when you say that you do understand? You say that you understand 80% of the words. How do we know that you understand these words accurately? And if someone says they understand 50% of a text, what is the significance of this? What is the value of understanding half a text? How could one test for comprehension using a method based on word counting? Do you present every word of the text and ask, Do you understand this word? For all these reasons, the test designers look more at what the person can do based on the information in the text. Designing good tests is a science and not necessarily very easy. This is why a lot of time is spent on calibrating the tests with native speakers. And this is also why in the Canadian government tests a number of dummy questions are included. They can be used to validate the accuracy of the good questions. As pointed out earlier, Paul Nation's work on vocabulary and text coverage states that a 95% coverage (knowing the words) is necessary for good comprehension. ( I don't think Nation ever speaks of measuring comprehension. ) Anything less than 95% coverage means that it is difficult to understand the text. This is exactly why I have used a three-level scale of comprehension: All, Some, Nothing. For me 95% coverage and up means All, Nothing is I have a absolutely no clue, perhaps because of the writing system. Some is in between, anything below 95% and above Nothing. 1 person has voted this message useful | ||||||
| s_allard Triglot Senior Member Canada Joined 5432 days ago 2704 posts - 5425 votes Speaks: French*, English, Spanish Studies: Polish | Message 72 of 211 13 August 2014 at 8:14pm | IP Logged | |||||
|
I think that my characterization of heritage speakers was somewhat incorrect. I was thinking relative to native
speakers. Generally speaking, heritage speakers have better oral comprehension than speaking skills. Reading skills tend to be very low because they are not formally educated in the language. As emk has pointed out, heritage speakers can vary considerably in their degree of contact with the language and their proficiency. The key factor is usually formal schooling in the language. For this reason, many immigrant communities provide some form of additional schooling in order to at least maintain the language. In many cases, the heritage language is the language of the home, religion and relations within the family. This is not conducive to developing a "large passive vocabulary." 1 person has voted this message useful |
|  | 
Printable version
|
| Copyright 2024 FX Micheloud - All rights reserved No part of this website may be copied by any means without my written authorization. |
