Register  Login  Active Topics  Maps  

Is comprehension measurable?

 Language Learning Forum : General discussion Post Reply
211 messages over 27 pages: << Previous 1 2 3 4 5 6 7 ... 16 ... 26 27 Next >>
Serpent
Octoglot
Senior Member
Russian Federation
serpent-849.livejour
Joined 6599 days ago

9753 posts - 15779 votes 
4 sounds
Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese
Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish

 
 Message 121 of 211
21 August 2014 at 1:31am | IP Logged 
Also it's ridiculous to assume that someone with a 75% comprehension won't understand simple words like number or boy. Someone who understood 75% of the whole book will understand more than 75% of this very paragraph (which is a bit untypical, since as emk mentioned, the beginning often features descriptions of the nature/settings etc).

Here's my version of what I'd assume if someone said they understood 75%. I'll replace the missing words with Portuguese (most of the members will have at least some associations) and Finnish (not familiar for most of the forum members, although two words won't be opaque).

Mr. and Mrs. Dursley, of number four, Alfeneiros Drive, were orgulhosos to say that they were perfectly normal, thank you very much. They were the last people you'd expect to se metessem in anything strange or mysterious, because they just didn't hold with such bobagem.

Mr. Dursley was the director of a firm called Grunnings, which made perfurações. He was a big, man with quase não any neck, although he did have very large bigodes. Mrs. Dursley was thin and blonde and had nearly twice the usual amount of neck, which came in very useful as she spent so much of her time espichando over garden cerca, espiando on the vizinhos. The Dursleys had a small son called Dudley and in their opinion there was no finer boy anywhere.


Mr. and Mrs. Dursley, of number four, Likusteri Drive, were ylpeitä to say that they were perfectly normal, thank you very much. They were the last people you'd expect to sekaantua in anything strange or mysterious, because they just didn't hold with such hölynpöly.

Mr. Dursley was the johtaja of a firm called Grunnings, which made poria. He was a big, man with tuskin any neck, although he did have very large viikset. Mrs. Dursley was thin and blonde and had nearly twice the usual amount of neck, which came in very useful as she spent so much of her time kurkotellen over garden aitojen, vakoillessaan on the naapurit. The Dursleys had a small son called Dudley and in their opinion there was no finer boy anywhere.


More details:
-if someone hasn't had formal schooling in the language, I'd not necessarily expect them to know the word "neck".
-"director" was replaced only in the Finnish variant, reflecting the fact that many get this particular word for this concept for free. the same applies to boss, but not other synonyms
-similarly, "strange and mysterious" shouldn't be too hard because you only need to know one of the two - and in many languages, mystical/mysterious will actually be more obvious
-at 75%, I'd expect one to understand at least one of proud, nonsense, moustache, neighbours. at 90% I'd expect one to understand all of them
-I removed the article before moustache because in Finnish it can only be plural here and the Portuguese translation uses plural too. I didn't always take the missing words from the translation though.
-I'd expect at least native speakers of European languages to associate Grunnings with something hard/heavy/metallic, maybe tools, or maybe just something boring. The exclusion of people who come from vastly different cultures is based on posts like this (phonosemantics and poetry go hand in hand). The same would apply to a Westerner learning Mandarin or Arabic, of course.
-I'd also expect most of the following things to be confusing:
1. thank you very much (in the ironical sense)
2. twice the usual amount of neck
3. came in very useful (the last word should make it clear, but someone who's used to formal learning and classes would feel the need to look up or underline the expression)
4. same goes for "hold with such nonsense", which I'd expect to be much clearer if it occured after the first 100 pages or so
5. the finer shades of "no finer boy anywhere" :)
6. maybe some other peculiar wording like "perfectly normal"
-My list obviously doesn't apply to diehard HP fans who know the text well.
-If someone was familiar with JKR's style in their native language (but not a crazy fan), I'd expect them to know when they are missing out on subtleties.
-Ignoring HP's popularity, I'd expect an educated adult to feel the author's disapproval of being so normal that it's boring, and to be aware from the beginning that adventures are coming to the Dursley family. At this point they could reasonably assume that Dudley would be more involved in the adventures though. At 75%, I'd also expect one to take the Dursleys as a moderate force in the story, definitely not the main villains.

Since it's not the most difficult part of the text, understanding it all can count for maybe 93-95% comprehension of the whole book. And understanding only half of what I mentioned would be around 80-85%. It's okay to miss an occasional high frequency word* if you get most of the subtleties.
*I only learned the word "destination" in English when I was 16 and came across the Finnish word for this.

Obviously that's just my opinion, and in reality my interpretation would rely on what I know about the learner (and the text, if I've read it).

Edited by Serpent on 21 August 2014 at 2:26am

6 persons have voted this message useful



luke
Diglot
Senior Member
United States
Joined 7207 days ago

3133 posts - 4351 votes 
Speaks: English*, Spanish
Studies: Esperanto, French

 
 Message 122 of 211
21 August 2014 at 1:34am | IP Logged 
s_allard wrote:
What is the difference between 25% and 75% comprehension?


50%
6 persons have voted this message useful



Serpent
Octoglot
Senior Member
Russian Federation
serpent-849.livejour
Joined 6599 days ago

9753 posts - 15779 votes 
4 sounds
Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese
Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish

 
 Message 123 of 211
21 August 2014 at 1:56am | IP Logged 
Also, if someone said "I read the first paragraph of Harry Potter and only understood 75%, should I try and read the rest of the book at my current level?" I'd say no. 75% comprehension of this specific paragraph is more similar to what s_allard described.

For the record, some exceptions where I'd recommend trying it would be:
-the person is a big fan but has only seen the movies
-they already speak a related/similar language. In the case of English, that would be a Germanic one (realistically, only German itself) or French
-the person has very little grammar knowledge but they are willing to start working on that NOW, along with reading the book
-or the opposite, the person has a strong aversion to grammar, but also doesn't want to read an easier children's book. I'd recommend LR or at least using a translation and making lots of SRS cards
-something similar to emk's story - your spouse speaks a different language, and currently they are reading HP with the kid, or another similar scenario

Edited by Serpent on 21 August 2014 at 1:56am

2 persons have voted this message useful



Serpent
Octoglot
Senior Member
Russian Federation
serpent-849.livejour
Joined 6599 days ago

9753 posts - 15779 votes 
4 sounds
Speaks: Russian*, English, FinnishC1, Latin, German, Italian, Spanish, Portuguese
Studies: Danish, Romanian, Polish, Belarusian, Ukrainian, Croatian, Slovenian, Catalan, Czech, Galician, Dutch, Swedish

 
 Message 124 of 211
21 August 2014 at 3:04am | IP Logged 
Serpent wrote:
He was a big, man with quase não any neck, although he did have very large bigodes.
-I removed the article before moustache because in Finnish it can only be plural here and the Portuguese translation uses plural too.

And btw, the plural ending makes it much clearer. A clever learner should figure out that it can only be a "double" body part, and while they might not remember the word for eyebrows they can recall that it included a familiar element like "eye", "up" or "hair". "Although" implies that the body part is fairly close to the neck, and if you've seen the movie you'll know he has a moustache.

Edited by Serpent on 21 August 2014 at 3:07am

1 person has voted this message useful



s_allard
Triglot
Senior Member
Canada
Joined 5432 days ago

2704 posts - 5425 votes 
Speaks: French*, English, Spanish
Studies: Polish

 
 Message 125 of 211
21 August 2014 at 4:10am | IP Logged 
Serpent wrote:
Also it's ridiculous to assume that someone with a 75% comprehension won't understand
simple
words like number or boy. Someone who understood 75% of the whole book will understand more than 75% of
this
very paragraph (which is a bit untypical, since as emk mentioned, the beginning often features descriptions of
the
nature/settings etc).

Here's my version of what I'd assume if someone said they understood 75%. I'll replace the missing words with
Portuguese (most of the members will have at least some associations) and Finnish (not familiar for most of the
forum members, although two words won't be opaque).

...

I rarely comment these posts but I have to say that a lot of work went into this one. Kudos I'm just not sure how
this relates to the thread.

Edited by s_allard on 21 August 2014 at 8:28am

1 person has voted this message useful



s_allard
Triglot
Senior Member
Canada
Joined 5432 days ago

2704 posts - 5425 votes 
Speaks: French*, English, Spanish
Studies: Polish

 
 Message 126 of 211
21 August 2014 at 9:24am | IP Logged 
emk wrote:
...
So what does partial comprehension feel like in practice? Here's language.com/forum/forum_posts.asp?TID=31338&PN=1&TPN=1">an example from 2012 in my log:

emk wrote:
I've now read 52% of Le Tour du Monde. It's actually getting harder—I'm running
into sentences that don't make sense, even after I look up all the words. Perhaps 60%
of the text is clear, another 30% makes sense if I think about it or look up some
words, and 10% is very difficult to understand. Whenever Verne starts talking about
steamboats or tropical plants, I start skimming.

As usual, I'm using a three-part breakdown between "(semi-)automatic/decipherable/opaque." But here I counted
sentences, not words, because my comprehension was still low enough that I'd lose whole sentences of the text.
Today, I usually count words in French, because I take the grammar as a given, and only a tiny handful of words
completely escape my comprehension—normally 0.1% to 2% by my count, depending on the genre and subject of
the text.

Now, when I use a percentage, it's often based on a quick mental calculation: "Hmm, I understand 3 out every
four sentences pretty easily, so 75%" or "I can follow a bit less than half the dialog in this TV show, so I can
follow 40%" or "I actually had to look up 10 words out of the last 40 pages, and this book runs roughly 350
words/page, so that's 10 word in 14,000, or less than 1 word in 1,000, so that's less than 0.1% 'opaque'." I
convert everything to percentages, because surprisingly many people have trouble telling whether 3/5ths is
bigger than 3/4ths, but everybody knows that 60% is less than 75%. It's just a notational convenience: I convert
everything to a single decimal scale for the reader, because not everybody likes doing fractions in their head.
There's a reason everybody but the US uses the metric system.

Of course, I don't necessarily assume that percentages are terribly precise: "75%" means "about 3/4ths, but it's
probably rounded or an imprecise measurement" when I read it, but "75.2%" means "I actually measured this and
got the equivalent of 752/1000." Similarly, "60%" means "about 6 out of 10," and interpreting it "600 out of
1,000" is actually inappropriate. "600 out of 1000" is written "60.0%". In fact, in the United States, they actually
teach this stuff in some math and science programs. I've had teachers who would actually mark students down
for writing "61.27%" when they should have written "about 60%." We were actually instructed in how to count
significant digits, and keep track of them through a longer calculation, and we were expected to round our
answers accordingly.

So if I write "I understood about 60%", you can safely interpret that "as more than half, but probably less than
2/3rds, using whatever counting methodology I just described." And that can be a useful thing to communicate,
because there really was a stage of my learning where I could understand roughly that fraction of the sentences
in Le Tour du Monde without making an effort—and I enjoyed the book greatly. One of the reasons I've kept
such a detailed log is because I wanted to communicate to other first-time students that it's possible to progress
from understanding roughly 3/5ths of the sentences to understanding all but the occasional obscure word, and
to do so within 10 to 40 books worth of reading (at least for an English speaker learning a Romance language).
It's much harder to explain that using a scale like "none/some/all." If you look back at my quote about Le Tour
du Monde
, I think that communicates a much more vivid and useful idea than merely saying "some."


I was intrigued by this sentence counting method and the observation emk could understand only 2/3rds of
the sentences of Jules Verne's Le tour du monde and still enjoy the book greatly. Basically, emk did not
understand one out of every three sentences but could still follow the story and appreciate the writing. So, I
decided to have a look at the Jules Verne text myself. I've read three of Verne's novels but not this one. I'm quite
used to his mid-19th century writing style. For those who can read French, here is the beginning.

En l’année 1872, la maison portant le numéro 7 de Saville-row, Burlington Gardens – maison dans laquelle
Sheridan mourut en 1814 –, était habitée par Phileas Fogg, esq., l’un des membres les plus singuliers et les plus
remarqués du Reform-Club de Londres, bien qu’il semblât prendre à tâche de ne rien faire qui pût attirer
l’attention.
À l’un des plus grands orateurs qui honorent l’Angleterre, succédait donc ce Phileas Fogg, personnage
énigmatique, dont on ne savait rien, sinon que c’était un fort galant homme et l’un des plus beaux gentlemen de
la haute société anglaise.
On disait qu’il ressemblait à Byron – par la tête, car il était irréprochable quant aux pieds –, mais un Byron à
moustaches et à favoris, un Byron impassible, qui aurait vécu mille ans sans vieillir.
Anglais, à coup sûr, Phileas Fogg n’était peut-être pas Londonner. On ne l’avait jamais vu ni à la Bourse, ni à la
Banque, ni dans aucun des comptoirs de la Cité. Ni les bassins ni les docks de Londres n’avaient jamais reçu un
navire ayant pour armateur Phileas Fogg. Ce gentleman ne   figurait     da ns       aucun comité d’administration.
Son nom
n’avait jamais retenti dans un collège d’avocats, ni au Temple, ni à Lincoln’s-inn, ni à Gray’s-inn. Jamais il ne
plaida ni à la Cour du chancelier, ni au Banc de la Reine, ni à l’Échiquier ni en Cour ecclésiastique. Il n’était ni
industriel, ni négociant, ni marchand, ni agriculteur. Il ne faisait partie ni de l’Institution royale de la Grande-
Bretagne, ni de l’Institution de Londres, ni de l’Institution des Artisans, ni de l’Institution Russell, ni de
l’Institution littéraire de l’Ouest, ni de l’Institution du Droit, ni de cette Institution des Arts et des Sciences réunis,
qui est placée sous le patronage direct de Sa Gracieuse Majesté. Il n’appartenait enfin à aucune des nombreuses
sociétés qui pullulent dans la capitale de l’Angleterre, depuis la Société de l’Armonica jusqu’à la Société
entomologique, fondée principalement dans le but de détruire les insectes nuisibles."

This is kind of French is rather heavy going by today's standards, although Verne could be compared to any of
the prolific writers of popular novels today.

There are two observations. First, how can someone totally understand, that is decode the grammar and
vocabulary, of two sentences and not a third one? Is the grammar of the third sentence more difficult than that of
the others, or, more likely, there are pieces of vocabulary that make the third sentence rough going? If someone
can understand the grammar of two sentences out of three, there is probably nothing that different in the
grammar of the third sentence. In other words, understanding 2/3rds of the sentences requires a very good
mastery of the language.

The vocabulary is the problem. This is not surprising because this is always the problem when reading older
literature, especially this kind of adventure novel with all kinds of historical and geographical references. There
are many things that are just not relevant or meaningful today. This is particularly the case of the last paragraph
of the excerpt above.

This leads to the second observation: much of Verne's writing is highly redundant with lots of fluff. This is not
surprising considering the massive output of this author of over 63 novels among other works. When reading
these works, one can easily skim over many sentences and not miss most of the story. This is
actually what modern readers do with this kind of writing. In the excerpt above, I would not be surprised if half
of the sentences could be omitted without changing the story much.

I'm not surprised that one could claim to understand two thirds of the sentences and still enjoy the story. On the
other hand, not understanding two thirds of the words spread throughout the work is another thing.

Edit: following emk's observation, I corrected the first line to "...understand only 2/3rd of the sentences."

Edited by s_allard on 21 August 2014 at 2:07pm

1 person has voted this message useful





Iversen
Super Polyglot
Moderator
Denmark
berejst.dk
Joined 6705 days ago

9078 posts - 16473 votes 
Speaks: Danish*, French, English, German, Italian, Spanish, Portuguese, Dutch, Swedish, Esperanto, Romanian, Catalan
Studies: Afrikaans, Greek, Norwegian, Russian, Serbian, Icelandic, Latin, Irish, Lowland Scots, Indonesian, Polish, Croatian
Personal Language Map

 
 Message 127 of 211
21 August 2014 at 10:07am | IP Logged 
As s_allard writes, vocabulary is the problem. OK, if t had been Chinese the writing system could have been even more problematic, and grammar can also be a problem, but by and large my own understanding of texts in weak languages is most closely tied to the amount of vocabulary I understand. Which is why I spend so much time on formal vocabulary training.

But comprehension is not a yes-no thing. Let me illustrate that with a look at a passage from the quote from Verne above:

"Son nom n’avait jamais retenti dans un collège d’avocats, ni au Temple, ni à Lincoln’s-inn, ni à Gray’s-inn. Jamais il ne plaida ni à la Cour du chancelier, ni au Banc de la Reine, ni à l’Échiquier ni en Cour ecclésiastique"

This is a translation of an English text. It contains a lot of halfway translated references to specific British institutions, which also happen to be geographically located in London. In this case I have seen the English names before (although without the typically French trait-d'union) so I know that there is a system with 'inns' and something called "Temple", but I have to turn the French translations an extra time in my head to connect them to the relevant British item - and even then I'm not sure what for instance 'Queen's bank' might refer to. (Google refers to a real Bank of that name, but in the context this would be unlikely). This may be a question of linguistic knowledge, but it is even more a question of historical/juridical/political and even geographical knowledge. And your understanding of such things can be more or less fuzzy.

In the case above I would claim that I understand the sentences on a purely linguistic level as long as I understand that the places named are British juridical institutions.

But let's assume that I can't decode the word "plaida". It could be that I didn't know that verbal form, but you wouldn't get far in the text before you had seen a fair number of passé simples. But you might be unaware of the precise meaning of the root word and still deduce from the context that it is is something a person can do in a court. But in principle it could be the judge who made a "plaidoyer" - French judges have some quite unexpected roles compared to those in Britain. In this case you end up with a partial (and partly wrong) understanding of the word - but that it still more than no understanding at all. You know that it is something juridical.

Whether you formulate that as a percentage may be a temperamental question more than a mathematical one, but it also reflects your estimate of the importance of precision for the plot in the book. Is the added knowledge that only lawyers/barristers (or rather their French counterparts) can "plaider" very important for your understanding then you would let this influence the percentage to high degree, but if it weren't really crucial you might indeed say that you understood more than 77,231 % of the meaning of sentences even if you didn't know the place/institution names and the exact meaning of "plaider". But a nitpick might just as rightfully say that that you hadn't understood more than 7,231 % of the meaning.

And the cumulative assessment of your understanding of the whole book is really just some foggy estimate of the mean value af all those micro-assessments, each of which are dependent on your own loose evaluation of the importance of an acribically correct decoding of the text. That is, on a factor which no scientific method can take into account with any degree of accuracy.


Edited by Iversen on 21 August 2014 at 10:22am

1 person has voted this message useful



tarvos
Super Polyglot
Winner TAC 2012
Senior Member
China
likeapolyglot.wordpr
Joined 4709 days ago

5310 posts - 9399 votes 
Speaks: Dutch*, English, Swedish, French, Russian, German, Italian, Norwegian, Mandarin, Romanian, Afrikaans
Studies: Greek, Modern Hebrew, Spanish, Portuguese, Czech, Korean, Esperanto, Finnish

 
 Message 128 of 211
21 August 2014 at 10:19am | IP Logged 
Serpent wrote:
Also it's ridiculous to assume that someone with a 75% comprehension
won't understand simple words like number or boy. Someone who understood 75% of the
whole book will understand more than 75% of this very paragraph (which is a bit
untypical, since as emk mentioned, the beginning often features descriptions of the
nature/settings etc).

Here's my version of what I'd assume if someone said they understood 75%. I'll replace
the missing words with Portuguese (most of the members will have at least some
associations) and Finnish (not familiar for most of the forum members, although two
words won't be opaque).

Mr. and Mrs. Dursley, of number four, Alfeneiros Drive, were orgulhosos
to say that they were perfectly normal, thank you very much. They were the last people
you'd expect to se metessem in anything strange or mysterious, because they
just didn't hold with such bobagem.

Mr. Dursley was the director of a firm called Grunnings, which made perfurações.
He was a big, man with quase não any neck, although he did have very large
bigodes. Mrs. Dursley was thin and blonde and had nearly twice the usual amount
of neck, which came in very useful as she spent so much of her time espichando
over garden cerca, espiando on the vizinhos. The Dursleys had a
small son called Dudley and in their opinion there was no finer boy anywhere.


Mr. and Mrs. Dursley, of number four, Likusteri Drive, were ylpeitä to
say that they were perfectly normal, thank you very much. They were the last people
you'd expect to sekaantua in anything strange or mysterious, because they just
didn't hold with such hölynpöly.

Mr. Dursley was the johtaja of a firm called Grunnings, which made poria.
He was a big, man with tuskin any neck, although he did have very large
viikset. Mrs. Dursley was thin and blonde and had nearly twice the usual amount
of neck, which came in very useful as she spent so much of her time kurkotellen
over garden aitojen, vakoillessaan on the naapurit. The Dursleys
had a small son called Dudley and in their opinion there was no finer boy anywhere.


More details:
-if someone hasn't had formal schooling in the language, I'd not necessarily expect
them to know the word "neck".
-"director" was replaced only in the Finnish variant, reflecting the fact that many get
this particular word for this concept for free. the same applies to boss, but not other
synonyms
-similarly, "strange and mysterious" shouldn't be too hard because you only need to
know one of the two - and in many languages, mystical/mysterious will actually be more
obvious
-at 75%, I'd expect one to understand at least one of proud, nonsense, moustache,
neighbours
. at 90% I'd expect one to understand all of them
-I removed the article before moustache because in Finnish it can only be plural here
and the Portuguese translation uses plural too. I didn't always take the missing words
from the translation though.
-I'd expect at least native speakers of European languages to associate Grunnings with
something hard/heavy/metallic, maybe tools, or maybe just something boring. The
exclusion of people who come from vastly different cultures is based on posts like
TID=38911&PN=1&TPN=1">this (phonosemantics and poetry go hand in hand). The same
would apply to a Westerner learning Mandarin or Arabic, of course.
-I'd also expect most of the following things to be confusing:
1. thank you very much (in the ironical sense)
2. twice the usual amount of neck
3. came in very useful (the last word should make it clear, but someone who's used to
formal learning and classes would feel the need to look up or underline the expression)
4. same goes for "hold with such nonsense", which I'd expect to be much clearer if it
occured after the first 100 pages or so
5. the finer shades of "no finer boy anywhere" :)
6. maybe some other peculiar wording like "perfectly normal"
-My list obviously doesn't apply to diehard HP fans who know the text well.
-If someone was familiar with JKR's style in their native language (but not a crazy
fan), I'd expect them to know when they are missing out on subtleties.
-Ignoring HP's popularity, I'd expect an educated adult to feel the author's
disapproval of being so normal that it's boring, and to be aware from the beginning
that adventures are coming to the Dursley family. At this point they could reasonably
assume that Dudley would be more involved in the adventures though. At 75%, I'd also
expect one to take the Dursleys as a moderate force in the story, definitely not the
main villains.

Since it's not the most difficult part of the text, understanding it all can count for
maybe 93-95% comprehension of the whole book. And understanding only half of what I
mentioned would be around 80-85%. It's okay to miss an occasional high frequency word*
if you get most of the subtleties.
*I only learned the word "destination" in English when I was 16 and came across the
Finnish word for this.

Obviously that's just my opinion, and in reality my interpretation would rely on what I
know about the learner (and the text, if I've read it).


All the other Finnish words are opaque to me, but actually, Likusteri would not be in
this case. Likusteri is a loanword I think, and Privet in Dutch is actually replaced by
Liguster. I am pretty sure that it must be a loanword from Swedish of some sort.
Sometimes weird shit crops up even in Finnish.


1 person has voted this message useful



This discussion contains 211 messages over 27 pages: << Prev 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27  Next >>


Post ReplyPost New Topic Printable version Printable version

You cannot post new topics in this forum - You cannot reply to topics in this forum - You cannot delete your posts in this forum
You cannot edit your posts in this forum - You cannot create polls in this forum - You cannot vote in polls in this forum


This page was generated in 0.4844 seconds.


DHTML Menu By Milonic JavaScript
Copyright 2024 FX Micheloud - All rights reserved
No part of this website may be copied by any means without my written authorization.