In the first question, items c) and e) were tested systematically. There were no questions, comments or spontaneous remarks on the introductory text either by the test persons or by the cognitive interviewers, so that it can be assumed that the introduction is understood by the test persons.
Among the spontaneous comments of the test persons there are four comments that refer to the scale. Two test persons express problems with the values "without problems" and "with certain difficulties":
Test person 02 also has problems with this and notes that one category is missing: "I think a 5th would be missing here. I find "I can't" and "I can do without any problems", which is like either "Yes" or "No". And this (...) here "with great difficulty" or "with certain difficulties", I find that it is as if one could not do it properly. Now I would have a problem with saying "with certain difficulties", for example. Problem-free is crucial for a "Yes, I can". I don't know, I kind of miss..." (TP 02)
A fourth test person states that the word "difficulties" is not appropriate and that effort would seem more appropriate for her at this point: "The word difficulties is not so clever here. With some effort would be better." (TP 07)
The cognitive interview asked what the difference between the "with great difficulty" and "with certain difficulties" options was for the test subjects. Two test subjects have problems distinguishing (TP 02, 11):
With a third test person the distinction is also unclear: "Great difficulties would be for me if one must know one thing in order to understand the other. But it would also be very difficult if I had to read two or three times. Small or certain difficulties if I have to read it twice or parts of it several times." (TP 05)
Of the 17 remaining test persons, ten test persons (TP 01, 03, 04, 06, 08, 13, 14, 15, 16, 20) associate "great difficulties" with the fact that the task or text is essentially not understood, i.e. that the (reading) activity cannot be carried out, for example:
These test persons have problems to distinguish the category "I can, but with great difficulty" from the category "I cannot".
On the other hand, it is easy to distinguish between great and certain difficulties. Certain difficulties means that the 17 test persons who have no problems with differentiation have fewer difficulties: "Certain difficulties are less than great difficulties." (TP 12).
Four test persons describe certain difficulties by saying that only parts of the (reading) activity cause difficulties (TP 04, 09, 16, 19), for example:
For four test persons as well, certain difficulties mean that they have to reread text passages repeatedly (TP 01, 13, 18, 20):
For another four test persons, the difference between great and certain difficulties is whether or not they have to ask someone or look something up (TP 07, 09, 10, 17):
Furthermore, there are certain difficulties if the solution of the task takes more time (TP 06, 09).
Overall, it can be observed that the test persons associate with the answer category "I can, but with great difficulty" that they cannot do it - this indicates that this category is used incorrectly.
| Item Text | Actively tested |
|---|---|
|
|
No |
|
|
No |
c. Read and understand an instruction manual or manual to operate a new appliance, such as a television or washing machine.Item Text:Recommendations:
Here it should be specified that longer texts are meant and not short How-Tos. The text of the question could be modified and simplified as follows: The term "devices" should be omitted and instead concrete examples should be given:
"Read and understand detailed operating instructions or a manual in order to operate a new TV set or washing machine, for example." [„Eine ausführliche Bedienungsanleitung oder ein Handbuch lesen und verstehen, um bspw. einen neuen Fernseher oder eine neue Waschmaschine bedienen zu können.“] Alternatively, only the word "detailed" can be added to the original item:: "Read and understand detailed operating instructions or a manual to be able to operate a new appliance, e.g. a television or washing machine.” [„Eine ausführliche Bedienungsanleitung oder ein Handbuch lesen und verstehen, um ein neues Gerät bedienen zu können, z. B. einen Fernseher oder eine Waschmaschine.“] Findings:For this item the full width of the scale was used: Ten test persons, i.e. half of them, indicate "I can without problems", eight test persons say "I can, but with certain difficulties", one test person (TP 12) indicates "I can, but with great difficulties" and another one (TP 03) indicates "I cannot".
Overall, the test persons have no problems with the description "to operate a device". Out of 20 test persons, 16 think of technical devices, of which 13 test persons mention concrete devices such as TV, DVD recorder, laptop or telephone. Three persons name (additionally) furniture like cupboard (TP 08), desk (TP 13) or laminate (TP 14). With two test persons it remains unclear what kind of devices they think of (TP 05, 19). Nine test persons think exclusively of longer texts (TP 01, 03, 06, 09, 12, 16, 17, 18, 19) and five test persons think exclusively of shorter texts (TP 05, 10, 11, 13, 14). Four test persons think of both shorter and longer texts (TP 02, 04, 07, 08) and with another two test persons it remains unclear what they think of (TP 15, 20). The test persons who stated that they are able to read and understand an instruction manual or a manual "with certain difficulties" justify this in three different ways. The first reason is that the operation or the construction of new things is complicated:
Among the test persons who answered with no problems, there are three who report difficulties in the explanation of their answer:
Question Topic:
Job and career/ Job situation & professional activity
Construct:Self-assessment of reading skills
|
Yes |
|
|
No |
e. Cross-reading different internet pages to find different information on a specific topic, e.g. about a disease.Item Text:Recommendations:
No changes recommended.
Findings:19 out of 20 respondents answered this item, test person 11 does not use the Internet and therefore does not answer item e). Twelve of the 19 respondents said that they could read different websites without problems, the remaining seven respondents could do so with certain difficulty.
All 19 test persons, who answered this item, think of different internet pages, i.e. of considering several text sources. Of the 20 test persons, 16 understand the term "cross-reading" correctly. Twelve test persons understand the term "skimming over a text", two persons say that they do not read everything (TP 17, 20) and one person each states that they do not read in detail (TP 06) or search for keywords in the text (TP 07). The following are two examples of how the term "cross-reading" is described:
The test persons 07 and 14 note that diseases are a bad example in their opinion: "The topic of illness has irritated me here. This is by no means something I can only read across." (TP 07) and two other test persons express this at least indirectly:
Question Topic:
Job and career/ Job situation & professional activity
Construct:Self-assessment of reading skills
|
Yes |
