General Information: *Note: The items were tested in German. These are English translations of the original German wordings. The translations do not correspond exactly to the formulations in the English PIAAC source questionnaire.*
Introduction: The following is about assessing your own reading skills. Reading is the ability to understand written text in the form of sentences and paragraphs.
[Im Folgenden geht es darum, dass Sie Ihre eigene Lesekompetenz einschätzen. Lesen ist die Fähigkeit, geschriebenen Text in Form von Sätzen und Abschnitten zu verstehen.]
Question Text: I will now read to you various reading activities. For each of these activities, please tell me how well you can perform them. Think about your experiences from your job and everyday life.
Please give me your answers using this list.
[Ich lese Ihnen jetzt verschiedene Lesetätigkeiten vor. Bitte sagen Sie mir für jede dieser Tätigkeiten, wie gut Sie diese durchführen können. Denken Sie dabei an Ihre Erfahrungen aus Beruf und Alltag.
Bitte geben Sie mir Ihre Antworten anhand dieser Liste.]
Instruction: (CI: Show list 1!) [(TL: Liste 1 vorlegen!)]
Answer Categories
I cannot [Kann ich nicht]

I can, but with great difficulty [Kann ich, aber mit großen Schwierigkeiten]

I can, but with certain difficulties [Kann ich, aber mit gewissen Schwierigkeiten]

I can without problems [Kann ich problemlos]
- Findings for Answer Categories
- Recommendations for Answer Categories

Findings/Recommendations for Multi-Item Scale:
- Findings for Multi-Item Scale
- Recommendations for Multi-Item Scale
Findings for Multi-Item Scale: In the first question, items c) and e) were tested systematically. There were no questions, comments or spontaneous remarks on the introductory text either by the test persons or by the cognitive interviewers, so that it can be assumed that the introduction is understood by the test persons.

Among the spontaneous comments of the test persons there are four comments that refer to the scale. Two test persons express problems with the values "without problems" and "with certain difficulties":

"What do you mean by "certain difficulties"? That it takes a little longer?" (TP 09)

"(...) if this includes reading not only once, but two or three times, then I can do it without problems.” (TP 16)

Test person 02 also has problems with this and notes that one category is missing: "I think a 5th would be missing here. I find "I can't" and "I can do without any problems", which is like either "Yes" or "No". And this (...) here "with great difficulty" or "with certain difficulties", I find that it is as if one could not do it properly. Now I would have a problem with saying "with certain difficulties", for example. Problem-free is crucial for a "Yes, I can". I don't know, I kind of miss..." (TP 02)

A fourth test person states that the word "difficulties" is not appropriate and that effort would seem more appropriate for her at this point: "The word difficulties is not so clever here. With some effort would be better." (TP 07)

The cognitive interview asked what the difference between the "with great difficulty" and "with certain difficulties" options was for the test subjects. Two test subjects have problems distinguishing (TP 02, 11):

"[...] This with difficulties, that is for me then again such a term as if I could not do anything wrong.” (TP 02)

"There's really no difference." (TP 11)

With a third test person the distinction is also unclear: "Great difficulties would be for me if one must know one thing in order to understand the other. But it would also be very difficult if I had to read two or three times. Small or certain difficulties if I have to read it twice or parts of it several times." (TP 05)

Of the 17 remaining test persons, ten test persons (TP 01, 03, 04, 06, 08, 13, 14, 15, 16, 20) associate "great difficulties" with the fact that the task or text is essentially not understood, i.e. that the (reading) activity cannot be carried out, for example:

"Great difficulties are that I just cannot cope at all [...]" (TP 06)

"I have great difficulties when I cannot solve things, when they are unsolvable.“ (TP 08)

"When I have great difficulty, I understand nothing and do not see through." (TP 14)

These test persons have problems to distinguish the category "I can, but with great difficulty" from the category "I cannot".

On the other hand, it is easy to distinguish between great and certain difficulties. Certain difficulties means that the 17 test persons who have no problems with differentiation have fewer difficulties: "Certain difficulties are less than great difficulties." (TP 12).

Four test persons describe certain difficulties by saying that only parts of the (reading) activity cause difficulties (TP 04, 09, 16, 19), for example:

"Certain difficulties are partial." (TP 04)

"Certain difficulties means that there are small parts where I have to take a closer look.” (TP 19)

For four test persons as well, certain difficulties mean that they have to reread text passages repeatedly (TP 01, 13, 18, 20):

"Certain difficulties are if I have to read it again." (TP 13)

"If I've read it two or three times, it'll come to me." (TP 18)

For another four test persons, the difference between great and certain difficulties is whether or not they have to ask someone or look something up (TP 07, 09, 10, 17):

"Big would mean if I would need help somewhere else again and have to ask someone else." (TP 07)

"So with certain difficulties I have to ponder briefly and with big ones I ask a friend or look on the Internet.” (TP 10)

Furthermore, there are certain difficulties if the solution of the task takes more time (TP 06, 09).

Overall, it can be observed that the test persons associate with the answer category "I can, but with great difficulty" that they cannot do it - this indicates that this category is used incorrectly.
Recommendations for Multi-Item Scale: Obviously there are problems with the classification of the skills. On the one hand, there are discrepancies between the self-assessment of the test persons and the free assessment by the cognitive interviewers and, on the other hand, the test persons have difficulties in understanding the naming of scales. Therefore, the scale should be adjusted, especially since the question itself is about "how well can you do". Furthermore, the scale is not balanced or symmetrical. Our recommendation for the scale is therefore:
not good at all - less good - moderate - quite good - very good
[gar nicht gut – weniger gut – mäßig – ziemlich gut – sehr gut]
alternatively:
I can't - I rather can't - I can to some extent - I rather can - I certainly can
[kann ich gar nicht – kann ich eher nicht – kann ich einigermaßen – kann ich eher - kann ich sicher]
or alternatively:
"How easy or difficult is it for you...?" with the answer categories: very difficult - rather difficult - moderate - rather easy - very easy
[„Wie leicht oder schwer fällt Ihnen…?“ mit den Antwortkategorien: sehr schwer – eher schwer – mäßig – eher leicht – sehr leicht]
A legitimate answer from respondents may also be that they do not perform the respective activity in their everyday life or do not need to do so. Therefore, we recommend including an additional answer category, which, however, is not on the list and is not read out, but is only recorded by the interviewer if respondents express this on their own initiative:
„I never do that“
[„das mache ich nie“]

Cognitive Techniques: General Probing, Specific Probing, Comprehension Probing.

Items

Item Text	Item Tested
a. Read a short text, e.g. an e-mail or a short message, cross-read and record what is important [Einen kurzen Text, z.B. eine E-Mail oder eine kurze Mitteilung, querlesen und das Wesentliche erfassen.]	No
b. Read and correctly follow written instructions, such as a recipe or work instructions. [Schriftliche Anweisungen, wie z.B. ein Rezept oder eine Arbeitsanweisung, lesen und korrekt befolgen.]	No
c. Read and understand an instruction manual or manual to operate a new appliance, such as a television or washing machine. [Eine Bedienungsanleitung oder ein Handbuch lesen und verstehen, um ein neues Gerät bedienen zu können, z. B. einen Fernseher oder eine Waschmaschine.] Tested Item and Recommendations Findings Topic and Construct Item Text: c. Read and understand an instruction manual or manual to operate a new appliance, such as a television or washing machine. [Eine Bedienungsanleitung oder ein Handbuch lesen und verstehen, um ein neues Gerät bedienen zu können, z. B. einen Fernseher oder eine Waschmaschine.] Recommendations: Here it should be specified that longer texts are meant and not short How-Tos. The text of the question could be modified and simplified as follows: The term "devices" should be omitted and instead concrete examples should be given: "Read and understand detailed operating instructions or a manual in order to operate a new TV set or washing machine, for example." [„Eine ausführliche Bedienungsanleitung oder ein Handbuch lesen und verstehen, um bspw. einen neuen Fernseher oder eine neue Waschmaschine bedienen zu können.“] Alternatively, only the word "detailed" can be added to the original item:: "Read and understand detailed operating instructions or a manual to be able to operate a new appliance, e.g. a television or washing machine.” [„Eine ausführliche Bedienungsanleitung oder ein Handbuch lesen und verstehen, um ein neues Gerät bedienen zu können, z. B. einen Fernseher oder eine Waschmaschine.“] Findings: For this item the full width of the scale was used: Ten test persons, i.e. half of them, indicate "I can without problems", eight test persons say "I can, but with certain difficulties", one test person (TP 12) indicates "I can, but with great difficulties" and another one (TP 03) indicates "I cannot". Overall, the test persons have no problems with the description "to operate a device". Out of 20 test persons, 16 think of technical devices, of which 13 test persons mention concrete devices such as TV, DVD recorder, laptop or telephone. Three persons name (additionally) furniture like cupboard (TP 08), desk (TP 13) or laminate (TP 14). With two test persons it remains unclear what kind of devices they think of (TP 05, 19). Nine test persons think exclusively of longer texts (TP 01, 03, 06, 09, 12, 16, 17, 18, 19) and five test persons think exclusively of shorter texts (TP 05, 10, 11, 13, 14). Four test persons think of both shorter and longer texts (TP 02, 04, 07, 08) and with another two test persons it remains unclear what they think of (TP 15, 20). The test persons who stated that they are able to read and understand an instruction manual or a manual "with certain difficulties" justify this in three different ways. The first reason is that the operation or the construction of new things is complicated: "But with other things, this is a very new area. And it's always a bit complicated for me." (TP 02) "Assemble a desk. Such a DIN A4 sheet with a thousand screws and things." (TP 13) "Sometimes the technical descriptions are too complicated and I am not that technically gifted and therefore sometimes there are difficulties when it is very extensive and complicated to understand the technology." (TP 18) A second reason is that operating instructions must be read several times in order to understand them: "If it was a technical manual, I would first read it and then try, but read it again to see if I did it right." (TP 07) "If I have such an instruction manual with pictures and text, then I have to find my way in. Then I read it through two or three times." (TP 08) "We have a big TV, it's new. Until I find out how it works, I read the instructions for use two or three times. (...) The instructions are not always 1A." (TP 17) Test person 17, as well as two other test persons, justified their answer with poorly written operating instructions: "This is a problem because these manuals are so bad. I could say I can do it without any problems if the manual is okay. If it's not okay, then I'm stuck. I recently bought laminate. There was a note in it that said, "This is ... It's so stupidly written.“ (TP 14) "Then I thought of the operating instructions that were translated, for example, from Chinese into Swedish and then at some point into German.” (TP 19) Of the ten test persons who stated that they could read and understand operating instructions without problems, test person 06 and test person 20 also spontaneously commented on the problem of bad instructions: "That depends on the instructions. Normally I can do that without any problems, but there are more complex instructions." (TP 06) "That depends on the description. Normally I don't have any problems, but it really depends on the manual." (TP 20) Five of the respondents with the answer "I can, without problems" justify this with intuition (TP 01), experience (TP 04), the short working steps (TP 05), that everything is described (TP 11) or that they understand it easily (TP 15). Among the test persons who answered with no problems, there are three who report difficulties in the explanation of their answer: "Most of the time these are very short steps, you can try them right away and then you're ready to go. With work instructions or packing lists it is already more difficult then. You first have to understand them so that you can implement them.” (TP 05) "I have now thought about this when I once wanted to reprogram my phone. It was so complex that it did not work the first time. I managed to do it then, therefore "without any problems", but it depends." (TP 06) "If it is just about a television, then it is already in such a way that I read it again and again, so that I not only read it once, but also a second or third time, then perhaps again in sections and then try it out on the television. I am already in a position to make it work afterwards." (TP 16) Conversely, test person 03 indicates certain difficulties, even if she understands simple operating instructions: "(...) And a manual is always so much. How to put a new kettle into operation is obvious: put water in, plug in and then turn it on. It is all the same. You don't even need to read it anymore, but for other things it is a very new area. And for me it's always a bit complicated.“ (TP 04) Question Topic: Job and career/ Job situation & professional activity Construct: Self-assessment of reading skills	Yes
d. Read and understand official documents, such as an employment contract, a rental agreement or an insurance policy. [Offizielle Dokumente lesen und verstehen, z.B. einen Arbeitsvertrag, einen Mietvertrag oder einen Versicherungsschein.]	No
e. Cross-reading different internet pages to find different information on a specific topic, e.g. about a disease. [Verschiedene Internetseiten querlesen, um unterschiedliche Informationen zu einem bestimmten Thema herauszufinden, z.B. über eine Krankheit.] Tested Item and Recommendations Findings Topic and Construct Item Text: e. Cross-reading different internet pages to find different information on a specific topic, e.g. about a disease. [Verschiedene Internetseiten querlesen, um unterschiedliche Informationen zu einem bestimmten Thema herauszufinden, z.B. über eine Krankheit.] Recommendations: No changes recommended. Findings: 19 out of 20 respondents answered this item, test person 11 does not use the Internet and therefore does not answer item e). Twelve of the 19 respondents said that they could read different websites without problems, the remaining seven respondents could do so with certain difficulty. All 19 test persons, who answered this item, think of different internet pages, i.e. of considering several text sources. Of the 20 test persons, 16 understand the term "cross-reading" correctly. Twelve test persons understand the term "skimming over a text", two persons say that they do not read everything (TP 17, 20) and one person each states that they do not read in detail (TP 06) or search for keywords in the text (TP 07). The following are two examples of how the term "cross-reading" is described: "So don't read through it completely, but fly over it. Don't read it properly, look through it quickly and get the most important things out." (TP 10) "If I do not read every line, but skim the text like this.” (TP 13) Four test persons do not understand the term "cross reading" correctly: Two test persons do not know it (TP 03, 11) and two have a wrong understanding (TP 12: normal reading, TP 14: comparative reading of two texts). The test persons 07 and 14 note that diseases are a bad example in their opinion: "The topic of illness has irritated me here. This is by no means something I can only read across." (TP 07) and two other test persons express this at least indirectly: "If these are any medical sites, I have no idea." (TP 01) "With illness, it's the same thing. You read so much and in the end you know nothing.”(TP 12) Question Topic: Job and career/ Job situation & professional activity Construct: Self-assessment of reading skills	Yes