Back to GESIS Homepage

Item List

Frage Thema Item Text Antwort Format Konstrukt Eingesetzte Kogn Techniken Ergebnis
c. Read and understand an instruction manual or manual to operate a new appliance, such as a television or washing machine. [Eine Bedienungsanleitung oder ein Handbuch lesen und verstehen, um ein neues Gerät bedienen zu können, z. B. einen Fernseher oder eine Waschmaschine.] Nein For this item the full width of the scale was used: Ten test persons, i.e. half of them, indicate "I can without problems", eight test persons say "I can, but with certain difficulties", one test person (TP 12) indicates "I can, but with great difficulties" and another one (TP 03) indicates "I cannot".

Overall, the test persons have no problems with the description "to operate a device". Out of 20 test persons, 16 think of technical devices, of which 13 test persons mention concrete devices such as TV, DVD recorder, laptop or telephone. Three persons name (additionally) furniture like cupboard (TP 08), desk (TP 13) or laminate (TP 14). With two test persons it remains unclear what kind of devices they think of (TP 05, 19).

Nine test persons think exclusively of longer texts (TP 01, 03, 06, 09, 12, 16, 17, 18, 19) and five test persons think exclusively of shorter texts (TP 05, 10, 11, 13, 14). Four test persons think of both shorter and longer texts (TP 02, 04, 07, 08) and with another two test persons it remains unclear what they think of (TP 15, 20).

The test persons who stated that they are able to read and understand an instruction manual or a manual "with certain difficulties" justify this in three different ways. The first reason is that the operation or the construction of new things is complicated:
  • "But with other things, this is a very new area. And it's always a bit complicated for me." (TP 02)
  • "Assemble a desk. Such a DIN A4 sheet with a thousand screws and things." (TP 13)
  • "Sometimes the technical descriptions are too complicated and I am not that technically gifted and therefore sometimes there are difficulties when it is very extensive and complicated to understand the technology." (TP 18)
A second reason is that operating instructions must be read several times in order to understand them:
  • "If it was a technical manual, I would first read it and then try, but read it again to see if I did it right." (TP 07)
  • "If I have such an instruction manual with pictures and text, then I have to find my way in. Then I read it through two or three times." (TP 08)
  • "We have a big TV, it's new. Until I find out how it works, I read the instructions for use two or three times. (...) The instructions are not always 1A." (TP 17)
Test person 17, as well as two other test persons, justified their answer with poorly written operating instructions:
  • "This is a problem because these manuals are so bad. I could say I can do it without any problems if the manual is okay. If it's not okay, then I'm stuck. I recently bought laminate. There was a note in it that said, "This is ... It's so stupidly written.“ (TP 14)
  • "Then I thought of the operating instructions that were translated, for example, from Chinese into Swedish and then at some point into German.” (TP 19)
Of the ten test persons who stated that they could read and understand operating instructions without problems, test person 06 and test person 20 also spontaneously commented on the problem of bad instructions:
  • "That depends on the instructions. Normally I can do that without any problems, but there are more complex instructions." (TP 06)
  • "That depends on the description. Normally I don't have any problems, but it really depends on the manual." (TP 20)
Five of the respondents with the answer "I can, without problems" justify this with intuition (TP 01), experience (TP 04), the short working steps (TP 05), that everything is described (TP 11) or that they understand it easily (TP 15).

Among the test persons who answered with no problems, there are three who report difficulties in the explanation of their answer:
  • "Most of the time these are very short steps, you can try them right away and then you're ready to go. With work instructions or packing lists it is already more difficult then. You first have to understand them so that you can implement them.” (TP 05)
  • "I have now thought about this when I once wanted to reprogram my phone. It was so complex that it did not work the first time. I managed to do it then, therefore "without any problems", but it depends." (TP 06)
  • "If it is just about a television, then it is already in such a way that I read it again and again, so that I not only read it once, but also a second or third time, then perhaps again in sections and then try it out on the television. I am already in a position to make it work afterwards." (TP 16)
Conversely, test person 03 indicates certain difficulties, even if she understands simple operating instructions: "(...) And a manual is always so much. How to put a new kettle into operation is obvious: put water in, plug in and then turn it on. It is all the same. You don't even need to read it anymore, but for other things it is a very new area. And for me it's always a bit complicated.“ (TP 04)
d. Read and understand official documents, such as an employment contract, a rental agreement or an insurance policy. [Offizielle Dokumente lesen und verstehen, z.B. einen Arbeitsvertrag, einen Mietvertrag oder einen Versicherungsschein.] Nein
e. Cross-reading different internet pages to find different information on a specific topic, e.g. about a disease. [Verschiedene Internetseiten querlesen, um unterschiedliche Informationen zu einem bestimmten Thema herauszufinden, z.B. über eine Krankheit.] Nein 19 out of 20 respondents answered this item, test person 11 does not use the Internet and therefore does not answer item e). Twelve of the 19 respondents said that they could read different websites without problems, the remaining seven respondents could do so with certain difficulty.

All 19 test persons, who answered this item, think of different internet pages, i.e. of considering several text sources.

Of the 20 test persons, 16 understand the term "cross-reading" correctly. Twelve test persons understand the term "skimming over a text", two persons say that they do not read everything (TP 17, 20) and one person each states that they do not read in detail (TP 06) or search for keywords in the text (TP 07). The following are two examples of how the term "cross-reading" is described:
  • "So don't read through it completely, but fly over it. Don't read it properly, look through it quickly and get the most important things out." (TP 10)
  • "If I do not read every line, but skim the text like this.” (TP 13)
Four test persons do not understand the term "cross reading" correctly: Two test persons do not know it (TP 03, 11) and two have a wrong understanding (TP 12: normal reading, TP 14: comparative reading of two texts).

The test persons 07 and 14 note that diseases are a bad example in their opinion: "The topic of illness has irritated me here. This is by no means something I can only read across." (TP 07) and two other test persons express this at least indirectly:
  • "If these are any medical sites, I have no idea." (TP 01)
  • "With illness, it's the same thing. You read so much and in the end you know nothing.”(TP 12)
a. Calculate percentages like e.g. calculating the value added tax. [Prozente berechnen wie z.B. die Mehrwertsteuer ausrechnen.] Nein 14 test persons stated that they could calculate percentages without any problems, four test persons could do so with certain difficulties and two test persons stated that they could do so only with great difficulties.

Two test persons asked the cognitive interviewer whether the calculation should be done in the head or with the help of a pocket calculator, whereby test person 06 (answer: with great difficulty) assumes mental arithmetic when answering and test person 20 (answer: I can do it without any problems) decides to use a pocket calculator as an aid when classifying their everyday mathematical competence.

With the help of the questionnaire it becomes clear that 14 test persons (TP 01, 04, 06, 07, 08, 09, 10, 11, 13, 15, 16, 17, 18, 19) thought of mental arithmetic when assessing their abilities, two test persons (TP 03, 20) thought of using a calculator or cell phone and four thought of both (TP 02, 05, 12, 14).

When asked whether they would have answered differently if they had thought about using a calculator, five of the 14 test persons who thought about mental arithmetic said that they could do this without any problems.
  • "If I had to do it, I would already know how to do the math. But when I'm standing in the store, I have a hard time doing it." (TP 06, answer: with great difficulty)
  • "If it is odd, it gets complicated, straight in the head. For example, if it's 27%, I need technical help.” (TP 07, answer: with certain difficulties)
  • “At 50 %, 20 % or 10 % this is still possible, but as soon as 3 % of an odd number [should be calculated], it becomes more difficult. If I would calculate more often with a calculator and I would know how to do it in the calculator, it would be no problem. But I hardly ever do that.”(TP 10, answer: with great difficulties)
  • "If you can write it down and calculate it with a calculator, then it works, but in your head like this.” (TP 12, answer: with certain difficulties)
Consequently, the variance of the scale results from the different ways in which technical aids are considered or not considered. In other words, those who claim to have some or great difficulties with the calculation of percentages, e.g. VAT, do so because they assume that the data should be calculated in their head. Only test person 05, who thinks of both mental arithmetic and the use of a calculator, sticks to her assessment that this task causes her certain difficulties: "I would have to think about it first and if I were to make a few mistakes, I would be fine. I would have to reach into certain areas of my brain that I otherwise never use. Nothing will change with a calculator, I think."
b. Calculate simple areas such as the area of a wall to buy tiles. [Einfache Flächen berechnen wie z.B. die Fläche einer Wand, um Fliesen zu kaufen.] Nein In terms of simple area calculation, e.g. the area of a wall to buy tiles, 15 test persons state that they can do this without any problems, four test persons are of the opinion that they can do this task with certain difficulties and one test person with great difficulties.

All test persons have a more or less comprehensive idea of what is meant by the term "simple surface". Apart from test person 12, who thinks of calculating the area of a circle, all test persons think of a rectangle when using this term or name the example wall. Furthermore, seven persons (TP 01, 04, 08, 10, 13, 15, 16) state that the term "simple surface" is used to connect a square and two test persons (TP 01, 04) triangles.

13 of the 15 test persons who state that they can easily calculate a simple area think of calculating a wall, i.e. a rectangle:
  • "Length times width." (TP 02)
  • "I also laid laminate myself in my apartment. You take length times width, how many square meters you need.” (TP 09)
  • "I can definitely calculate the area of a wall. Length times width. A two-dimensional surface." (TP 16)
  • "2.5 meters by 4 meters, that's relatively easy." (TP 19)
Test persons 08 and 11, who also report no problems with the calculation, think of somewhat more complex area calculations:
  • "Multiply in your head, estimate and measure. That is no problem to calculate an area. Most of the time you have the formulas in your head and then you can easily calculate the area." (TP 08)
  • "I have to know how many square meters. First in meters and then in square meters, then I have to add it up. Of course I can't calculate the number of plates, you have to tell the man where I buy [the plates].” (TP 11)
Of the five test persons who would have certain (TP 05, 06, 12, 17) or great (TP 03) difficulties with this task, four think of more extensive calculations of areas:
  • "There I add up the 4 walls, so 3.5 x 4 m for wallpaper, for example." (TP 03)
  • "Length times width would be easy, but as soon as a part has to be moved e.g. away from the heating, then I don't know how to subtract something from what." (TP 05)
  • "Calculate a barrel or a circle." (TP 12)
  • "I have the problem, I need the formula. There are people who can get it out of their heads, but I always need a formula." (TP 17)
Test person 06, who also has certain difficulties, however, thinks of simple calculations.

Again, the variance of the scale is largely due to the presumed complexity of the computational operations to be performed and not exclusively to the assessment of the underlying computational competence.
c. Convert units such as milliliters to liters or miles to kilometers. [Einheiten umrechnen wie z.B. Milliliter in Liter oder Meilen in Kilometer.] Nein Twelve test persons state that they manage to convert units without any problems, five test persons find this task somewhat difficult and three test persons even find it very difficult.

None of the 20 test persons express any difficulties in understanding the term "mile". However, eight test persons (TP 03, 08, 09, 10, 13, 15, 16, 17) do not think of the conversion of miles into kilometers when answering, but of something else:
  • "I immediately think of cooking or baking with American recipes, where everything is given in "cups”.” (TP 05)
  • "First of all I think of measures of length, cm in dm and m etc." (TP 10)
  • "If I measure something in the measuring cup, e.g. half a liter, then that is 50 cl and 500 ml. Or meters in kilometers." (TP 12)
  • "Convert one pound of coffee. I prefer grams." (TP 13)
Another ten test persons (TP 01, 05, 06, 07, 11, 12, 14, 18, 19, 20) can understand the term "mile", but they do not know the conversion factor:
  • "All those simple units from milli to micro or no idea what, yes, but from miles to kilometers that's something else. I'd have to look up, what is a mile." (TP 01)
  • "Well, there are certain difficulties with miles and kilometers, because I would have to look it up first because I don't know. I'd have to know how long a mile is first." (TP 14)
  • "I don't know now how to express miles in kilometers." (TP 19)
This leaves two test persons (TP 02, 04) who know the conversion factor from miles to kilometers and take this into account in their answers. Test person 04 explicitly deals with the conversion of both mentioned examples: "Milliliter in liter is a decimal shift and miles in kilometer is 1.6 and this has to be converted.” (TP 04)

Except for test persons 02 and 04 (conversion of miles to kilometers) and 05 (conversion of "cups" to grams), the majority of respondents only consider the conversion of units within the metric system when answering.

When asked how easy or difficult it was for them to answer this statement about the conversion of units, only four test persons (TP 06, 07, 08, 14) classified this as "rather difficult", mainly because they did not know the conversion factor from miles to kilometers.
d. Perform calculations that require several steps or arithmetic operations, such as calculating the cheapest cell phone contract for specific needs. [Berechnungen durchführen, die mehrere Schritte oder Rechenoperationen erfordern, wie z.B. den günstigsten Handyvertrag für bestimmte Bedürfnisse ausrechnen.] Nein Also in this everyday mathematical activity almost three quarters of the test persons (n=14) state that they can do this without any problems, two test persons can do this with certain difficulties and three with great difficulties. One test person does not make any statement because he says of himself that he never calculates cell phone contracts for certain needs.

There are two questions of particular interest in this item. One is to find out if the test persons think of several arithmetical operations when answering the question and the other is to find out if the mentioned example of calculating the most favourable cell phone contract for certain needs is useful for answering the question. When asked which arithmetical operations the test persons thought of, the answers vary from basic arithmetical operations such as addition and multiplication to equations or trisentence calculation up to the calculation of functions e.g. a straight line. With a total of 14 test persons (TP 01, 02, 03, 04, 05, 06, 07, 09, 12, 13, 14, 16, 18, 19) it becomes clear that they thought of several arithmetical operations when answering, regardless of whether they were referring to cell phone contracts or not:
  • "Calculate three-sentence calculation in the same way as for fuel consumption. I do that occasionally in my head and it's easy for me.” (TP 04)
  • "I think that's something you do more often. The contract has this and the other has that. I might not use one of them. Or I have free SMS. [...] For a short time I thought of a formula where different things are used one after the other, but you can't calculate that directly with the contract. You have different steps." (TP 06)
  • "I do also always make that, if I must decide between two offers, that does not have to be al-ways a cell phone contract, then one goes also there and computes only times. For example also electricity providers. That I compare the fixed costs and also the variable costs and look at me, what I need and/or use. That can also be a multi-level comparison calculation." (TP 09)
  • "I was thinking about some text tasks where you have to do several steps. First like this, then plus, minus and then a three-part sentence in the back." (TP 16)
The remaining six test persons (TP 08, 10, 11, 15, 17, 20) refer to the example "cell phone contract", but in their explanations they only refer to comparing the costs of different flat rates:
  • "It is not a multiple bill. You compare who offers the cheapest Allnet flat rate and has a good net and all that. Well, I'll manage that, I'll have to find the right cell phone contract.” (TP 10)
  • "To the calculation from the telephone provider for me. 25€ and 9,95€ are a big difference for me.” (TP 11)
  • "I was thinking about the cell phone contract. Per month, with Internet flat rate without Internet flat rate, i.e. the services per month." (TP 17)
  • "You read it through. So and so much Flatrate for so much euro etc. and compare that." (TP 20)
There is no assessment of their everyday mathematical competence with regard to their ability to use a multi-stage calculation method. These six test persons all decide on the answer category "I can without problems". This means that six of the 14 persons who claim to be able to do this without any problems, may be misplaced due to an incorrect understanding. Furthermore, it means that the example of calculating the cheapest cell phone contract no longer works today, since the costs for the widespread smart phones are based on flat rates and there is no longer any need for a multi-stage calculation method with basic arithmetic operations. In addition, there are also comparison computers on the Internet that can perform the actual computing power when the user enters the individual usage data.
e. Understanding and interpreting statistical information or data, such as a graph showing the development of housing prices in different districts. [Statistische Angaben oder Daten verstehen und interpretieren wie z.B. eine Abbildung mit der Entwicklung von Wohnungspreisen in verschiedenen Stadtteilen.] Nein This is the first item to use the full scale width. While eight test persons state that they can easily understand and interpret statistics and data using the example of housing price developments in different parts of the city, six test persons believe they can do so with some difficulty and one person with great difficulty. Three test persons state that they are not able to do so.

There are indications that two test persons (TP 03, 11) do not understand the statement in the intended sense:
  • "I know what it costs here, what it costs in Frankfurt, what it costs in Heidelberg. That is no problem. Frankfurt is expensive, Mannheim is cheaper, Heidelberg is in the middle." (TP 03, answer: I can without problems)
  • "I have a house and know how many square meters that is, but the other one no. In the newspaper, I can understand that. Often there are certain square meters for apartment rentals. I understand that, but how much it should cost and how expensive the real estate agent is, I have never dealt with that." (TP 11, answer: I cannot)
These two people think of costs that may be incurred for rental housing, but not of illustrations of price developments and their interpretation.

In contrast, the following test persons, for example, refer to concrete statistical information that may contain images. Here it becomes clear that persons have very different associations.
  • "We had this once in an Excel course with bar charts or something and I never knew what to get out of it, honestly." (TP 05)
  • "This is again so that it is not commonplace. When you look at something like this, you first have to come in and understand what the axes mean and that's what I find the certain difficulties." (TP 08)
  • "A coordinate system with an x and y axis and there you can see the development. I can also interpret it." (TP 09)
  • TP 12: "I haven't done that yet. Maybe I could, I can't tell you."
  • CI: "What if it was about a diagram in the newspaper?"
  • TP 12:"I probably could. But I was thinking about big plans and calculations etc."
  • "You'll have to get your head into it. Then there are circles. Then it says what costs what. But I don't deal with that in the same way." (TP 13)
  • "I thought of a cake like this [pie chart]. I can read that." (TP 16)
For the evaluation of this item it is of particular relevance whether the test persons perceive the activity in the sense of "understand and interpret" according to the actual wording. The above quotations show that they correctly refer to the reception and not to the active creation - as in item f).
f. Summarize or display information and data from different sources in tables or graphics. [Informationen und Daten aus unterschiedlichen Quellen in Tabellen oder Grafiken übersichtlich zusammenfassen oder darstellen.] Nein The full scale width is also used for this item. A total of eight test persons state that they can summarize or display information and data in tables and graphics, six persons can do so with certain difficulties, three with great difficulties and three test persons state that they cannot do so.

When asked, twelve test persons (TP 01, 05, 06, 07, 08, 09, 14, 15, 16, 17, 19, 20) stated that they had thought of creating tables or graphics themselves when answering:
  • "Because I had to work with illustrations and statistics for my final papers in my student days, or I had to create one myself. Then this goes along with the fact that I have to present information in illustrations." (TP 09)
  • "I always enjoy doing that. Take stock of who has come to me in practice and I also create graphics myself." (TP 17)
The other eight test persons (TP 02, 03, 04, 10, 11, 12, 13, 18), on the other hand, refer to understanding tables and graphics, i.e. merely receiving them and not actively creating them. "If you are busy reading something like this every day. General graphics and statistics.“ (TP 13)

In general, it is noticeable that the test subjects rarely refer to concrete information or data that are to be summarized or presented, but rather mostly make generalized statements. A total of nine persons (TP 01, 02, 03, 10, 11, 12, 13, 14, 16) state that they have great or certain difficulties with it or are unable to do so because they are not concerned with it or are simply afraid of it:
  • "I would have to deal with everything first. Think about it for a moment, but then it should work out." (TP 01, answer: certain difficulties)
  • "I am missing an example here. I cannot answer, because there is no example to take away my fear. Then the question wouldn't be so bad either." (TP 10, Answer: great difficulties)
  • "Too many tables and graphics. So I assumed directly that I couldn't do it." (TP 12, Answer: I cannot)
  • "That sounds so complicated and that's something I don't really like to do. It sounds complicated and you really have to think about what is meant by that." (TP 16, Answer: certain difficulties)
Those test persons (N=8) who state that they can easily master this everyday mathematical activity think of graphics, illustrations or the presentation of statistics in tables:
  • "For example, surveys in the university, you also have to present them in tabular form.” (TP 08)
  • "For example, I was thinking about complex Excel spreadsheets." (TP 19)
While in items a)-e) an example is given of what is meant by the everyday thematic activity described above, no specification is given here. Apart from test person 10, there is no indication that this omission leads to problems in answering. On the other hand, it is not always understandable why twelve test persons choose answers ranging from "I cannot" to "I can but with certain difficulties". This kind of answering behavior can be based on the fact that they do not deal with the subject matter in everyday life or have little practice, perhaps they do not even need it in everyday life or it is unclear to them what the item is aimed at.

The use of the term "or" in the phrase "tables or graphs" is not perceived as problematic here, rather it appears as a non-exclusive disjunction, i.e. an inclusive "or". This enables the test persons to refer to one of the two aspects or both together. No test person indicates to answer differently for the creation of tables than for graphics.
a. Television [Fernseher] Nein