to Pretest Database Pretest Database
Project Title:PIAAC-Longitudinal (PIAAC-L) 2016 - Acquisition of Competencies (English Version)
  1. General Information: *Note: This item was tested in German. This is an English translation of the original German wording. The translation does not correspond exactly to the wording in the English PIAAC source questionnaire.*

    Questions 13-15 were only asked to those persons who stated in question 12 that they had been employed in the previous week.
  2. Introduction: Now it's all about your work-related skills that you have learned, in other words your ability. These can be practical skills, such as the correct operation of a machine, but also logical or creative thinking.
    [Nun geht es um Ihre arbeitsbezogenen Fertigkeiten, die Sie erlernt haben, also Ihr Können. Dies können praktische Fertigkeiten, wie bspw. das richtige Bedienen einer Maschine, aber auch logisches oder kreatives Denken sein.]
  3. Question Text: How do you rate your skills in comparison to what is required for the completion of your work?
    [Wie schätzen Sie Ihr Können im Vergleich dazu ein, was für die Erledigung Ihrer Arbeit erforderlich ist?]
  4. Instruction: INT: Read out the instructions. [INT: Vorgaben vorlesen.]
  5. Answer Categories 1 I can do more than is necessary. [Ich kann mehr als erforderlich ist.]

    2 For this work I would need other skills that I should acquire. [Für diese Arbeit bräuchte ich weitere Fertigkeiten, die ich mir aneignen sollte.]

    3 I can do exactly as much as is necessary. [Ich kann genau so viel wie erforderlich ist.]


    1. Recommendations: The recommendations are similar to question 14:
      As a general rule, before the set of questions on work-related knowledge and skills, information should be given in a clearly defined introductory text that different areas are to be surveyed, so that those surveyed do not take all work-related skills into account in the first question (question 14) and as a result find it difficult to distinguish between knowledge and skills (question 15).
      Introduction: Now it's all about your work-related skills that you have learned, in other words your ability. These can be practical skills, such as the correct operation of a machine, but also logical or creative thinking.
      [Nun geht es um Ihre arbeitsbezogenen Fertigkeiten, die Sie erlernt haben, also Ihr Können. Dies können praktische Fertigkeiten, wie bspw. das richtige Bedienen einer Maschine, aber auch logisches oder kreatives Denken sein.]
      Question: Please think about the day-to-day execution of your work: How do you rate your skills compared to what is required to complete your work? Again, it is not about the possible extensions of tasks or your career in the job.
      [Bitte denken Sie an die alltägliche Ausführung Ihrer Arbeit: Wie schätzen Sie Ihr Können im Vergleich dazu ein, was für die Erledigung Ihrer Arbeit erforderlich ist? Es geht auch hier nicht um die möglichen Erweiterungen der Aufgaben oder Ihre Karriere im Job.]
      (INT: Present list with answer scale) [(INT: Liste mit Antwortskala vorlegen.)]
      Answer options: The answer category "For this work I would need other skills that I should acquire" needs to be adjusted, as it is not clear that it is about skills that are not sufficient to do the work. In addition, a scale with three response options allows for little variance and promotes socially desirable response behavior. To counteract this, we recommend the use of a bipolar, end-point and center-labeled 11-point scale:
      1 Much less knowledge [Sehr viel weniger Kenntnisse]
      2
      3
      4
      5
      6 Just as much knowledge as necessary [Genauso viele Kentnisse wie erforderlich]
      7
      8
      9
      10
      11 Much more knowledge [Sehr viel mehr Kentnisse]
      Alternative naming, which explicitly takes up the target dimension "ability" in the scale, but is longer:
      1 I can do much less than necessary [Ich kann sehr viel weniger als erforderlich ist.]
      2
      3
      4
      5
      6 I can do just as much as necessary. [Ich kann genauso viel wie erforderlich ist.]
      7
      8
      9
      10
      11 I can do much more than necessary [Ich kann sehr viel mehr als erforderlich ist.]
  1. Cognitive Techniques:Information image/link to cognitive pretesting General Probing, Specific Probing, Comprehension Probing
  2. Findings for Question: In total, seven (TP 04, 06, 08, 09, 11, 14, 16) of the 14 test persons say that they can do more than is necessary to do their job, two (TP 01, 07) admit that they need additional skills for their work that they should acquire and five (TP 02, 03, 05, 13, 15) say that they can do exactly as much as is necessary.

    It is also noticeable in question 15 that the two test persons (TP 01, 07) who choose the answer category "For this work I would need further skills that I should acquire" choose this answer because they think that you can always learn more: "You can always do better. I don't think I would ever answer that question any other way." (TP 07).

    In addition, eleven test persons find it difficult to answer the question, either because they perceive several stimuli (TP 14, 15), or because they find it difficult to make a general distinction between work-related knowledge and skills (TP 01, 03, 04, 06, 07, 08, 09, 13, 16). Test persons 14 and 15 take up different aspects of the given definition to explain their response in more detail and assess their skills for the individual stimuli, sometimes differently:
    • "Yes, well I find it difficult to answer, very difficult. Well, I think the examples are good, but in my case I still lack a few skills to use the tool properly. However, when it comes to logical or creative thinking, I have more skills than I need.” (TP 14; Answer: can do more than necessary)
    • "Concerning as skills that one acquires, for example, a machine setting [...] there, I am definitely not beyond skills. As for analytical thinking, it's difficult to say. I don't know. But I don't think I'm beyond what is required. What I bring to the table is what is required. Yeah, I can't say beyond that, I don't know. I don't have enough experience for that, maybe in other areas as well.“ (TP 15; Answer: can do as much as necessary)
    The other nine test persons, who find it difficult to answer the question, see little difference between work-related knowledge and skills in relation to their respective jobs:
    • "Skill sounds a little more technical than knowledge, but in principle, there's no difference." (TP 01)
    • "If I know something, then I can use it somehow and then I have the skill and I can use it, make it and put it into practice. So how can you really separate the two?" (TP 03)
    • "I would answer the question as before [like question 14], because I cannot and would not distinguish greatly between knowledge and skills. It would be different if I were physically working." (TP 04)
    • "Whether that is competence or skill. That's a fine line. Communication would again be a skill or the preparation for a lesson." (TP 06)
    Asked what the 14 test persons understand by "work-related skills" and whether they can think of other examples than those mentioned in the question, they tend to associate the term with practical activities or doing, often in connection with physical work:
    • "In the penultimate [question 14] I was already thinking about what I was going to do with the goods, where they would arrive and the logistics, and in the last [question 15] I was thinking about how best to pack a box.” (TP 01)
    • "Skill is something you learn. By skill I mean something routine, something you do every day, and just something physical." (TP 04)
    • "We have a Smartboard in the classroom. It would be a skill to use, and I have no trouble with it." (TP 06)
    • "So knowledge [question 14] means I need to know what I'm doing in production. I must have the knowledge of how the whole production should or can run. [...] And the other area [skills] would be rather, how do I implement the whole thing in order to get a smooth process to satisfy the customer.“ (TP 13)
    • "Well, I would delimit it from the previous [question 14] like this: knowledge is more theory and skills are more practice.“ (TP 14)
    Test person 16 mentions rather general skills that are important in her job: "The job I'm doing at the moment requires a lot of concentration, conscientiousness, accuracy, you have to be able to calculate, calmness. So, specifically in relation to the job."

    If you compare the answers to question 14 and question 15 directly, you can immediately see that nine of the fourteen test persons gave identical and five different answers to the two questions. Six test persons (TP 01, 06, 08, 09, 13, 16), who gave identical answers to questions 14 and 15, made no or only minor differences between knowledge and skills. This is also the case for three (TP 03, 04, 07) of the five test persons who answer the two questions differently.
  1. Question Topic: Job and career/ Job situation & professional activity
  2. Construct: Self-assessed skills compared to professional requirements