to Pretest Database Pretest Database
Project Title:European Working Conditions Survey 2024
  1. General Information: *Note: The item was tested in English, German and Polish.*
  2. Introduction: English version 1, 2 and 3:
    The following questions are about your health, not necessarily related to your work.

    German version 1, 2 and 3:
    In den folgenden Fragen geht es um Ihre Gesundheit im Allgemein.
  3. Question Text: English version 1, 2 and 3:
    Over the last 12 months, did you suffer under any of the following health problems?

    German version 1, 2 and 3:
    Haben Sie innerhalb der letzten 12 Monate an einem der folgenden gesundheitlichen Probleme gelitten?
  4. Instruction: English question version 1: Please select all that apply.

    German question version 1: Bitte wählen Sie alle zutreffenden Antworten aus.
  5. Answer Categories:

    English version 1:
    Backache
    Muscular pains in shoulders, neck and/or upper limbs (arms, elbows, wrists, hands, etc.)

    Muscular pains in lower limbs (hips, legs, knees, feet, etc.)
    Headaches, eyestrain
    Anxiety
    Overall fatigue
    Other, namely:
    None of the above

    English version 2 (item-by-item design):
    Backache [Yes/No]
    Muscular pains in shoulders, neck and/or upper limbs (arms, elbows, wrists, hands, etc.) [Yes/No]

    Muscular pains in lower limbs (hips, legs, knees, feet, etc.) [Yes/no]
    Headaches, eyestrain [Yes/No]
    Anxiety [Yes/No]
    Overall fatigue [Yes/No]
    Other, namely:

    English version 3 (grid):
    Backache [Yes/No]
    Muscular pains in shoulders, neck and/or upper limbs (arms, elbows, wrists, hands, etc.) [Yes/No]

    Muscular pains in lower limbs (hips, legs, knees, feet, etc.) [Yes/no]
    Headaches, eyestrain [Yes/No]
    Anxiety [Yes/No]
    Overall fatigue [Yes/No]
    Other, namely:

    German version 1:
    Rückenschmerzen
    Muskelschmerzen in den Schultern, im Nacken und/oder in den oberen Gliedmaßen (Arme, Ellenbogen, Handgelenke, Hände usw.)

    Muskelschmerzen in den unteren Gliedmaßen (Hüfte, Beine, Knie, Füße usw.)
    Kopfschmerzen, Überanstrengung der Augen
    Depressionen oder Angstgefühle
    Allgemeine Erschöpfung
    Sonstiges, und zwar
    Keine der genannten Probleme

    German version 2 (item-by-item design):
    Rückenschmerzen [Ja/Nein]
    Muskelschmerzen in den Schultern, im Nacken und/oder in den oberen Gliedmaßen (Arme, Ellenbogen, Handgelenke, Hände usw.) [Ja/Nein]

    Muskelschmerzen in den unteren Gliedmaßen (Hüfte, Beine, Knie, Füße usw.) [Ja/Nein]
    Kopfschmerzen, Überanstrengung der Augen [Ja/Nein]
    Depressionen oder Angstgefühle [Ja/Nein]
    Allgemeine Erschöpfung [Ja/Nein]
    Sonstiges, und zwar

    German version 3 (grid):
    Rückenschmerzen [Ja/Nein]
    Muskelschmerzen in den Schultern, im Nacken und/oder in den oberen Gliedmaßen (Arme, Ellenbogen, Handgelenke, Hände usw.) [Ja/Nein]

    Muskelschmerzen in den unteren Gliedmaßen (Hüfte, Beine, Knie, Füße usw.) [Ja/Nein]
    Kopfschmerzen, Überanstrengung der Augen [Ja/Nein]
    Depressionen oder Angstgefühle [Ja/Nein]
    Allgemeine Erschöpfung [Ja/Nein]
    Sonstiges, und zwar


    1. Recommendations: We recommend to employ an item-by-item format (question version 2) rather than a check-all-that apply format.

      We recommend reviewing the translation of the listed health problems to ensure that differences found between countries are caused by existing differences in health.
  1. Cognitive Techniques:Information image/link to cognitive pretesting Specific Probing
  2. Findings for Question: Findings Web Probing:

    Across all countries, backaches were the most commonly reported health problem, followed by muscular pains in shoulders, neck and/or upper limbs (see Table 41). Of the pre-defined health problems, anxiety was named least often, in particular in Germany.

    Respondents who used a mobile device were generally more likely (84%, n = 259) to report at least one health problem than respondents who used a PC or tablet (77%, n = 360; χ2(1,779) = 5.960, p = .015). Respondents who answered the questionnaire on PC or tablet were significantly more likely to report at least one health problem when they were presented the question in an item-by-item format (see Table 42). There were no significant differences by question version for smartphone users. Within the PC/tablet users, Polish respondents were more likely to report at least one health problem (83%, n = 127) than respondents from Germany (76%, n = 126) and the UK (71%, n = 107; χ2(2,470) = 6.322, p = .042). Among PC/tablet users, respondents in the UK were significantly more likely to report at least one health problem when the question was presented in an item-by-item format than in the other two question versions.

    An examination of the mean number of reported health problems showed that respondents who filled out the questionnaire on a smartphone reported a significantly higher number of health problems (mean: 3.19) than respondents answering on a PC/tablet (mean: 2.60; T(777) = 3.837; p < .001). Regardless of which device respondents used, respondents reported a significantly lower number of health problems when the question was presented in a check-all-that-apply format than as an item-by-item or grid (see Table 43). Respondents from Poland reported significantly more health problems (mean: 3.13) than respondents from Germany (mean: 2.36) or the UK (mean: 2.33; F(2,267) = 7.462, p = .001). Among PC/tablet users, respondents from Germany and Poland reported significantly fewer health problems in the CATA format that in the item-by-item format. The difference between CATA and grid was significant for PC/tablet users from Poland (see Table 43).

    To better understand the differences between question formats, a closed probing question (P1_Q78) asked respondents to rate how strongly the reported health problems impacted everyday life on a five-point scale ranging from “Very weak impact” to “Very large impact”.

    The rationale was to discover whether the check-all-that-apply format promoted respondents to only report severe health problems that impacted them in their everyday life, rather than reporting all health problems they had had. However, there were no significant differences in the level of impact between question versions (see Table 44). This means that respondents in the check-all-that-apply format did not interpret the question differently than respondents who were presented the other formats and did not misconceive the question to be asking about more severe health problems. Thus, the differences in the frequencies with which health problems were reported in the survey question can only be attributed to the CATA format promoting a satisficing behaviour among respondents, resulting in the lower share of reported health problems using this format.

    While there were no differences in the perceived impact on daily life by question version, there were significant differences by device and country. Respondents using a smartphone reported a significantly stronger impact (mean: 3.24) than respondents on a PC/tablet (mean: 2.86; T(617) = 5.120; p < .001). Moreover, Polish respondents reported a significantly stronger impact (mean: 3.02) than German (2.74) or UK respondents (2.81; F(2,357) = 3.289; p = .038).

    Summary:
    • Respondents who were presented the question in the check-all-that-apply format reported a significantly lower number of health problems than in the other two formats.
    • There were no differences in the perceived impact of these health problems depending on question format, indicating that the question presentation in the CATA format promotes incomplete reporting (satisficing behaviour).
    • Response behaviour differed depending on which device was used to fill out the questionnaire. Respondents who filled out the questionnaire on a smartphone reported a higher number of health problems and rated these problems as having a higher impact on their daily life than respondents using a PC or tablet.
    • Polish respondents reported a higher number of health problems and a stronger impact of these health problems on their everyday life than respondents from Germany or the UK. Further research should clarify whether these country-specific differences are caused by differences in health across countries or by the translation of health problems.
  1. Question Topic: Public health/ Health impairments
  2. Construct: Health problems