Findings Web Probing:
Across all countries, backaches were the most commonly reported health problem, followed by muscular pains in shoulders, neck and/or upper limbs (see Table 41). Of the pre-defined health problems, anxiety was named least often, in particular in Germany.
Respondents who used a mobile device were generally more likely (84%, n = 259) to report at least one health problem than respondents who used a PC or tablet (77%, n = 360; χ2(1,779) = 5.960, p = .015). Respondents who answered the questionnaire on PC or tablet were significantly more likely to report at least one health problem when they were presented the question in an item-by-item format (see Table 42). There were no significant differences by question version for smartphone users. Within the PC/tablet users, Polish respondents were more likely to report at least one health problem (83%, n = 127) than respondents from Germany (76%, n = 126) and the UK (71%, n = 107; χ2(2,470) = 6.322, p = .042). Among PC/tablet users, respondents in the UK were significantly more likely to report at least one health problem when the question was presented in an item-by-item format than in the other two question versions.
An examination of the mean number of reported health problems showed that respondents who filled out the questionnaire on a smartphone reported a significantly higher number of health problems (mean: 3.19) than respondents answering on a PC/tablet (mean: 2.60; T(777) = 3.837; p < .001). Regardless of which device respondents used, respondents reported a significantly lower number of health problems when the question was presented in a check-all-that-apply format than as an item-by-item or grid (see Table 43). Respondents from Poland reported significantly more health problems (mean: 3.13) than respondents from Germany (mean: 2.36) or the UK (mean: 2.33; F(2,267) = 7.462, p = .001). Among PC/tablet users, respondents from Germany and Poland reported significantly fewer health problems in the CATA format that in the item-by-item format. The difference between CATA and grid was significant for PC/tablet users from Poland (see Table 43).
To better understand the differences between question formats, a closed probing question (P1_Q78) asked respondents to rate how strongly the reported health problems impacted everyday life on a five-point scale ranging from “Very weak impact” to “Very large impact”.
The rationale was to discover whether the check-all-that-apply format promoted respondents to only report severe health problems that impacted them in their everyday life, rather than reporting all health problems they had had. However, there were no significant differences in the level of impact between question versions (see Table 44). This means that respondents in the check-all-that-apply format did not interpret the question differently than respondents who were presented the other formats and did not misconceive the question to be asking about more severe health problems. Thus, the differences in the frequencies with which health problems were reported in the survey question can only be attributed to the CATA format promoting a satisficing behaviour among respondents, resulting in the lower share of reported health problems using this format.
While there were no differences in the perceived impact on daily life by question version, there were significant differences by device and country. Respondents using a smartphone reported a significantly stronger impact (mean: 3.24) than respondents on a PC/tablet (mean: 2.86; T(617) = 5.120; p < .001). Moreover, Polish respondents reported a significantly stronger impact (mean: 3.02) than German (2.74) or UK respondents (2.81; F(2,357) = 3.289; p = .038).
Summary:
