Findings Web Probing:
Respondents were randomly assigned to either question version 1 or question version 2. For the first
item, there were significant differences between countries in both question versions (see Table 51;
question version 1: χ2(8,395) = 17.020, p = .030; question version 2: χ2(12,397) = 24.734, p = .016), with German respondents being more likely to report that they worked at a very high speed. In contrast,
there were no significant differences between countries in either question version of the second
item (see Table 52; question version 1: χ2(8,395) = 7.544, p = .479; question version 2: χ2(12,396) = 17.334,
p = .137).
Which response options in the two scales correspond to each other?
From a logical and linguistic point of view, the end points of the scale should approximately correspond to each other (“Always” and “All of the time”, as well as “Never”), as should the second-lowest frequency (“Rarely” and “Almost never”). The response option “Often” from the 5-point scale should encompass the response options “Almost all of the time” and “Around ¾ of the time” from the seven-point scale, while the response option “Sometimes” should encompass the response options “Around half of the time” and “Around ¼ of the time” (see Table 49 in Q30). The response distributions confirm this notion (see Table 53).
Findings Cognitive Interviews:
Respondents were first shown question version 1 (5-point scale) and after discussing this question version, they were shown question version 2 (7-point scale) on a separate screen. The probes aimed at examining how the respondents understood the response options in both question versions and whether they preferred the 5-point or the 7-point scale.
In the question version using the five-point scale, none of the Polish respondents answered either item with “always”, while none of the German respondents answered the first item with “never” (see Table 54 and Table 55).
How did respondents interpret the response options on the 5-point scale?
Respondents who said they “always” had to work at very high speed or to tight deadlines referred to having constant pressure:
- “I always have to hurry and am always under pressure to get everything done. In nursing, you never actually get to sit down.” (“always, highspeed”)
- “Our diary is always full. When I open the diary in the morning and see that appointments are double- and triple-booked, I already know how the day will go.” ( “always, tightdead”)
Respondents who answered one or both items with “often” explained that this meant “almost all of the time” or “almost every day”, but that working at a high speed or working towards tight deadlines was the typical, but not the only state of working they knew:
- “There are phases in which I have deadline pressure, but that doesn’t necessarily mean one has to work at a fast pace.” (“often, tight deadlines”)
- “It is not always the case, sometimes there are quiet phases.” (“often, highspeed”)
- “[Often] means ‘practically every day’. We get so-called online orders and must complete them. Given the current situation, people would rather shop online than in a stationary shop.“ (“often, highspeed”)
The scale point “sometimes” was interpreted as meaning “from time to time” or “not too often”, while “rarely” was associated with something that happens extremely seldom. Some respondents explained that these were cases that occurred, but were not their typical work situation, as their main tasks involved the opposite of working at a high speed or towards tight deadlines. Thus, the scale points “sometimes” and “rarely” describe events that are seldom or irregular:
- “My job rather requires concentration than fast pace.” (“rarely, highspeed”)
- “It depends on clients who expect me to respond quickly to some notification, but these are not frequent cases.” (“sometimes, highspeed”)
One respondent said that she needs an additional response option “regular” because “often” is not suitable for “working under tight deadlines”:
- “I can't answer the question ‘working under tight deadlines’. I would need a category like ‘regularly’. For me, that's regularly every month. ‘Often’ is not necessarily regularly.” (“refusal, tight deadlines”)
How did respondents interpret the response options on the 7-point scale?
On the 7-point-scale respondents calculated the amount of time they work at very high speed or toward tight deadlines into hours per day, per week or per month. When they were uncertain about the exact share of working time, they preferred the vaguely worded response options “almost all of the time” instead of “around ¾ of the time” and “almost never” instead of “around ¼ of the time”:
- “I don't know how to calculate it, honestly. A quarter of the time is 25%. Twenty-five percent of 30 hours doesn't seem like it to me. It happens once or twice a week that a patient comes in. That is not 25%.” (DE14, “almost never, highspeed”)
- “It is difficult to distinguish between ‘almost all of the time’ and ‘around ¾ of the time’. These two phrases are very much alike.” The respondent further explained that she chose ‘almost all of the time’ because in the other case, she would have had to calculate whether she worked at a high speed around six hours a day; therefore, the general wording was closer to her perceived reality (PL10, “almost all of the time, highspeed”)
- “I believe [the seven-point scale is] better, because it is a measurable scale. It can be compared to something. If I were to answer here, I would indicate ‘almost never’ or ‘around ¼ of the time’. ‘Around ¼ of the time’ of a 40-hour working week is ten hours. Therefore, I have to indicate ‘almost never’.” (PL11, “almost never, highspeed”)
- One respondent understood “almost all of the time” to imply that he must work at a high speed on a daily basis: “Every day at some moment of the day, that’s for sure.” (PL13, “almost all of the time, highspeed”)
- „I hesitate between ‘around ¼ of the time’ and ‘almost never’, because ‘around ¼ of the time’ is two hours out of eight, I guess. That could be a bit much.“ (PL16, “almost never, highspeed”)
One respondent refused to answer both items using the seven-point scale, because she could not calculate a share of working time (PL07).
Which response options in the two scales correspond to each other?
Based on the assumed corresponding response options (see Table 49), the majority of respondents (Q49A: DE:
n = 10, PL:
n = 11) (Q49B: DE:
n = 12, PL:
n = 8) used corresponding response options on the five-point and seven-point scales. Respondents who diverged were more likely to report a lower frequency using the seven-point scale (Q49A: DE:
n = 5, PL:
n = 4) (Q49B: DE:
n = 3, PL:
n = 5). One German respondent reported a higher frequency using the seven-point scale for the first item, and one Polish respondent reported a higher frequency using the seven-point scale for the second item. The remaining two respondents refused to answer the item using the five-point scale only (DE, highspeed), the seven-point scale only (PL, both items). In the following, the reasons why respondents chose diverging response options on the two scales are discussed:
- The respondent who answered both items with “Always” on the five-point scale and “Almost all of the time” using the seven-point scale explained that she had to work at a very high speed under time pressure. For her, these scale points corresponded to each other because she interpreted “Always” to mean that she works at a high speed and to tight deadlines “every day”, but she chose “Almost all of the time” instead of “All of the time” because on most days, she had the opportunity to sit down and take a break.
- Respondent who answered both items with “Often” using the five-point scale and “Around half of the time” using the seven-point scale: Directly upon seeing the five-point scale, this respondent commented that they would prefer a seven-point scale and explained the choice of the response option “Often” to the first item as meaning “I say half of my working time is fast paced”
.
- The respondent who answered the first item with “Often” and refused to answer the second item using the five-point scale but chose “Around ¼ of the time” for both items using the seven-point scale explained that the workload was usually very high towards the beginning of the month, but decreased towards the end, and that the seven-point scale made it easier to indicate this.
- The respondent who answered the first item with “sometimes” on the five-point scale but refused to answer the item using the seven-point scale explained the difference between the scales as follows: “‘Around ¼ of the time’ would mean two hours a day. If I worked at a very high speed [two hours a day], this would seem like a stressful job and would be exaggerated. In fact, I do not know which answer to indicate. The term ‘sometimes’ implies that it simply happens, but I am not able to define it precisely, because it is not every day or with any regularity.”
- The respondent who answered the first item with “often” on the five-point scale and “around half of the time” on the seven-point scale explained that he spent half of his working time processing online orders, which required him to work quickly. This share of working time corresponded to the response option “often” on the five-point scale.
Both respondents who chose a higher frequency on the seven-point scale than on the five-point scale chose “rarely” on the five-point and “around ¼ of the time” of the time on the seven-point scale (DE06, highspeed; PL16, tightdead).
Respondents indicated a slight preference for the 5-point scale in Germany and for the seven-point-scale in Poland, resulting in equal proportions across both countries (5-point scale preferred: DE:
n = 9, PL:
n = 6) (7-point scale preferred: DE:
n = 6, PL:
n = 9). In each country, one respondent had no preference. The five-point scale was considered more intuitive and reader-friendly but was criticised as being less exact and objective. The seven-point scale was considered more precise and the more objective measure but required more reflection and calculating hours.
Summary:
- Based on the response distributions in web probing, the five-point and seven-point scales corresponded well to each other, indicating that both can be used to capture the constructs they measure.
- The results from the cognitive interviews confirmed that many respondents used corresponding response options on both scales. When there were differences, respondents tended to report a lower frequency using the seven-point scale.
- Cognitive interviews did not reveal a clear preference for one scale type. The seven-point scale was considered more exact, and the five-point scale more intuitive.
