Findings Web Probing:
Respondents were randomly assigned to question version 1, which included an explicit “Don’t Know” option, or question version 2, which did not. In total, seven respondents (2%) chose the “Don’t Know” option when it was available. No respondent left Q16a unanswered in either question version. There were no significant differences in response behaviour between question versions (χ2(8,792) = 13.912,
p = .084) or countries (χ2(16,792) = 25.546,
p = .061).
The seven respondents who answered “Don’t Know” were asked the reason for this answer in a closed probing question with an additional open-ended response option (P1_Q16a). Two respondents chose the predefined response option “I recently joined the company”, and one respondent chose “The company I work for is being/was recently restructured”. The other four respondents all chose “Another reason”. One of these respondents answered “about 60”, indicating that this respondent could well have found a suitable response option. The other three respondents named legitimate reasons why they did not know the number of people working at their workplace, those being that they had not
“counted the staff”, that they are a temporary worker, or that they simply do not know.
Did respondents include and exclude the correct people as part of their workplace?
In another closed probing question with an open-ended “Other” response option (P3_Q16a), respondents were asked which groups of people they had included or, if they work alone, which groups of people they would have included in their response. This probing question served as an attention check to determine whether respondents had read the information provided to them. The question text said to refer to people working at their “local site”, the ToolTip specified that people working for other companies but at the local site should be included, and the instruction below the question text clarified that co-workers working at other sites should be excluded. Thus, the correct response to the probing question was to select the first and third response, but not the second. Only 2% of respondents in the UK (
n = 6) and Germany (
n = 4) answered correctly (i.e., selected both the first and the third option but not the second), and none of the respondents in Poland. This indicates that most respondents did not pay attention to the exact question wording and instructions presented alongside the question text or the ToolTips. However, despite this, it is reasonable to assume that most respondents answered correctly pertaining to their personal situation. 75% (
n = 592) of respondents correctly included people working for their company at their local site. One in five respondents (20%,
n = 158) incorrectly answered that they were supposed to include people working at other sites. Respondents in the UK were significantly more likely than respondents in Germany or Poland to correctly include people working for other organisations, but at their local site (χ2(2,792) = 13.957,
p = .001). Respondents who chose “Other people, namely” in the probing question usually only commented on their personal situation and that they work alone.
What did respondents understand under the term “workplace”?
Four out of five respondents (79%,
n = 626) gave substantive responses regarding their understanding of the term “workplace” (P2_Q16a). A response was coded as non-substantive if respondents refused to answer (that is wrote nothing or only random characters into the text field) (
n = 49) or when they simply repeated their answer to the survey question or answered off-topic (
n = 118). Non-substantive responses may be a sign of satisficing, that is respondents being unwilling to respond to the probing question but may also indicate that respondents have difficulty describing what “workplace” means in their personal working situation. Supporting this second notion, respondents in atypical working situation were significantly more likely to give non-substantive responses (28%,
n = 81) than respondents who were employed (15%,
n = 42) or self-employed (19%,
n = 44) (χ2(2,792) = 15.754,
p < .001). Moreover, Polish respondents (29%,
n = 76) were significantly more likely to give non-substantive responses than German (18%,
n = 48) or UK respondents (16%,
n = 43) (χ2(2,792) = 14.402,
p = .001).
Among the substantive responses, respondents demonstrated a comprehensive and correct understanding of the term “workplace”. Half of the respondents (50%,
n = 313) described their workplace as the specific physical location(s) they work at, such as their building, site, school, hospital, store, warehouse, construction site or the docks, or a part of such a building, such as their department, their personal office or desk or the ward of a hospital. A quarter of the respondents (26%,
n = 164) gave a vaguely worded definition such as
“where I work” or
“place of work”, which likely also refers to the location. One out of ten respondents (9%,
n = 59) specified their home office as their workplace, and 3% (
n = 19) mentioned client premises. Thirteen respondents stated that they do not have a “fixed” workplace or a workplace “per se”, but that their workplace is wherever they opened their laptop. Ten respondents defined the term workplace as the location where a team comes together to cooperate for a task (
“An environment in which everyone works together as a team”). One out of ten respondents (9%,
n = 57) referred to their company (regardless of the local site). There were significant differences in the distribution of codes between respondents based on their working status, with employed respondents being most likely to name the physical location(s) they work, self-employed being more likely than the other groups to name their home office or agile working places, and respondents in atypical working situations being more likely to offer a vaguely worded definition (χ2(10,626) = 55.316,
p < .001).
Findings Cognitive Interviews:
All respondents answered the question. Only one participant did not know how many people work at his workplace, because he just started working there, and therefore chose “Don’t know” (DE01).
What did respondents understand under the term “workplace”?
Respondents understood “workplace” as the physical location where they work, such as the individual building or building complex, premises in larger buildings, or their home. Regarding businesses with multiple sites, respondents understood their workplace to refer to the building where they mainly work:
- “The company has a building near the station and another big one near the school. And I am in an extra factory again. So, we are not in this main building. I understood that it's only about the people who work directly with me.” (DE03, 10-49)
- “I thought of the place of business, the place where I work. I only took into account the people who work at my place of business. We are represented throughout Germany, but I excluded the others.” (DE11, 100-249)
Whom did respondents include and exclude in the number of people at their workplace, and how confident were they about their answer?
When explaining their answer, most respondents correctly referred to their local site and correctly defined who they included and excluded when counting the number of people at their workplace:
- “I was thinking of the nursing home and the people who work there, that is the nursing staff, the housekeepers, the caretakers. Everyone who is there regularly and works there.”
- “I thought about my basic place of work and first calculated only people teaching children, then I added all the administration staff.“
- “One person exactly, I work on my own. In future there will be another business in the same building, so there will be one more person, but right now I am the only person with some business here.”
However, in several cases, probing showed that some respondents had to interpret parts of the instructions and clarifications to adapt them to their work situation. This was most likely to be the case when respondents were self-employed and either had no business premises or rented premises in larger buildings with unrelated businesses:
- A free-lance actor considered his main client at the time of the interview to be his workplace and referred to the artistic and logistic personnel of the specific play
- A self-employed non-medical practitioner first answered “one person”, but after reading the ToolTip decided to include a person she employed to do the bookkeeping once a month or once every two months
- One self-employed coach had several employees, but no business location, as everyone either worked from home or on client premises. This respondent instead counted the number of people working for her business
- An employed respondent working in a warehouse was unsure whether his “workplace” or “local site” referred only to the warehouse or the entire store. The respondent decided to only consider his direct working environment of the warehouse
- A self-employed designer had his business premises in a large office building that housed many companies and several hundred employees, but only considered his own company consisting of two business partners within the building
All respondents were either “very sure” (DE:
n = 14; PL:
n = 11) or “rather sure” (DE:
n = 2; PL:
n = 5) that their answers were correct. Respondents who indicated that they were only “rather sure” either did not know the exact number of people working in the building they were referring to (e.g., DE02) or were uncertain whom exactly to include in their specific case. The respondent who occasionally had assistance with bookkeeping remained uncertain whether including this person was correct . In Polish, the instruction could be understood as meaning to exclude workers at “other sites or premises” or at “other sites or in other departments”. This caused some respondents to consider only their direct department (
n = 1) or the entire company (
n = 2).
Did respondents read the instructions and ToolTip, and did they find the information useful?
Four respondents from Germany and nine from Poland did not read the instructions. In Poland, three respondents claimed not to have noticed the instructions. All others explained that the question was clear for them without reading the instruction:
- “I did not read that, but my answer is still correct. So, I took it into account without having read it.” (DE05)
- “I didn't miss it, I skipped it because I was very sure. But because I was curious and I was able to answer it quickly, I read it afterwards.” (DE10)
- “The instruction sort of blurs with the question, but the question was obvious.” (PL06)
- “I skipped it because I knew how many people work here.” (PL07)
In both countries, six respondents read the
ToolTip spontaneously, while ten did not use the ToolTip for “local site” . These respondents explained that they did not think it was necessary because for them the term was self-explanatory:
- “Because for me the term is actually clear.” (DE04)
- “Because it was clear to me that it could only be the building where I now work with many other colleagues who also work in other departments, but that all belongs to the workplace.” (DE08)
All respondents indicated that the information in the instruction and the ToolTip was generally useful and relevant at least to some respondents. However, several respondents had difficulties combining the information from the instructions with the information from the ToolTip on whom to include and exclude in their response. For instance, two respondents in Poland remained uncertain how to deal with businesses based in large building complexes that contained many unrelated companies (PL11, PL14):
- “I cannot really understand the third point [in the ToolTip]. It says ‘workplace’ and I consider my company to be my workplace, and not the physical place only […] If there was a big office building and my company employed ten people, while 3,000 people work in the office building, should I indicate ten people or 3,000? […] If I [read] the tooltip, I would indicate 3,000 people. If you had not asked me to read it, I would have indicated ten.” (PL11)
One respondent (DE03) suggested to move some parts of the ToolTip to the instruction:
- “I would display the last two points [of the ToolTip] directly below the question. Because these are not explanations, but important information for answering the question. For example, ‘Please refer only to your main place of work’ - I don't know that unless I click on [the ToolTip].” (DE03)
Summary:
- Based on the findings from both web probing and cognitive interviewing, only few respondents did not know how many people worked at their workplace. These people, however, required a “Don’t Know” response option, for instance, because they recently joined the company or worked for a temporary work agency.
- In both web probing and cognitive interviews, the term “workplace” was clear to most respondents, regardless of their personal working situation, and whether they read the ToolTip.
- Based on the findings from both cognitive interviews and web probing, most respondents generally did not read the instructions and clarifications in detail. However, in most cases, respondents seemed to give the correct answer despite this.
- Most of the cognitive interviewing respondents estimated the people based on the building in which they mainly work. In ambiguous cases, respondents had to make decisions whom to include. For instance, when a company was located in a larger building complex containing many unrelated companies, respondents referred to their company only. Respondents who employed external assistance (e.g., bookkeeping or cleaning) included these people (see also the subchapter on the translation of “workplace” in chapter 2).
