to Pretest Database Pretest Database
Project Title:European Working Conditions Survey 2024
  2. General Information: *Note: The item was tested in English, German and Polish.*
  3. Question Text: English version:
    What sector of business or industry is the company or organisation where you work mainly active in?

    German version:
    In welcher Branche oder in welchem Wirtschaftszweig ist das Unternehmen bzw. die Organisation, für die Sie arbeiten, überwiegend tätig?
  4. Instruction: English version:
    Please describe the sector of business or industry in as much detail as possible. Examples are: “retail sale of food, beverages and tobacco” (not: “trade”), “manufacture of cutlery” (not: “factory”), “management of real estate on a fee or contract basis“ (not: “service sector”), or “software development” (not: “IT”).

    German version:
    Bitte beschreiben Sie die Branche/den Wirtschaftszweig so detailliert wie möglich. Beispiele sind: "Einzelhandelsverkauf von Lebensmitteln, Getränken und Tabakwaren" (nicht: "Handel"), "Herstellung von Besteck" (nicht: Fabrik), "Verwaltung von Immobilien auf Honorar- oder Vertragsbasis" (nicht: "Dienstleistungssektor"), oder "Softwareentwicklung" (nicht: "IT").
    1. Recommendations: Regarding the question format, the results from cognitive pretesting show that a four-digit coding of the NACE level cannot be achieved for a clear majority of respondents using open-ended questions. If a four-digit NACE level coding is required, we suggest that this information is collected by interviewers in the course of an offline recruitment.

      Regarding the question text, the Polish translation of the word “sector” should be omitted, and only the translation of the word “industry” kept, to avoid confusion with the public and private sector

      Regarding the instruction, we recommend
      • mirroring the grammatical structure of the instructions in Q5 (i.e., “write ‘[detailed example]’ instead of ‘[vague example]’”)
      • including examples from common industries, such as construction, transport, health sector, or teaching
  2. Cognitive Techniques:Information image/link to cognitive pretesting Cognitive interviews: Specific Probing
  3. Findings for Question: Findings Web Probing:

    Despite the fact that the instruction in question 13 asked respondents to describe the sector of business or industry in as much detail as possible, most of the answers in all three countries were very short (UK: M = 2.3 words, DE: M = 2.2 words, PL: M = 2.2 words). Even when additionally taking the respondents’ answers on their occupation in Q5 and Q6 into account, it was only possible to assign the 3-digit NACE code to approximately 60% of the respondents (see Table 68).

    There were no significant differences in codability depending on country (see Table 68; χ2(4,792) = 9.098, p = .059), gender or age. However, there were significant differences in the codability of responses based on the working situation of respondents (see Table 69; χ2(4,792) = 12.872, p < .05). Respondents in atypical working situations were most likely to not provide any form of substantive answer. Importantly, however, respondents who were employed were significantly more likely to provide substantive, but non-codable responses (39%, n = 108) than respondents who were self-employed (30%, n = 69) or in atypical working situations (31%, n = 89). This finding, which is consistent with the pattern of response quality to Q5 and Q6, indicates that respondents who are employed are least likely to find their occupation and sector worth a detailed explanation.

    Also mirroring the findings from the questions on occupation, responses that could be coded at the three-digit NACE 2.0 level were significantly longer than responses that could not be coded at this level (see Table 70; T(746) = -4.568, p < .001). However, with codable responses to Q13 averaging between two and three words for codable responses, minimum requirements on response length cannot be recommended to improve response quality.

    The low share of codable responses suggests that most web respondents either did not read the instruction (thoroughly) or were not motivated to spend much time on typing in more detailed answers. The findings from the cognitive interviews should shed light on this question.

    Findings Cognitive Interviews:

    Did the answers to the open-ended question allow for classifying the sector of business or industry in which the respondents work in at the 3-digit level of NACE 2.0?

    As shown in Table 71, ten of the 16 German respondents provided sufficient information in their answers to question 13 to classify the sector of business or industry at the 3-digit level of NACE 2.0. If we additionally considered the respondents’ answers to questions 5 and 6, it was possible for all but one German respondent to classify the sector at the 3-digit level.

    Regarding the Polish respondents, only eight provided sufficient information to classify the sector of business or industry at the 3-digit level of NACE 2.0. If we additionally considered the respondents’ answers to questions 5 and 6, it was possible for a total of twelve Polish respondents to classify the sector at the 3-digit level.

    All in all, it was possible to determine the NACE 2.0 code for 84% (n = 27) of the cognitive interview participants.

    Did respondents read the instruction below the question text and did they find it understandable?

    All but one respondent (PL12) indicated that they had read the instruction when answering question 13 and most found the instruction “very” or “rather understandable”. However, as mentioned above, the answers of more than half of the respondents (n = 17) were not coherent with the instruction, suggesting that it needs to be revised.

    Three German respondents criticised the grammatical structure of the examples given (with the “not:...” in brackets). This would make it unnecessarily complicated to understand what kind of answer is desired:
    • “By ‘not’, do you mean not to write it? I understand it to mean that you want me to be more specific. So, I shouldn't just write such a broad term? […] Perhaps it would be better to turn it around and not write it in brackets but integrate it directly into the sentence. So ‘Instead of factory, please write manufacture...’.” (DE03)
    Seven respondents (DE: n = 3, PL: n = 4) suggested revising some of the examples to make them clearer and adding or substituting some of them:
    • “The example ‘management of real estate on a fee or contract basis’ is already very specific, whereas ‘software development’ is still a very general word and not very meaningful in this case. If for ‘factory’ the specification is ‘manufacture of cutlery’, for ‘software development’ I also expect a more precise specification such as ‘software development for computer games/apps’.”
    • “The examples all refer to the economy, so it might be helpful to include something that is not purely economic. Education, for example. That the explanation is a little more diverse.”
    • “They are helpful, but I wonder whether it would not be worth having a close look at the industry or sectors which are dominant in the economy (in terms of number of people employed in them), because ‘cutlery production’ is rather detailed and few people would use this hint. I understand that this is supposed to show how detailed the answer should be. Simply, it could be better to name more popular industries.”
    Four Polish respondents pointed out that the term “market sector” was confusing in this question, because they associated it with the notions of “private sector”, “public sector”, and “not for profit sector”. The term “market sector” was interpreted much broader than “industry” and using both terms together in the question text confused respondents.

    Summary:
    • Only about 60% of the web probing respondents gave sufficiently detailed answers to Q13 to assign the 3-digit NACE code, even when combining their answers to Q13 with those to Q5 and Q6. In the cognitive interviews, results were markedly better, with 84% of respondents providing sufficient details.
    • Respondents found the instruction clear and helpful, however, more than half of them did not answer in accordance with the instruction. Some suggestions were made on how the instruction could be revised.
    • Several Polish respondents were confused by the term “market sector” in the question text, which they associated with the private, public or not-for-profit sector. Using it together with the term “industry” made them wonder what kind of information was required.
  2. Question Topic: Job and career/ Job situation & professional activity
  3. Construct: Sector