to Pretest Database Pretest Database
Project Title:European Working Conditions Survey 2024
  2. General Information: *Note: The item was tested in English, German and Polish.*
  3. Question Text: English version 1, 2 and 3:
    Would you decribe yourself as...?

    German version 1, 2 and 3:
    Wie würden Sie sich selbst hinsichtlich Ihrer Geschlechtsidentität beschreiben?
  5. Answer Categories:

    English version 1:
    a man
    a woman
    other


    English version 2:
    a man
    a woman
    other, namely:


    English version 3:
    a man
    a woman


    German version 1:
    Männlich
    Weiblich
    Andere Geschlechtsidentität


    German version 2:
    Männlich
    Weiblich
    Andere Geschlechtsidentität, und zwar:


    German version 3:
    Männlich
    Weiblich

    1. Recommendations: Based on cognitive pretesting, no changes to the question are recommended.

      However, the question should be revisited from the perspective of cross-cultural comparability. While the English source questionnaire asks about gender identity without using the word “gender” in any way by asking how the respondents would “describe themselves”, the German translation directly asks about “gender identity” and the Polish translation is worded as a sentence and simply asks respondents to indicate their “gender”. Formulating a question that unmistakenly asks about gender identity, and thus self-identification, remains challenging in many languages, and it is not possible to word the question in the same way across languages. For instance, in German, the meaning of a literal translation of the source “Would you describe yourself as…” remains unclear until the words “regarding your gender identity” are added to the question text. In Polish, in contrast, asking about gender self-identification is considered highly intrusive. Thus, a careful compromise must be found between keeping the construct of gender identity comparable across languages and countries, and remaining sensitive to linguistic and cultural nuances. We recommend that each country document their way of translating providing a rationale and an explanatory back translation.
  2. Cognitive Techniques:Information image/link to cognitive pretesting Web probing: Specific Probing
    Cognitive interviews: Specific Probing
  3. Findings for Question: Findings Web Probing:

    In the web probing study, the question on gender was quota-relevant and thus presented at the beginning of the survey. Respondents were randomly assigned to question version 1, 2 or 3.

    Did respondents select the response option “other” and what did respondents mean when they chose the gender option “other”?

    Only one respondent who completed the survey chose the third gender option “other” (see Table 56). This person was assigned to question version 1 with the closed response option; therefore, the respondent’s exact self-identification remains unknown.

    Of those respondents who did not complete the survey , seven respondents chose the option “other”. Of these, four were assigned to question version 1 with the closed “other” category, and three to question version 2 with the open-ended “other” category. Two were from the UK, three from Germany, and two from Poland. One of the three respondents with the open-ended “other” category inserted a specification (“trans woman”), indicating a correct understanding and usage of the response option.

    Did respondents break off the survey when a non-binary gender option was shown?

    In total, 157 respondents dropped out of the survey. Of these, only nine respondents dropped out on the survey page that included the question on respondent gender. There were no significant differences between countries or the question version (see Table 57).

    Did respondents perceive the non-binary category as a provocation?

    A closed probing question asked respondents in question versions 1 and 2 what they thought about presenting a third gender category in a survey (P1_Q2new). Respondents could indicate whether the found it “(rather) appropriate”, “(rather) inappropriate”, or whether it didn’t matter to them if a third option was presented (see Table 58). Across all countries and question versions, 15% of respondents expressed a negative attitude, finding it “(rather) inappropriate” to present a third gender option. There was a significant difference in response to the probing question between countries (χ2(4,531) = 47.895, p < .001). In the UK and Germany, the most common response was that it didn’t matter. In Poland, in contrast, the most common answer was that respondents found it appropriate to include a third gender option. There were no significant differences depending on question version in any country (UK: χ2(2,180) = 2.625, p = .269; DE: χ2(2,173) = 1.532, p = .465; PL: χ2(2,178) = 2.364, p = .307).

    Findings Cognitive Interviews:

    The cognitive interviews aimed to examine if the third non-binary response option was accepted and which question version the respondents preferred. Respondents first answered question version 1 (with the closed “other” category) and after responding to two probes on this version, they were shown the alternative question versions 2 and 3 on a separate survey page. None of the respondents chose the response category “other” (see Table 59).

    While there were no problems in understanding this question in Germany, four respondents in Poland stated that they did not know who the question is addressing. The Polish translation literally read “Please indicate [your] gender”, with the pronoun “your” not being explicitly named in the question text. This caused confusion among two respondents who were still contemplating the questions on their boss’s gender:
    • “The question is not really clear because it does not specifically say whose gender should be marked here.” (PL16)
    • “This statement is too general, especially as we’ve just had questions about the male boss, female boss, manager and so on.” (PL11)

    Did respondents perceive the non-binary category as a provocation?

    In Germany, 14 respondents stated that they found the non-binary response option “(rather) appropriate” and two indicated that it “doesn’t matter”. In Poland, ten respondents stated that they found the non-binary response option “(rather) appropriate”, four indicated that it “doesn’t matter”, one found it “(rather) inappropriate”, and one respondent had no preference. Most respondents argued that it was good that people who cannot identify themselves as either male or female had an opportunity to respond appropriately and that this was in accordance with the times:
    • “Personally, I don't care, but I think it's appropriate because nowadays there are also people who are not even aware whether they are a man or a woman or something. That's why I think the answer option ‘Other gender identity’ is also good.” (DE07)
    • “I think it's good that people who don't want to clearly label themselves as male or female are given this option. This should be used as often as possible against discrimination. These people should be seen and noticed.” (DE12)
    • "It doesn't matter for me, but I understand that in the light of what is happening now, which is related to different genders, to human freedom, I think that [‘other’] should be here.“ (PL09)
    One respondent (PL03) who found the non-binary option “(rather) inappropriate” as well as one respondent (PL16) who stated that it “doesn’t matter” explained that there were only two genders for them, namely men and women.

    Which of the three question versions did respondents prefer?

    In Germany, eight respondents indicated that they preferred question version 1 with the closed “other” category, seven indicated that they preferred question version 2 with the open-ended “other” category and one respondent had no preference. In contrast, the open variant was preferred by most respondents in Poland (n = 11). Three respondents indicated that they preferred question version 1 with the closed “other” category and two preferred the question version with only two response options (PL03, PL16).

    Those who indicated that they preferred the question version with the closed "other" category explained that answering the question in an open format would contain too much private information:
    • “It's enough to state that it's a different gender identity. What exactly is not anyone's business.” (DE02)
    • “The [closed] ‘other’ category, and there does not have to be anything to specify here. I don’t think anyone would like to get into details.” (PL12)
    Respondents who indicated that they preferred question version 2 explained that this version offered the opportunity to describe themselves in more detail while at the same time allowing for simply selecting the answer option without specifying it any further:
    • “You can enter everything here, there is room for all possibilities. And you can also just click on it and leave the text field open.” (DE01)
    • “[…] gives the option of a) the other gender identity and b) especially the description. Whether you use it or not is the question, you can also write in, don't want to make a comment, but the option should actually be there.” (DE04)
    One respondent did not indicate a preference, as she did not fall into the category of "other", and therefore found it difficult to assess which of the two alternatives was the better one: “Since I don't fall into this category myself, I don't know to what extent someone there would like to put that down or not” (DE15).

    Summary:
    • In web probing, only one respondent used the third gender option “other”. This person was assigned to the question version with the closed response option; therefore, the respondent’s exact self-identification remains unknown.
    • Break offs on the survey page with the gender question were extremely rare. Offering a third response option did not influence break offs.
    • In web probing, unfavourable reactions to the third gender option were voiced by 15% of respondents. Polish respondents reacted more favourably to the inclusion of a third gender category in response to a (closed) probe than respondents in the UK or Germany, who were in turn more likely to react indifferently towards the third option. Whether the third option was presented as a closed or open-ended response option did not influence the reaction of the category.
    • In the cognitive interviews, all except two Polish respondents found the question versions offering a non-binary response option appropriate. There was no clear preference for this response option to be closed or open-ended.
  2. Question Topic: Social demographics
  3. Construct: Gender identity