2. General Information: *Note: The items were tested in German. These are English translations of the original German wordings.
3. Question Text: You hear different opinions about Jews here and there. We have compiled some of them here. Would you please tell us to what extent you agree or disagree with these statements?
   [Man hört hier und da verschiedene Meinungen über Juden. Wir haben einmal einige davon hier zusammengestellt. Würden Sie uns bitte sagen, inwieweit Sie diesen Aussagen zustimmen oder nicht zustimmen?]
5. Answer Categories

   Strongly disagree [Stimme überhaupt nicht zu]

   Rather disagree [Stimme eher nicht zu]

   Neither agree nor disagree [Teils-teils]

   Rather agree [Stimme eher zu]

   Strongly agree [Stimme voll und ganz zu]

   Can't answer [Kann ich nicht beantworten]

   Prefer not to answer [Möchte ich nicht beantworten]

   • Findings for Answer Categories
   • Recommendations for Answer Categories

---

6. Findings/Recommendations for Multi-Item Scale:

   • Findings for Multi-Item Scale
   • Recommendations for Multi-Item Scale

   1. Findings for Multi-Item Scale: In contrast to the previous questions, two non-substantive answer categories were displayed for question 10 in order to give respondents the (explicit) option of leaving the items unanswered. The proportion of respondents who selected the non-substantive response categories "Can't answer" or “Prefer not to answer" was sometimes considerable (see Table 11): For items a and g, their share was about 28 % of responses, and for item h, it was as high as 39 % of responses. Overall, slightly more than half of the subjects (53 %) gave a non-substantive answer for at least one of the items.
      In addition, there was no item nonresponse for question 10, that is, all eight items were answered by the 240 subjects. The respondents used the full range of the response scale for all items.
      
      Looking at the substantive response categories ("Strongly disagree" to "Strongly agree"), it can be seen that the frequency distributions of items a to d and item f are right skewed, that is, the (large) majority of respondents did not agree (fully/rather) with these statements. For items e and h, about the same number of subjects expressed agreement as disagreement (item e: 33 % vs. 39 %, item h: 19 % vs. 20 %), while item g had more agreement than disagreement (30 % vs. 19 %).
      
      To determine the internal consistency of the 8-item scale, Cronbach's alpha was calculated. This internal consistency was good (cf. George & Mallery, 2002, p. 231), with Cronbach's alpha = 0.86.
      
      The aim of the pretest was to investigate the reason why the subjects selected the non-substantive categories ("Can't answer," "Prefer not to answer"). In addition, the aim was to analyze whether respondents perceived differences between the extreme (items c and h) and moderate items and whether they considered answering these items in surveys to be appropriate. The corresponding cognitive follow-up questions (N1_F10 to N3_F10, see Appendix) were administered to the 116 subjects who were (randomly) assigned to Group 2. Within this group, four subgroups were randomly formed, each of which was to assess two of the eight items in terms of their appropriateness (Group 2a: items a & f, Group 2b: items b & e, Group 2c: items c & g, Group 2d: items d & h).
      
      Reasons for selecting a non-substantive response category
      The vast majority of respondents (83 %) justified their non-substantive responses to the items by saying they had not previously thought about the topic and had too little knowledge to express an informed opinion:
      
      • "I do not feel well enough informed to express an opinion on this." (TP274)
      • "I haven't looked into this issue too much." (TP280)
      Sporadically (n = 4), subjects indicated that they did not want to give reasons for their answers or did not want to disclose their opinions:
      
      • "I don't want to comment on this." (TP311)
      • "I do not want to reveal my personal opinion here." (TP550)
      
      Evaluation of the adequacy of statements in a questionnaire.
      As shown in Table 12, respondents felt that items a, b, and f were rather inappropriate to answer and items e, h, and g were rather appropriate to answer in a survey. Items c and d were in the middle in terms of ratings. Thus, with the exception of item e, those statements in which Jews are named as a group were rated as more inappropriate than those statements in which Israel's treatment of Palestinians is addressed.
      
      When asked about the reasons why the individual items were perceived as "rather/very inappropriate," the test persons explained that items a, b, and f were racist, antisemitic, or inappropriate in a survey, while for items g and h they argued primarily that they lacked the knowledge to answer the items:
      
      • "I don't think it's okay to ask such questions." (TP320, item a)
      • "The statement is antisemitic and provocative." (TP360, item a)
      • "Because this question alone can be discriminatory to some. And it blankets Jews." (TP249, item b)
      • "I find this statement rather inappropriate as it is rather blunt. Certainly, only racists and antisemites will agree with this statement." (TP420, item b)
      • "I find the phrase 'makes me dislike more and more' very inappropriate." (TP293, item f)
      • "It's not appropriate." (TP344, item f)
      • "I haven't dealt with it enough to have a real opinion on it." (TP367, item h)

   1. Recommendations for Multi-Item Scale: If the item battery is to be shortened for the actual survey, we recommend deleting items g and h, since the proportion of non-substantive response is relatively high here and they often capture a (lack of) geopolitical knowledge on the part of the respondents rather than antisemitic attitudes.
      
      Furthermore, it has been shown that “Can’t answer” in particular was chosen comparatively fre-quently and also much more often than “Prefer not to answer”. The display of two non-substantive response categories offers respondents a comparatively easy way to avoid commenting on their atti-tudes towards Jews. Since the majority of the items are attitudinal questions, we recommend at least omitting the “Can’t answer” response for these items in order to reduce the proportion of non-substantive responses (although some respondents will then probably select “Prefer not to answer” instead). The latter answer, however, at least allows for a stronger interpretation in terms of content.

---

1. Cognitive Techniques: Specific Probing

Items