In contrast to the previous questions, two non-substantive answer categories were displayed for question 10 in order to give respondents the (explicit) option of leaving the items unanswered. The proportion of respondents who selected the non-substantive response categories "Can't answer" or “Prefer not to answer" was sometimes considerable (see Table 11): For items a and g, their share was about 28 % of responses, and for item h, it was as high as 39 % of responses. Overall, slightly more than half of the subjects (53 %) gave a non-substantive answer for at least one of the items.
In addition, there was no item nonresponse for question 10, that is, all eight items were answered by the 240 subjects. The respondents used the full range of the response scale for all items.
Looking at the substantive response categories ("Strongly disagree" to "Strongly agree"), it can be seen that the frequency distributions of items a to d and item f are right skewed, that is, the (large) majority of respondents did not agree (fully/rather) with these statements. For items e and h, about the same number of subjects expressed agreement as disagreement (item e: 33 % vs. 39 %, item h: 19 % vs. 20 %), while item g had more agreement than disagreement (30 % vs. 19 %).
To determine the internal consistency of the 8-item scale, Cronbach's alpha was calculated. This internal consistency was good (cf. George & Mallery, 2002, p. 231), with Cronbach's alpha = 0.86.
The aim of the pretest was to investigate the reason why the subjects selected the non-substantive categories ("Can't answer," "Prefer not to answer"). In addition, the aim was to analyze whether respondents perceived differences between the extreme (items c and h) and moderate items and whether they considered answering these items in surveys to be appropriate. The corresponding cognitive follow-up questions (N1_F10 to N3_F10, see Appendix) were administered to the 116 subjects who were (randomly) assigned to Group 2. Within this group, four subgroups were randomly formed, each of which was to assess two of the eight items in terms of their appropriateness (Group 2a: items a & f, Group 2b: items b & e, Group 2c: items c & g, Group 2d: items d & h).
Reasons for selecting a non-substantive response category
The vast majority of respondents (83 %) justified their non-substantive responses to the items by saying they had not previously thought about the topic and had too little knowledge to express an informed opinion:
- "I do not feel well enough informed to express an opinion on this." (TP274)
- "I haven't looked into this issue too much." (TP280)
Sporadically (
n = 4), subjects indicated that they did not want to give reasons for their answers or did not want to disclose their opinions:
- "I don't want to comment on this." (TP311)
- "I do not want to reveal my personal opinion here." (TP550)
Evaluation of the adequacy of statements in a questionnaire.
As shown in Table 12, respondents felt that items a, b, and f were rather inappropriate to answer and items e, h, and g were rather appropriate to answer in a survey. Items c and d were in the middle in terms of ratings. Thus, with the exception of item e, those statements in which Jews are named as a group were rated as more inappropriate than those statements in which Israel's treatment of Palestinians is addressed.
When asked about the reasons why the individual items were perceived as "rather/very inappropriate," the test persons explained that items a, b, and f were racist, antisemitic, or inappropriate in a survey, while for items g and h they argued primarily that they lacked the knowledge to answer the items:
- "I don't think it's okay to ask such questions." (TP320, item a)
- "The statement is antisemitic and provocative." (TP360, item a)
- "Because this question alone can be discriminatory to some. And it blankets Jews." (TP249, item b)
- "I find this statement rather inappropriate as it is rather blunt. Certainly, only racists and antisemites will agree with this statement." (TP420, item b)
- "I find the phrase 'makes me dislike more and more' very inappropriate." (TP293, item f)
- "It's not appropriate." (TP344, item f)
- "I haven't dealt with it enough to have a real opinion on it." (TP367, item h)
If the item battery is to be shortened for the actual survey, we recommend deleting items g and h, since the proportion of non-substantive response is relatively high here and they often capture a (lack of) geopolitical knowledge on the part of the respondents rather than antisemitic attitudes.
Furthermore, it has been shown that “Can’t answer” in particular was chosen comparatively fre-quently and also much more often than “Prefer not to answer”. The display of two non-substantive response categories offers respondents a comparatively easy way to avoid commenting on their atti-tudes towards Jews. Since the majority of the items are attitudinal questions, we recommend at least omitting the “Can’t answer” response for these items in order to reduce the proportion of non-substantive responses (although some respondents will then probably select “Prefer not to answer” instead). The latter answer, however, at least allows for a stronger interpretation in terms of content.
