to Pretest Database Pretest Database
Project Title:German Longitudinal Election Study (GLES) 2021 (English Version)
  2. General Information: *Note: The item was tested in German. This is an English translation of the original German wording.
  3. Question Text: Version a. When you use social media, how often do you encounter information about political topics?
    [Wenn Sie soziale Medien nutzen, wie häufig begegnen Ihnen da Informationen über politische Themen?]

    Version b. When you think of the information you see on social media, how often is there information about political topics? Please estimate the share in percentage.
    [Wenn Sie an alle Informationen denken, die Sie in sozialen Medien sehen, wie häufig sind da-runter Informationen über politische Themen? Bitte schätzen Sie den Anteil in Prozent.]
  5. Answer Categories Version a:

    always [immer]

    frequently [oft ]

    occasionally [gelegentlich]

    rarely [selten]

    never [nie]

    Don't know [weiß nicht]

    Version b: open question

    1. Recommendations: Question: Since question variant 5a was rated as easier to answer overall, we recommend using this variant. In addition, there were indications in question variant 5b that at least one test person did not answer it according to the question intention. However, it should be kept in mind that both variants aim at different findings. If the evaluative dimension of the perception of the frequency with which respondents encounter political content is of interest, question 5a should be used (relative frequency question). If, on the other hand, group comparisons are in the foreground (as, for example, in the question of whether people with a strong political interest encounter political content in social media more frequently than people with less political interest), we recommend asking for absolute frequencies as in question 5b.

      Response format: No changes recommended.
  2. Cognitive Techniques:Information image/link to cognitive pretesting Difficulty Probing, Comprehension Probing, Specific Probing
  3. Findings for Question: Questions 5a and 5b aim to assess how politicized the content is that respondents encounter on social media. A closed and an open question variant were tested.

    1. Which question variant is rated better by respondents?
    The test persons first answered question 5a and were then asked how easy or difficult it was for them to answer the question. All test persons stated that they found it "very easy" (n = 3) or "rather easy" (n = 7).
    After answering question 5b, persons were asked to compare the two questions and indicate which was easier for them to answer. Five persons preferred question 5a, two persons preferred question 5b, and three persons indicated they liked both equally. Supporters of question 5a argued that it was easier to give a relative frequency than an exact percentage:
    • "[Question 5b is more difficult] because it asks for the exact percentage. If you have a few an-swers there that you can choose [like in question 5a], I think it's easier." (TP 01)
    • "I always find it difficult in general to give exact, precise information. [...] I have now indicated 70%. I couldn't distinguish now whether it's 72 or 75 or 68%. That's why I find it a bit harder than answering verbal linguistic things." (TP 03)
    Proponents of question 5b argued the opposite, i.e., that it was easier to state an absolute frequency than a relative frequency:
    • "The wording is much better." (TP 02)
    • "The number thing is easier for me. When I reflect on how much I use that, why I use that, since I tend to think in terms of numbers anyway. "(TP 10)
    If one compares the answers of the test persons to both questions (see table above), it is noticeable that the relative frequency data from question 5a cannot be transferred into absolute frequencies in question 5b. While "always" from test person 09 corresponds to a frequency of 20 %, "always" from test person 06 means a frequency of 95 %. And also, the understanding of "often" ranges from 50 % (TP 02, 08) over 70 % (TP 03) up to 85 % of the information (TP 04). It can be assumed that subject 09 indicated the estimated proportion of political content within a social network in question 5b; this would explain her discrepant response behavior. In addition, it must be considered that relative frequencies always include a valuation of the respondents. If this evaluative dimension of the perception of the frequency with which respondents encounter political content is of interest, question 5a should be used. If, on the other hand, group comparisons are in the foreground (as, for example, in the case of the question of whether people with a strong political interest encounter political content in social media more frequently than people with less political interest), it is recommended that absolute frequencies be asked, as in question 5b.

    2. Does the understanding of "information about political issues" vary?
    The test persons had a very homogeneous understanding of "information about political topics" and understood it to mean articles, statements and news with political content. Information or opinions on current topics such as the corona virus or the U.S. presidential election were frequently cited as examples:
    • "I interpreted that for me as any news or statement that has any political implication for anything. If a friend writes me, [my city] is now risk area, then for me it's also [a] political information or political issue, because it's restrictions from the country, so it's political decisions." (TP 04)
    • "Statements of individual private persons who have an opinion on a certain matter that is discussed in public. And also articles, for example." (TP 05)
    • "Political issues are issues that are current in the world, globally, what interests us all or moves us all. It's not something that's private. For example, if Donald Trump is elected, I personally can't do anything about it. Those are political issues for me. Or with the coronavirus, all the measures on how to behave. Those are political issues for me." (TP 08)
  2. Question Topic: Digitalization/ Use of digital systems
  3. Construct: Proportion of political content when using social media (closed vs. open)