to Pretest Database Pretest Database
Project Title:Project “Complexity and intrinsic motivation in climate protection” (English Version)
  1. General Information: Question: Control Treatment 1 [excl. time dimension]

    *Note: The item was tested in German. This is an English translation of the original German wording.*
  2. Introduction
    (Findings/Recommendations): The European Union (EU) wants to reduce emissions of climate-damaging greenhouse gases (e.g. CO2) quickly and significantly and achieve climate neutrality by 2050. In order to effectively control the emission of these pollutants, large power plants and industrial facilities must acquire and surrender a so-called emission right for each ton of green-house gas emitted. The quantity of these emission rights issued by the EU is strictly limited. At the beginning of February 2021, an emission right cost around EUR 42 (incl. VAT) per ton of CO2. Through consumption, electricity use, heating and mobility, each German causes an average of eight tons of CO2 per year.
    If you buy an emission right and decommission it, it is no longer available to the power plants. The power plants can therefore emit one ton of CO2 less. This effectively reduces overall emissions in the EU and makes an active contribution to climate protection.
    [Die Europäische Union (EU) möchte den Ausstoß von klimaschädlichen Treibhausgasen (z. B. CO2) schnell und deutlich reduzieren und bis 2050 klimaneutral sein. Um den Ausstoß dieser Schadstoffe wirksam zu kontrollieren, müssen große Kraftwerke und Industrieanlagen für jede ausgestoßene Tonne Treibhausgas ein sogenanntes Emissionsrecht erwerben und abgeben. Die von der EU ausgegebene Menge dieser Emissionsrechte ist strikt begrenzt. Ein Emissionsrecht kostete Anfang Februar 2021 etwa 42 EUR (inkl. MwSt.) pro Tonne CO2. Durch Konsum, Stromverbrauch, Heizen und Mobilität verursacht jede/r Deutsche im Durchschnitt acht Tonnen CO2 pro Jahr.
    Kauft man ein Emissionsrecht und legt es still, so steht es den Kraftwerken nicht mehr zur Verfügung. Die Kraftwerke können somit eine Tonne CO2 weniger ausstoßen. Man senkt damit wirksam die Gesamtemissionen in der EU und leistet einen aktiven Beitrag zum Klimaschutz.]
  3. Question Text: You now have the opportunity to decommission an emission right that entitles you to emit one ton of CO2. The costs for the procurement and decommissioning of the emission right are borne by the client of the study. You can choose to decommission the emission right now or to receive 5 EUR instead.
    How do you decide?
    [Sie haben jetzt die Gelegenheit ein Emissionsrecht, das zum Ausstoß einer Tonne CO2 berechtigt, stillzulegen. Die Kosten für die Beschaffung und Stilllegung des Emissionsrechts trägt der Auftraggeber der Studie. Sie können wählen, ob Sie das Emissionsrecht jetzt stilllegen oder ob Sie stattdessen 5 EUR erhalten möchten.
    Wie entscheiden Sie sich?]
  4. Answer Categories Decommission the emission right now [Das Emissionsrecht jetzt stilllegen]

    Receive 5 EUR [5 EUR erhalten]

    don’t know/not specified [weiß nicht/keine Angabe]

    1. Recommendations: Question: Some test persons felt the need to read the explanations again while reading the question text, but only one test person went back to the introduction page to do so. Therefore, we recommend including a clearly visible information button on the question page that can be used to display the explanations.
      In addition, we recommend formulating the question text in such a way that both the choice faced by the respondents and the significance of the individual answer options become clear. This could be done, for example, as follows:

      “Today, we offer to purchase an emission right for you, which entitles you to emit one ton of CO2. The costs for the procurement of the emission right are borne by us as the client of the study. You can choose whether you want to decommission the emission right now or whether you want to receive 5 EUR instead.
      If you decide to decommission the emission right right now, the industry will have to emit one ton of CO2 less from now on than before.
      If you choose to receive 5 EUR, it will be credited to your account via [survey company].”
      [„Wir bieten Ihnen heute an, für Sie ein Emissionsrecht, das zum Ausstoß einer Tonne CO2 berechtigt, zu kaufen. Die Kosten für die Beschaffung des Emissionsrechts tragen wir als Auftraggeber der Studie. Sie können wählen, ob Sie das Emissionsrecht jetzt stilllegen oder ob Sie stattdessen 5 EUR erhalten möchten.
      Wenn Sie sich dafür entscheiden, das Emissionsrecht jetzt stillzulegen, muss die Industrie ab sofort eine Tonne CO2 weniger ausstoßen als vorher.
      Wenn Sie sich dafür entscheiden, 5 EUR zu erhalten, werden Ihnen diese über [Umfrageunternehmen] auf Ihrem Konto gutgeschrieben.“]

      Answer format: The answer choices should be explained in the question text or in an instruction below, as described above, to prevent confusion about the combination of answer options.
      A good half of the respondents chose the answer "don't know/not specified" in the pretest, although this was for different reasons. In order to be able to distinguish between different triggers for refusing to answer in the survey, we recommend asking respondents who choose "don't know/not specified" a question. A combination of closed answer options and the open text field facilitates the evaluation of such a demand and offers respondents who would like to comment on the content of the topic or their decision the opportunity to do so.

      This could look like this:
      “You answered the previous question with ‘don't know/not specified’. Why did you choose this answer?
      • I did not understand the question or the explanations about emission right
      • I did not want to select any of the other answer choices
      • I am not interested in the subject
      • Other (Please explain in the text field below)
      • [open text field]”
      [Sie haben die vorangegangene Frage mit ‚weiß nicht/keine Angabe‘ beantwortet. Wieso haben Sie sich für diese Antwort entschieden?
      • Ich habe die Frage bzw. die Erläuterungen zu Emissionsrecht nicht verstanden
      • Ich wollte keine der anderen Antwortmöglichkeiten auswählen
      • Ich interessiere mich nicht für das Thema
      • Anderes (Bitte im untenstehenden Textfeld erläutern)
      • [offenes Textfeld]“]
      The list of reasons can be extended as needed.
  1. Cognitive Techniques:Information image/link to cognitive pretesting Introductory text: General Probing, Difficulty Probing, Paraphrasing, Specific Probing
    Question: General Probing, Comprehension Probing, Category Selection Probing, Difficulty Probing, Specific Probing
  2. Findings for Question: After reading the introductory text for the first time, the test persons first saw and answered the first version of the question (Control Treatment 1), followed by follow-up questions from the interviewer. Subsequently, they were shown the second version of the question (Control Treatment 2) and answered follow-up questions about it.

    Control Treatment 1: For what reasons do test persons choose their response option?

    In the first version of the question, half of the test persons chose to decommission the emission right now, and the other half chose "don't know/not specified". With the exception of one test person who stated that she did not understand the question (TP 06), the test persons did not seem to have any problems with the question text.

    The three test persons who chose to decommission the emissions rights justified their responses by saying they wanted to do the right thing for the environment:
    • “I think it's great if less CO2 is emitted. I can't do anything with the 5 euros. I'd rather have one ton less CO2.“ (TP 01)
    • “Getting the 5 euros would be in my self-interest, and based on this introductory text, it just sounds good to shut down this emission right so that less CO2 is produced.” (TP 03)
    • “Because the overall environmental impact is declining. Something has to be done.” (TP 04)
    However, one of these test persons was confused by the answer options and had to reassure the interviewer that she had understood the question correctly before committing to an answer:
    • “I have not yet understood the system, I must say. So, from the introductory text and the question, it's not clear to me what it's all about now. [...] Does that mean that the power plant can no longer emit a ton of emissions or that I receive 5 euros?” (TP 01)
    Control Treatment 1: Why do test persons choose the "don't know/not specified" category?

    Similar uncertainty was evident among the three test persons who opted for "don't know/not specified". The test persons were unsure or even irritated by the combination of answer options. They seemed to have difficulty reconciling the alternatives of decommission the emission right or receiving a monetary reward for not decommission it:
    • “I don't understand [that] now." [reads question aloud again] "[...] Whether I would rather shut it down or get 5 euros? [...] To undo the emission right, so that it doesn't come into force. Or to get 5 euros. I think that's kind of funny and stupid. [...] That makes no sense to me. Why should I get 5 euros now?” (TP 02)
    • “So, I have the option to shut something down and the cost is borne by the client. I'm not sure what the 5 euros is supposed to do. That's why I'm a little confused there.” (TP 06)
    The third test persons expressed anger at the choice of making a seemingly environmentally friendly decision (which she did not think was the right one in terms of climate protection) or a selfish, environmentally damaging one:
    • “I wouldn't want to pollute the atmosphere even for the Judas wage of 5 euros, but I'm not in favor of simply buying an emission right and thinking that now you have a good conscience. Therefore, I would not want to make a statement, because that is just too extensive to explain my position. Although that doesn't express my opinion, I just don't want to be committed to the simple alternative.” (TP 05)
    She wished there was an additional open response field where she could write in her opinion, as this was not possible with the given response options.

    Control Treatment 1: Does the introductory text provide enough information to answer the question?

    Three test persons stated that they would have liked more detailed information, but only one of them referred to the explanations on emission rights in the introductory text (TP 01). The other two would have liked the implications of the answer options to be explained in more detail, i.e., what the consequences of deciding to decommission the emission right would be or why they should receive five euros for not decommission it (TP 02, 06). These test persons both chose to answer the question with "don't know/not specified." Another indication that these two test persons had not taken the relevant information from the introductory text and the question text was that both answered the question as to who would bear the costs for the emission right that this was the respondent.

    Control Treatment 1: How easy or difficult was it for the test persons to answer the question?

    All three test persons who had decided to have the emission right decommissioned stated that the question had been "very easy" for them because it was a decision in the interest of climate protection:
    • "After all, there are two ways to choose. Either I take the 5 euros or I emit a ton of CO2 less. There's not that much to think about." (TP 01)
    • “I think people are brought up a bit like that these days. It's just an important topic now, and I think there's a kind of 'awareness' that's being built up in the media. That's why it was an easy decision for me.” (TP 03)
    • “If you look a little bit at the environmental issues and you know how much CO2 is emitted every day, this answer ['decommission right now'] is obvious.” (TP 04)
    The three test persons who answered the question with "don't know/not specified", on the other hand, stated that they found the answer "rather easy" (TP 05) or "rather difficult" (TP 02, 06). Unknown or unexplained terms did not play a role here, but rather the content of the question or the combination of answer options:
    • “This decision, whether I want to decommission the emission right now or get 5 euros for it, [is difficult for me]. I find it difficult to decide whether I, as an individual, should buy a share and decommission it.” (TP 02)
    • “The pure verbal form of the question is simple. What lies behind it, in implications, that's the real problem.” (TP 05)
    Do test persons find it easier to understand Control Treatment 1 or 2?

    Three test persons found Control Treatment 2, including the time dimension, easier to answer than the first version. One test person reasoned that the additional option of giving industry another year reinforced her decision to have the emission right decommissioned immediately ("It's definitely easy, but I think [the second version] is better because you can make more decisions. If I had a year now to decommission the [emission right] and give industry more opportunities to emit CO2, I think the answer of decommissioning the [emission right] immediately is better." , TP 01). However, another test person found the second version easier only because she had not read the first version completely (TP 06).

    Two test persons found Control Treatment 1, i.e., without the time dimension, easier to answer because they had to think more in Control Treatment 2 (TP 04) or because they were confused by the additional answer option whose meaning was not explained (TP 03). One test person found both versions equally good (TP 05).
  1. Question Topic: Environment/ Climate protection
  2. Construct: Decommissioning an emission right: exclusive time dimension