GESIS Pretest Datenbank

Pretest Database

Project Title:

Project “Complexity and intrinsic motivation in climate protection” (English Version)

Tested Question and Recommendations
Findings
Topic and Construct

General Information: Question: Control Treatment 1 [excl. time dimension]

*Note: The item was tested in German. This is an English translation of the original German wording.*
Introduction
(Findings/Recommendations): The European Union (EU) wants to reduce emissions of climate-damaging greenhouse gases (e.g. CO2) quickly and significantly and achieve climate neutrality by 2050. In order to effectively control the emission of these pollutants, large power plants and industrial facilities must acquire and surrender a so-called emission right for each ton of green-house gas emitted. The quantity of these emission rights issued by the EU is strictly limited. At the beginning of February 2021, an emission right cost around EUR 42 (incl. VAT) per ton of CO2. Through consumption, electricity use, heating and mobility, each German causes an average of eight tons of CO2 per year.
If you buy an emission right and decommission it, it is no longer available to the power plants. The power plants can therefore emit one ton of CO2 less. This effectively reduces overall emissions in the EU and makes an active contribution to climate protection.
[Die Europäische Union (EU) möchte den Ausstoß von klimaschädlichen Treibhausgasen (z. B. CO2) schnell und deutlich reduzieren und bis 2050 klimaneutral sein. Um den Ausstoß dieser Schadstoffe wirksam zu kontrollieren, müssen große Kraftwerke und Industrieanlagen für jede ausgestoßene Tonne Treibhausgas ein sogenanntes Emissionsrecht erwerben und abgeben. Die von der EU ausgegebene Menge dieser Emissionsrechte ist strikt begrenzt. Ein Emissionsrecht kostete Anfang Februar 2021 etwa 42 EUR (inkl. MwSt.) pro Tonne CO2. Durch Konsum, Stromverbrauch, Heizen und Mobilität verursacht jede/r Deutsche im Durchschnitt acht Tonnen CO2 pro Jahr.
Kauft man ein Emissionsrecht und legt es still, so steht es den Kraftwerken nicht mehr zur Verfügung. Die Kraftwerke können somit eine Tonne CO2 weniger ausstoßen. Man senkt damit wirksam die Gesamtemissionen in der EU und leistet einen aktiven Beitrag zum Klimaschutz.]
- Findings for Introduction
- Recommendations for Introduction
Findings for Introduction: In order not to influence the later question, the research design provided for the test persons to first read the introductory page and to continue with the first version of the question without any follow-up questions on the part of the interviewer. After reading, answering and discussing the two versions of the question, the test persons read the introductory page again and were asked follow-up questions about it.

Did the test persons express comprehension problems when reading the introduction?

Two problems became apparent when reading the introductory text. The first problem was that the information from the last paragraph was not processed correctly during the first reading. As a result, the core message that the decommissioning of an emission right means that it is no longer available to power plants, and that in this way an active contribution is made to environmental protection, was not received when the introductory text was read. This problem affected four of the six test persons:

Test person 03 was already occupied with the last paragraph for a longer time during the first reading. She stated that she did not fully understand it, but also said that this would perhaps be clarified by the questions. When answering the first version of the question, she also spontaneously commented that this had been the case. However, when she reread the introductory text after reading and discussing the two versions of the question, she was surprised to find that information she had missed when answering the questions had been included in the introductory text. She asked the interviewer if it was the same text as at the beginning. When he answered in the affirmative, she replied: “Funny, because that's somehow clearer now. It even says in the last sentence that you then make an active contribution to climate protection. That didn't really stick in my mind when I first read it.”

When test person 04 reread the introductory page after answering the questions, she only understood the meaning of the last paragraph and changed her answer to the second version of the question from "decommission in a year’s time" to "decommission right now." “Now I have only really understood this. Now, of course, I'm in favor of an immediate shut-down. After all, that gives me direct access to environmental pollution.”

Test person 05 commented on the first reading that she was "on topic" and understood the text. However, when she read the question text on the next page, she clicked right back, read the last paragraph aloud again, and commented: “So if I take one [emission right] out of circulation, you would have one less ton in the atmosphere. That's what it's about now, all right.”

Test person 06 was also unable to recall the last paragraph of the introduction upon repeated reading: “I mean, the [introduction] is worded differently, [...] especially the last part. 'If you buy an emission right and shut it down, it's no longer available to the power plant.' That wasn't in there earlier, I don't think. I don't remember it. If the text had been there, with the last sentence especially, then the first question would have been easier to answer.”

One reason for overlooking the last paragraph could have its origin in the second problem. This is because some test persons remained unclear about the connection between their role as a private individual and the industry's emission output:

This was evident, for example, in test person 03's first reading of the introductory text: “I don't quite understand the last two sentences. So, I was wondering if this is now in the interest of a company to buy this emission right or if this is in the right of someone else." On rereading, the test person tried to explain the reason for her confusion and make suggestions for improvement: “I think it would help more in understanding if you didn't write how much a German household produces in CO2, but an average company or just a company that comes into question with the emission right. That would help me a lot in understanding. [...] It confuses me that there is this comparison of private households. I was thinking more of companies.”

“I don't know if you can buy emission rights as a private person. I don't know and it doesn't come out through the text.” (TP 06)

Consequently, when reading the first question, one test person expressed uncertainty as to whether she could and should actually decide on this issue (“If I were in the position to decide that now?”, TP 02).

Furthermore, one test person believed that the introductory text was about a current innovation in emissions legislation, which would make emissions rights subject to a charge for the first time:

“Emission right is the permission to blow CO2 into the atmosphere. This is to be charged for. This permission is no longer available for free, as before, but only for a financial contribution. That should have been made clear.” This, in turn, would have the effect of increasing the price for the end consumer, but the text of the agreement underestimates this: “It is not just a matter of the power plants emitting less, but of them producing as much energy as is demanded on the market. This market mechanism of the energy market should be made clear. [...] The addressee of this story is not primarily the power plant industry, but it is about the end consumer having to pay more and thus being allowed to consume less. That must be clear.” (TP 05)

Do the test persons already know the word "emission right" in advance?

Four test persons stated that they had already known the term emissions law prior to the interview (TP 01, 03, 05, 06). Of these, two test persons stated that they could explain both the term emission rights and emissions trading (TP 01, 05). However, only test person 01 easily gave a definition of emission rights or emission trading that correctly defined both terms and related them to each other:

“You can 'buy your way out' of emissions trading. For example, you can emit more pollutants or buy pollutants from other countries. These are all things that I really don't think are great. Simply to emit more pollutants yourself, you can buy your way out, so to speak. This is about emission rights. Well, basically it's also a trade. If I now buy a ton of CO2 for 42 euros, then the industry or the power plant operators can't emit it."

The other test persons gave explanations that showed a partly correct, partly abbreviated or even incorrect understanding of emission rights or emissions trading:

"I don't understand what the power plants and industrial plants are buying. Then these limited rights that are there. And then what do they do? [...] They then buy something, decommission it, and then they get money for it?" (TP 02)

"This is a government initiative for CO2 reduction. The [government wants] to use it to restrict how many tons of CO2 can be produced. You can buy this emission right for a certain contribution, as a company, and then decommission it, and that's just restricting other companies." (TP 03)

"The companies buy the emission right for 42 euros per ton. I consume eight tons, which means they have to spend about 250 euros just for me to be allowed to emit the emissions. [...] The word [emissions trading] itself doesn't come up, but basically it's already about the fact that emissions rights are traded." (TP 04)

“Ultimately, [the text] is about emissions trading. The emission right has to be acquired somewhere, or it has to be paid for. So far, it's been free. But it doesn't say who collects the money, whether the government [puts] it into its tax coffers or whether anything is done with it. But there is a trade there; the emission right is sold.” (TP 05)

“This is a trade that is for environmental policy, to reduce ... So, by that I mean greenhouse gas emissions and the emissions law ... I'm a little bit overwhelmed right now. The emission right is simply the certain amount that is emitted, as far as my general knowledge goes. I don't know why you should shut that down altogether.” (TP 06)

No test persons indicated that the introductory text contained words unfamiliar to them.
Recommendations for Introduction: The introduction should be formulated in such a way that it clearly conveys to the respondents that a) the decommission of an emission right represents a contribution to climate protection, and b) they, as individuals, may subsequently decide on the decommission of an emission right.
The core message of the introduction is currently summarized in the last paragraph of the introduction page. Some test persons did not pay enough attention to this paragraph. Others did not take from the paragraph that they, as a private individual, are given the opportunity to decommission an emission right, which in turn has implications for companies.
We recommend that the core message that respondents can choose between a contribution to climate protection (for the public good) and a sum of money (for themselves) be prominently placed at the beginning of the experiment. This can be done, for example, on a separate survey page. Then, on a second page, the respondents are given an explanation (varied in complexity) of the principle of emissions trading.
The mention of the average CO2 consumption of a private household is not necessary for understanding the question and could be deleted to reduce complexity. The current placement also led respondents to fear a shortage of supply. However, since this information is to be retained to serve as an order of magnitude to better estimate personal consumption, we recommend that the sentence be placed in a way that does not create uncertainty about whether the decommission of an emission right could lead to a shortage of personal energy supply. This information could be placed accordingly as an introduction to the experiment.

An implementation could look as follows:
First page:
“Through consumption, electricity consumption, heating and mobility, each German produces an average of eight tons of CO2 per year.
In this survey, you are asked to decide whether you would like to decommission a so-called emission right and thus do something for climate protection, or whether you would like to receive 5 EUR instead.
By decommissioning an emission right, private individuals can make an active contribution to climate protection.
Before you decide, you will find additional background information on emission rights on the next page.”
[„Durch Konsum, Stromverbrauch, Heizen und Mobilität verursacht jede/r Deutsche im Durchschnitt acht Tonnen CO2 pro Jahr.
Im Rahmen dieser Umfrage werden Sie gebeten, sich zu entscheiden, ob Sie ein sogenanntes Emissionsrecht stilllegen und damit etwas für den Klimaschutz tun möchten oder ob Sie stattdessen 5 EUR erhalten möchten.
Durch das Stilllegen eines Emissionsrechts kann man als Privatperson einen aktiven Beitrag zum Klimaschutz leisten.
Bevor Sie sich entscheiden, erhalten Sie auf der nächsten Seite zusätzliche Hintergrundinformationen zu Emissionsrechten.“]
Second page:
“The European Union (EU) wants to quickly and significantly reduce emissions of climate-damaging greenhouse gases (e.g. CO2) and be climate-neutral by 2050. In order to effectively control the emission of these pollutants, large power plants and industrial facilities must acquire a so-called emission right for each ton of greenhouse gas emitted. The amount of emission rights issued by the EU is strictly limited. At the beginning of February 2021, an emission allowance cost around EUR 42 (incl. VAT) per ton of CO2.
As a private individual, one can buy such an emission right and decommission it. If you do this, it is no longer available to power plants and industrial facilities. The power plants can therefore emit one ton of CO2 less. By decommissioning, you make an active contribution to climate protection.”
[„Die Europäische Union (EU) möchte den Ausstoß von klimaschädlichen Treibhausgasen (z. B. CO2) schnell und deutlich reduzieren und bis 2050 klimaneutral sein. Um den Ausstoß dieser Schadstoffe wirksam zu kontrollieren, müssen große Kraftwerke und Industrieanlagen für jede ausgestoßene Tonne Treibhausgas ein sogenanntes Emissionsrecht erwerben. Die von der EU ausgegebene Menge dieser Emissionsrechte ist strikt begrenzt. Ein Emissionsrecht kostete Anfang Februar 2021 etwa 42 EUR (inkl. MwSt.) pro Tonne CO2.
Man kann als Privatperson ein solches Emissionsrecht kaufen und stilllegen. Wenn man dies macht, steht es den Kraftwerken und Industrieanlagen nicht mehr zur Verfügung. Die Kraftwerke können somit eine Tonne CO2 weniger ausstoßen. Durch die Stilllegung leistet man einen aktiven Beitrag zum Klimaschutz.“]
Question Text: You now have the opportunity to decommission an emission right that entitles you to emit one ton of CO2. The costs for the procurement and decommissioning of the emission right are borne by the client of the study. You can choose to decommission the emission right now or to receive 5 EUR instead.
How do you decide?
[Sie haben jetzt die Gelegenheit ein Emissionsrecht, das zum Ausstoß einer Tonne CO2 berechtigt, stillzulegen. Die Kosten für die Beschaffung und Stilllegung des Emissionsrechts trägt der Auftraggeber der Studie. Sie können wählen, ob Sie das Emissionsrecht jetzt stilllegen oder ob Sie stattdessen 5 EUR erhalten möchten.
Wie entscheiden Sie sich?]
Answer Categories Decommission the emission right now [Das Emissionsrecht jetzt stilllegen]
Receive 5 EUR [5 EUR erhalten]

don’t know/not specified [weiß nicht/keine Angabe]
- Findings for Answer Categories
- Recommendations for Answer Categories
1. Recommendations:
  Question: Some test persons felt the need to read the explanations again while reading the question text, but only one test person went back to the introduction page to do so. Therefore, we recommend including a clearly visible information button on the question page that can be used to display the explanations.
  In addition, we recommend formulating the question text in such a way that both the choice faced by the respondents and the significance of the individual answer options become clear. This could be done, for example, as follows:
  
  “Today, we offer to purchase an emission right for you, which entitles you to emit one ton of CO2. The costs for the procurement of the emission right are borne by us as the client of the study. You can choose whether you want to decommission the emission right now or whether you want to receive 5 EUR instead.
  If you decide to decommission the emission right right now, the industry will have to emit one ton of CO2 less from now on than before.
  If you choose to receive 5 EUR, it will be credited to your account via [survey company].”
  [„Wir bieten Ihnen heute an, für Sie ein Emissionsrecht, das zum Ausstoß einer Tonne CO2 berechtigt, zu kaufen. Die Kosten für die Beschaffung des Emissionsrechts tragen wir als Auftraggeber der Studie. Sie können wählen, ob Sie das Emissionsrecht jetzt stilllegen oder ob Sie stattdessen 5 EUR erhalten möchten.
  Wenn Sie sich dafür entscheiden, das Emissionsrecht jetzt stillzulegen, muss die Industrie ab sofort eine Tonne CO2 weniger ausstoßen als vorher.
  Wenn Sie sich dafür entscheiden, 5 EUR zu erhalten, werden Ihnen diese über [Umfrageunternehmen] auf Ihrem Konto gutgeschrieben.“]
  
  Answer format: The answer choices should be explained in the question text or in an instruction below, as described above, to prevent confusion about the combination of answer options.
  A good half of the respondents chose the answer "don't know/not specified" in the pretest, although this was for different reasons. In order to be able to distinguish between different triggers for refusing to answer in the survey, we recommend asking respondents who choose "don't know/not specified" a question. A combination of closed answer options and the open text field facilitates the evaluation of such a demand and offers respondents who would like to comment on the content of the topic or their decision the opportunity to do so.
  
  This could look like this:
  “You answered the previous question with ‘don't know/not specified’. Why did you choose this answer?
  - I did not understand the question or the explanations about emission right
  - I did not want to select any of the other answer choices
  - I am not interested in the subject
  - Other (Please explain in the text field below)
  - [open text field]”
  [Sie haben die vorangegangene Frage mit ‚weiß nicht/keine Angabe‘ beantwortet. Wieso haben Sie sich für diese Antwort entschieden?
  - Ich habe die Frage bzw. die Erläuterungen zu Emissionsrecht nicht verstanden
  - Ich wollte keine der anderen Antwortmöglichkeiten auswählen
  - Ich interessiere mich nicht für das Thema
  - Anderes (Bitte im untenstehenden Textfeld erläutern)
  - [offenes Textfeld]“]
  The list of reasons can be extended as needed.

Cognitive Techniques: Introductory text: General Probing, Difficulty Probing, Paraphrasing, Specific Probing
Question: General Probing, Comprehension Probing, Category Selection Probing, Difficulty Probing, Specific Probing
Findings for Question:
After reading the introductory text for the first time, the test persons first saw and answered the first version of the question (Control Treatment 1), followed by follow-up questions from the interviewer. Subsequently, they were shown the second version of the question (Control Treatment 2) and answered follow-up questions about it.

Control Treatment 1: For what reasons do test persons choose their response option?

In the first version of the question, half of the test persons chose to decommission the emission right now, and the other half chose "don't know/not specified". With the exception of one test person who stated that she did not understand the question (TP 06), the test persons did not seem to have any problems with the question text.

The three test persons who chose to decommission the emissions rights justified their responses by saying they wanted to do the right thing for the environment:
- “I think it's great if less CO2 is emitted. I can't do anything with the 5 euros. I'd rather have one ton less CO2.“ (TP 01)
- “Getting the 5 euros would be in my self-interest, and based on this introductory text, it just sounds good to shut down this emission right so that less CO2 is produced.” (TP 03)
- “Because the overall environmental impact is declining. Something has to be done.” (TP 04)
However, one of these test persons was confused by the answer options and had to reassure the interviewer that she had understood the question correctly before committing to an answer:
- “I have not yet understood the system, I must say. So, from the introductory text and the question, it's not clear to me what it's all about now. [...] Does that mean that the power plant can no longer emit a ton of emissions or that I receive 5 euros?” (TP 01)
Control Treatment 1: Why do test persons choose the "don't know/not specified" category?

Similar uncertainty was evident among the three test persons who opted for "don't know/not specified". The test persons were unsure or even irritated by the combination of answer options. They seemed to have difficulty reconciling the alternatives of decommission the emission right or receiving a monetary reward for not decommission it:
- “I don't understand [that] now." [reads question aloud again] "[...] Whether I would rather shut it down or get 5 euros? [...] To undo the emission right, so that it doesn't come into force. Or to get 5 euros. I think that's kind of funny and stupid. [...] That makes no sense to me. Why should I get 5 euros now?” (TP 02)
- “So, I have the option to shut something down and the cost is borne by the client. I'm not sure what the 5 euros is supposed to do. That's why I'm a little confused there.” (TP 06)
The third test persons expressed anger at the choice of making a seemingly environmentally friendly decision (which she did not think was the right one in terms of climate protection) or a selfish, environmentally damaging one:
- “I wouldn't want to pollute the atmosphere even for the Judas wage of 5 euros, but I'm not in favor of simply buying an emission right and thinking that now you have a good conscience. Therefore, I would not want to make a statement, because that is just too extensive to explain my position. Although that doesn't express my opinion, I just don't want to be committed to the simple alternative.” (TP 05)
She wished there was an additional open response field where she could write in her opinion, as this was not possible with the given response options.

Control Treatment 1: Does the introductory text provide enough information to answer the question?

Three test persons stated that they would have liked more detailed information, but only one of them referred to the explanations on emission rights in the introductory text (TP 01). The other two would have liked the implications of the answer options to be explained in more detail, i.e., what the consequences of deciding to decommission the emission right would be or why they should receive five euros for not decommission it (TP 02, 06). These test persons both chose to answer the question with "don't know/not specified." Another indication that these two test persons had not taken the relevant information from the introductory text and the question text was that both answered the question as to who would bear the costs for the emission right that this was the respondent.

Control Treatment 1: How easy or difficult was it for the test persons to answer the question?

All three test persons who had decided to have the emission right decommissioned stated that the question had been "very easy" for them because it was a decision in the interest of climate protection:
- "After all, there are two ways to choose. Either I take the 5 euros or I emit a ton of CO2 less. There's not that much to think about." (TP 01)
- “I think people are brought up a bit like that these days. It's just an important topic now, and I think there's a kind of 'awareness' that's being built up in the media. That's why it was an easy decision for me.” (TP 03)
- “If you look a little bit at the environmental issues and you know how much CO2 is emitted every day, this answer ['decommission right now'] is obvious.” (TP 04)
The three test persons who answered the question with "don't know/not specified", on the other hand, stated that they found the answer "rather easy" (TP 05) or "rather difficult" (TP 02, 06). Unknown or unexplained terms did not play a role here, but rather the content of the question or the combination of answer options:
- “This decision, whether I want to decommission the emission right now or get 5 euros for it, [is difficult for me]. I find it difficult to decide whether I, as an individual, should buy a share and decommission it.” (TP 02)
- “The pure verbal form of the question is simple. What lies behind it, in implications, that's the real problem.” (TP 05)
Do test persons find it easier to understand Control Treatment 1 or 2?

Three test persons found Control Treatment 2, including the time dimension, easier to answer than the first version. One test person reasoned that the additional option of giving industry another year reinforced her decision to have the emission right decommissioned immediately ("It's definitely easy, but I think [the second version] is better because you can make more decisions. If I had a year now to decommission the [emission right] and give industry more opportunities to emit CO2, I think the answer of decommissioning the [emission right] immediately is better." , TP 01). However, another test person found the second version easier only because she had not read the first version completely (TP 06).

Two test persons found Control Treatment 1, i.e., without the time dimension, easier to answer because they had to think more in Control Treatment 2 (TP 04) or because they were confused by the additional answer option whose meaning was not explained (TP 03). One test person found both versions equally good (TP 05).

Question Topic: Environment/ Climate protection
Construct: Decommissioning an emission right: exclusive time dimension