GESIS Pretest Datenbank

Pretest Database

Project Title:

Project “Complexity and intrinsic motivation in climate protection” (English Version)

Tested Question and Recommendations
Findings
Topic and Construct

General Information: Question: Control Treatment 2 [incl. time dimension]

*Note: The item was tested in German. This is an English translation of the original German wording.*
Introduction
(Findings/Recommendations): The European Union (EU) wants to reduce emissions of climate-damaging greenhouse gases (e.g. CO2) quickly and significantly and achieve climate neutrality by 2050. In order to effectively control the emission of these pollutants, large power plants and industrial facilities must acquire and surrender a so-called emission right for each ton of green-house gas emitted. The quantity of these emission rights issued by the EU is strictly limited. At the beginning of February 2021, an emission right cost around EUR 42 (incl. VAT) per ton of CO2. Through consumption, electricity use, heating and mobility, each German causes an average of eight tons of CO2 per year.
If you buy an emission right and decommission it, it is no longer available to the power plants. The power plants can therefore emit one ton of CO2 less. This effectively reduces overall emissions in the EU and makes an active contribution to climate protection.
[Die Europäische Union (EU) möchte den Ausstoß von klimaschädlichen Treibhausgasen (z. B. CO2) schnell und deutlich reduzieren und bis 2050 klimaneutral sein. Um den Ausstoß dieser Schadstoffe wirksam zu kontrollieren, müssen große Kraftwerke und Industrieanlagen für jede ausgestoßene Tonne Treibhausgas ein sogenanntes Emissionsrecht erwerben und abgeben. Die von der EU ausgegebene Menge dieser Emissionsrechte ist strikt begrenzt. Ein Emissionsrecht kostete Anfang Februar 2021 etwa 42 EUR (inkl. MwSt.) pro Tonne CO2. Durch Konsum, Stromverbrauch, Heizen und Mobilität verursacht jede/r Deutsche im Durchschnitt acht Tonnen CO2 pro Jahr.
Kauft man ein Emissionsrecht und legt es still, so steht es den Kraftwerken nicht mehr zur Verfügung. Die Kraftwerke können somit eine Tonne CO2 weniger ausstoßen. Man senkt damit wirksam die Gesamtemissionen in der EU und leistet einen aktiven Beitrag zum Klimaschutz.]
- Findings for Introduction
- Recommendations for Introduction
Findings for Introduction: In order not to influence the later question, the research design provided for the test persons to first read the introductory page and to continue with the first version of the question without any follow-up questions on the part of the interviewer. After reading, answering and discussing the two versions of the question, the test persons read the introductory page again and were asked follow-up questions about it.

Did the test persons express comprehension problems when reading the introduction?

Two problems became apparent when reading the introductory text. The first problem was that the information from the last paragraph was not processed correctly during the first reading. As a result, the core message that the decommissioning of an emission right means that it is no longer available to power plants, and that in this way an active contribution is made to environmental protection, was not received when the introductory text was read. This problem affected four of the six test persons:

Test person 03 was already occupied with the last paragraph for a longer time during the first reading. She stated that she did not fully understand it, but also said that this would perhaps be clarified by the questions. When answering the first version of the question, she also spontaneously commented that this had been the case. However, when she reread the introductory text after reading and discussing the two versions of the question, she was surprised to find that information she had missed when answering the questions had been included in the introductory text. She asked the interviewer if it was the same text as at the beginning. When he answered in the affirmative, she replied: “Funny, because that's somehow clearer now. It even says in the last sentence that you then make an active contribution to climate protection. That didn't really stick in my mind when I first read it.”

When test person 04 reread the introductory page after answering the questions, she only understood the meaning of the last paragraph and changed her answer to the second version of the question from "decommission in a year’s time" to "decommission right now." “Now I have only really understood this. Now, of course, I'm in favor of an immediate shut-down. After all, that gives me direct access to environmental pollution.”

Test person 05 commented on the first reading that she was "on topic" and understood the text. However, when she read the question text on the next page, she clicked right back, read the last paragraph aloud again, and commented: “So if I take one [emission right] out of circulation, you would have one less ton in the atmosphere. That's what it's about now, all right.”

Test person 06 was also unable to recall the last paragraph of the introduction upon repeated reading: “I mean, the [introduction] is worded differently, [...] especially the last part. 'If you buy an emission right and shut it down, it's no longer available to the power plant.' That wasn't in there earlier, I don't think. I don't remember it. If the text had been there, with the last sentence especially, then the first question would have been easier to answer.”

One reason for overlooking the last paragraph could have its origin in the second problem. This is because some test persons remained unclear about the connection between their role as a private individual and the industry's emission output:

This was evident, for example, in test person 03's first reading of the introductory text: “I don't quite understand the last two sentences. So, I was wondering if this is now in the interest of a company to buy this emission right or if this is in the right of someone else." On rereading, the test person tried to explain the reason for her confusion and make suggestions for improvement: “I think it would help more in understanding if you didn't write how much a German household produces in CO2, but an average company or just a company that comes into question with the emission right. That would help me a lot in understanding. [...] It confuses me that there is this comparison of private households. I was thinking more of companies.”

“I don't know if you can buy emission rights as a private person. I don't know and it doesn't come out through the text.” (TP 06)

Consequently, when reading the first question, one test person expressed uncertainty as to whether she could and should actually decide on this issue (“If I were in the position to decide that now?”, TP 02).

Furthermore, one test person believed that the introductory text was about a current innovation in emissions legislation, which would make emissions rights subject to a charge for the first time:

“Emission right is the permission to blow CO2 into the atmosphere. This is to be charged for. This permission is no longer available for free, as before, but only for a financial contribution. That should have been made clear.” This, in turn, would have the effect of increasing the price for the end consumer, but the text of the agreement underestimates this: “It is not just a matter of the power plants emitting less, but of them producing as much energy as is demanded on the market. This market mechanism of the energy market should be made clear. [...] The addressee of this story is not primarily the power plant industry, but it is about the end consumer having to pay more and thus being allowed to consume less. That must be clear.” (TP 05)

Do the test persons already know the word "emission right" in advance?

Four test persons stated that they had already known the term emissions law prior to the interview (TP 01, 03, 05, 06). Of these, two test persons stated that they could explain both the term emission rights and emissions trading (TP 01, 05). However, only test person 01 easily gave a definition of emission rights or emission trading that correctly defined both terms and related them to each other:

“You can 'buy your way out' of emissions trading. For example, you can emit more pollutants or buy pollutants from other countries. These are all things that I really don't think are great. Simply to emit more pollutants yourself, you can buy your way out, so to speak. This is about emission rights. Well, basically it's also a trade. If I now buy a ton of CO2 for 42 euros, then the industry or the power plant operators can't emit it."

The other test persons gave explanations that showed a partly correct, partly abbreviated or even incorrect understanding of emission rights or emissions trading:

"I don't understand what the power plants and industrial plants are buying. Then these limited rights that are there. And then what do they do? [...] They then buy something, decommission it, and then they get money for it?" (TP 02)

"This is a government initiative for CO2 reduction. The [government wants] to use it to restrict how many tons of CO2 can be produced. You can buy this emission right for a certain contribution, as a company, and then decommission it, and that's just restricting other companies." (TP 03)

"The companies buy the emission right for 42 euros per ton. I consume eight tons, which means they have to spend about 250 euros just for me to be allowed to emit the emissions. [...] The word [emissions trading] itself doesn't come up, but basically it's already about the fact that emissions rights are traded." (TP 04)

“Ultimately, [the text] is about emissions trading. The emission right has to be acquired somewhere, or it has to be paid for. So far, it's been free. But it doesn't say who collects the money, whether the government [puts] it into its tax coffers or whether anything is done with it. But there is a trade there; the emission right is sold.” (TP 05)

“This is a trade that is for environmental policy, to reduce ... So, by that I mean greenhouse gas emissions and the emissions law ... I'm a little bit overwhelmed right now. The emission right is simply the certain amount that is emitted, as far as my general knowledge goes. I don't know why you should shut that down altogether.” (TP 06)

No test persons indicated that the introductory text contained words unfamiliar to them.
Recommendations for Introduction: The introduction should be formulated in such a way that it clearly conveys to the respondents that a) the decommission of an emission right represents a contribution to climate protection, and b) they, as individuals, may subsequently decide on the decommission of an emission right.
The core message of the introduction is currently summarized in the last paragraph of the introduction page. Some test persons did not pay enough attention to this paragraph. Others did not take from the paragraph that they, as a private individual, are given the opportunity to decommission an emission right, which in turn has implications for companies.
We recommend that the core message that respondents can choose between a contribution to climate protection (for the public good) and a sum of money (for themselves) be prominently placed at the beginning of the experiment. This can be done, for example, on a separate survey page. Then, on a second page, the respondents are given an explanation (varied in complexity) of the principle of emissions trading.
The mention of the average CO2 consumption of a private household is not necessary for understanding the question and could be deleted to reduce complexity. The current placement also led respondents to fear a shortage of supply. However, since this information is to be retained to serve as an order of magnitude to better estimate personal consumption, we recommend that the sentence be placed in a way that does not create uncertainty about whether the decommission of an emission right could lead to a shortage of personal energy supply. This information could be placed accordingly as an introduction to the experiment.

An implementation could look as follows:
First page:
“Through consumption, electricity consumption, heating and mobility, each German produces an average of eight tons of CO2 per year.
In this survey, you are asked to decide whether you would like to decommission a so-called emission right and thus do something for climate protection, or whether you would like to receive 5 EUR instead.
By decommissioning an emission right, private individuals can make an active contribution to climate protection.
Before you decide, you will find additional background information on emission rights on the next page.”
[„Durch Konsum, Stromverbrauch, Heizen und Mobilität verursacht jede/r Deutsche im Durchschnitt acht Tonnen CO2 pro Jahr.
Im Rahmen dieser Umfrage werden Sie gebeten, sich zu entscheiden, ob Sie ein sogenanntes Emissionsrecht stilllegen und damit etwas für den Klimaschutz tun möchten oder ob Sie stattdessen 5 EUR erhalten möchten.
Durch das Stilllegen eines Emissionsrechts kann man als Privatperson einen aktiven Beitrag zum Klimaschutz leisten.
Bevor Sie sich entscheiden, erhalten Sie auf der nächsten Seite zusätzliche Hintergrundinformationen zu Emissionsrechten.“]
Second page:
“The European Union (EU) wants to quickly and significantly reduce emissions of climate-damaging greenhouse gases (e.g. CO2) and be climate-neutral by 2050. In order to effectively control the emission of these pollutants, large power plants and industrial facilities must acquire a so-called emission right for each ton of greenhouse gas emitted. The amount of emission rights issued by the EU is strictly limited. At the beginning of February 2021, an emission allowance cost around EUR 42 (incl. VAT) per ton of CO2.
As a private individual, one can buy such an emission right and decommission it. If you do this, it is no longer available to power plants and industrial facilities. The power plants can therefore emit one ton of CO2 less. By decommissioning, you make an active contribution to climate protection.”
[„Die Europäische Union (EU) möchte den Ausstoß von klimaschädlichen Treibhausgasen (z. B. CO2) schnell und deutlich reduzieren und bis 2050 klimaneutral sein. Um den Ausstoß dieser Schadstoffe wirksam zu kontrollieren, müssen große Kraftwerke und Industrieanlagen für jede ausgestoßene Tonne Treibhausgas ein sogenanntes Emissionsrecht erwerben. Die von der EU ausgegebene Menge dieser Emissionsrechte ist strikt begrenzt. Ein Emissionsrecht kostete Anfang Februar 2021 etwa 42 EUR (inkl. MwSt.) pro Tonne CO2.
Man kann als Privatperson ein solches Emissionsrecht kaufen und stilllegen. Wenn man dies macht, steht es den Kraftwerken und Industrieanlagen nicht mehr zur Verfügung. Die Kraftwerke können somit eine Tonne CO2 weniger ausstoßen. Durch die Stilllegung leistet man einen aktiven Beitrag zum Klimaschutz.“]
Question Text: You now have the opportunity to decommission an emission right that entitles you to emit one ton of CO2. The costs for the procurement and decommissioning of the emission right are borne by the client of the study. You can choose to decommission the emission right now or in one year's time, or you can choose to receive 5 EUR instead.
How do you decide?
[Sie haben jetzt die Gelegenheit ein Emissionsrecht, das zum Ausstoß einer Tonne CO2 berechtigt, stillzulegen. Die Kosten für die Beschaffung und Stilllegung des Emissionsrechts trägt der Auftraggeber der Studie. Sie können wählen, ob Sie das Emissionsrecht jetzt oder erst in einem Jahr stilllegen oder ob Sie stattdessen 5 EUR erhalten möchten.
Wie entscheiden Sie sich?]
Answer Categories Decommission the emission right now [Das Emissionsrecht jetzt stilllegen]
Decommission the emission right in one year’s time [Das Emissionsrecht in einem Jahr stilllegen]

Receive 5 EUR [5 EUR erhalten]

don’t know/not specified [weiß nicht/keine Angabe]
- Findings for Answer Categories
- Recommendations for Answer Categories
1. Recommendations:
  Question: Some test persons felt the need to read the explanations again while reading the question text, but only one test person went back to the introduction page to do so. Therefore, we recommend including a clearly visible information button on the question page that can be used to display the explanations.
  
  “Today, we offer to purchase an emission right for you, which entitles you to emit one ton of CO2. The costs for the procurement of the emission right are borne by us as the client of the study. You can choose whether you want to decommission the emission right now or whether you want to receive 5 EUR instead.
  If you decide to decommission the emission right right now, the industry will have to emit one ton of CO2 less from now on than before.
  If you choose to receive 5 EUR, it will be credited to your account via [survey company].”
  [„Wir bieten Ihnen heute an, für Sie ein Emissionsrecht, das zum Ausstoß einer Tonne CO2 berechtigt, zu kaufen. Die Kosten für die Beschaffung des Emissionsrechts tragen wir als Auftraggeber der Studie. Sie können wählen, ob Sie das Emissionsrecht jetzt stilllegen oder ob Sie stattdessen 5 EUR erhalten möchten.
  Wenn Sie sich dafür entscheiden, das Emissionsrecht jetzt stillzulegen, muss die Industrie ab sofort eine Tonne CO2 weniger ausstoßen als vorher.
  Wenn Sie sich dafür entscheiden, 5 EUR zu erhalten, werden Ihnen diese über [Umfrageunternehmen] auf Ihrem Konto gutgeschrieben.“]
  
  Answer format: The answer choices should be explained in the question text or in an instruction below, as described above, to prevent confusion about the combination of answer options.
  A good half of the respondents chose the answer "don't know/not specified" in the pretest, although this was for different reasons. In order to be able to distinguish between different triggers for refusing to answer in the survey, we recommend asking respondents who choose "don't know/not specified" a question. A combination of closed answer options and the open text field facilitates the evaluation of such a demand and offers respondents who would like to comment on the content of the topic or their decision the opportunity to do so.
  
  This could look like this:
  “You answered the previous question with ‘don't know/not specified’. Why did you choose this answer?
  - I did not understand the question or the explanations about emission right
  - I did not want to select any of the other answer choices
  - I am not interested in the subject
  - Other (Please explain in the text field below)
  - [open text field]”
  [Sie haben die vorangegangene Frage mit ‚weiß nicht/keine Angabe‘ beantwortet. Wieso haben Sie sich für diese Antwort entschieden?
  - Ich habe die Frage bzw. die Erläuterungen zu Emissionsrecht nicht verstanden
  - Ich wollte keine der anderen Antwortmöglichkeiten auswählen
  - Ich interessiere mich nicht für das Thema
  - Anderes (Bitte im untenstehenden Textfeld erläutern)
  - [offenes Textfeld]“]
  The list of reasons can be extended as needed.

Cognitive Techniques: Introductory text: General Probing, Difficulty Probing, Paraphrasing, Specific Probing
Question: General Probing, Comprehension Probing, Category Selection Probing, Difficulty Probing, Specific Probing
Findings for Question:
After reading the introductory text for the first time, the test persons first saw and answered the first version of the question (Control Treatment 1), followed by follow-up questions from the interviewer. Subsequently, they were shown the second version of the question (Control Treatment 2) and answered follow-up questions about it.

Control Treatment 2: Do test persons answer the second version of the question differently than the first, and if so, why?

Three of the six test persons stuck to their previous answer. Two of them decided in both question versions to decommission the emission right immediately (TP 01, 03). One test person stated that it would be of no benefit to the environment to wait a year before decommissioning and would therefore like to continue to decommission the emission right immediately (TP 01). The second test person also stuck to her answer of decommissioning the emission right immediately, but was unsettled by the additional answer option:
- “I'm still taking the 'decommission the emission right now' option, but the decision is not so easy anymore because it's not quite clear to me what the impact will be if I decommission this in a year's time.” (TP 03)
The third test person answered both question versions with “don't know/not specified” (TP 05), as she had been dissatisfied with the available answer options in both question versions.

The other three persons answered the second version of the question differently from the first and changed their answer to the additional answer option "Decommission the emission right in one year’s time". Of these, two test persons had answered "don't know/not specified" in the first version (TP 02, 06) and one had answered "decommission the emission right now" (TP 04). In two cases, the test persons justified this by stating that they were more comfortable with the lead time in order to obtain information (TP 02) or to prepare the decommissioning of the emission right (TP 04). In the third case, the test person did not seem to have read the first version of the question correctly (TP 06):
- “I would then take the one in a year's time, so that in the meantime you can get more information. What is meant by that, what that means.” (TP 02).
- “This whole process requires a certain amount of preparation time. The purchaser who buys the emission right must also first look at how he will then proceed.” (TP 04)
- “The question was easier to answer now. I don't know if the question was asked differently, but now it says that the emission right to emit one ton of CO2 is entitled to decommissioning. I'm not sure if it said that before, because that helped me a little bit to choose my answer.” (TP 06)
Control Treatment 2: How do test persons understand the additional answer option "De-commission the emission right in a year’s time" in the second version of the question?

Most test persons speculated that the additional response option was intended as a time delay, e.g., to inform themselves, or for industry and policymakers to prepare for decommissioning. They imagined that decommission the emission right one year later would be easier to implement, or that during this year it could be renegotiated whether the emission right should be decommissioned at all:
- “So that you can then check again what it means, what is being decommissioned, what the industry is doing. All the things that are hanging on the back of it. It all takes time, you can't even decide that now.” (TP 02)
- “The most I can think of is that then the companies have time to adjust, maybe if it's not one ton, but several tons, as in real life. In the real world, of course, it's better to decommission it immediately, but I [can't judge] how realistic that is.” (TP 03)
- “From my point of view, you simply need preparations to tackle other ways so that there is no gap in supply. The question is, what happens afterwards if I decommission it immediately? So I would say, after a year, there is a possibility that the whole thing will go down reasonably and alternatives will be there.” (TP 04)
- “That this right may be negotiated again and certain things, basic framework conditions are dis-cussed again. That's why I opted for 'decommission in a year’s time'. [...] There is an emissions trading agreement in the EU, as far as I remember correctly, where they discuss everything. [...] That certain things are discussed again, for example: ‘Hey, maybe only in one year and then half. The factories have a certain emission output and therefore only in a year's time.’” (TP 06)
After reading the second version of the question, the test persons were asked whether they thought it would be better for the environment if the emission right was decommissioned immediately or only in a year's time. This question confirmed that none of the test persons had any knowledge of the fact that it could be advantageous for climate protection reasons to decommission the emission right only one year later, regardless of whether the test persons assessed their knowledge about climate protection in general as "very good" (TP 04), "rather good" (TP 01, 05), "average" (TP 03, 06) or "rather poor" (TP 02).

Five test persons indicated that it would clearly be better to decommission the emission right immediately. One test person did not want to commit herself because she was unsure whether immediate decommissioning might not result in a supply gap and whether decommissioning the emission right with a lead time might be more reasonable (TP 04). After reading the introductory page a second time, the test person also came to the conclusion that immediate decommissioning was in the interest of climate protection.

Do test persons find it easier to understand Control Treatment 1 or 2?

Three test persons found Control Treatment 2, including the time dimension, easier to answer than the first version. One test person reasoned that the additional option of giving industry another year reinforced her decision to have the emission right decommissioned immediately ("It's definitely easy, but I think [the second version] is better because you can make more decisions. If I had a year now to decommission the [emission right] and give industry more opportunities to emit CO2, I think the answer of decommissioning the [emission right] immediately is better." , TP 01). However, another test person found the second version easier only because she had not read the first version completely (TP 06).

Two test persons found Control Treatment 1, i.e., without the time dimension, easier to answer because they had to think more in Control Treatment 2 (TP 04) or because they were confused by the additional answer option whose meaning was not explained (TP 03). One test person found both versions equally good (TP 05).

Question Topic: Environment/ Climate protection
Construct: Decommissioning an emission right: inclusive time dimension