GESIS Pretest Datenbank

Pretest Database

Project Title:

FReDA-W2b - Questions on the topics of women's fertility, affective touching behavior, sexual orientation, and transnational families (English Version)

Tested Question and Recommendations
Findings
Topic and Construct

Question Text: In your opinion, what is the probability that a woman aged 30 to 35 will become pregnant if she has unprotected sexual intercourse for one year?
Instruction: Please give a percentage value from 0-100.
Answer Categories
(Findings/Recommendations):
Version 1
0 percent

1 to 19 percent

20 to 39 percent

40 to 59 percent

60 to 79 percent

80 to 100 percent

Version 2
0 to 9 percent

10 to 29 percent

30 to 49 percent

50 to 69 percent

70 to 89 percent

90 to 100 percent

Version 3
open answer
- Findings for Answer Categories
- Recommendations for Answer Categories
Findings for Answer Categories: Experimental variation and cognitive follow-up questions:

As part of the cognitive pretest, in addition to the question version developed by FReDA (version 1), two other variations were tested that differed in the design of the response categories (version 2) or contained no response categories (version 3).

The draft question developed by the principals (version 1) used closed response categories and contained a "0 percent" response category and five additional ascending categories in increments of 20 or 21. Responses that were correct according to the client were either at the low end of a response category (Question 1: approximately 80 percent, category: "80 to 100 percent"; Question 2: approximately 40 percent, category: "40 to 59 percent") or at the high end of a response category (Question 3: approximately 15 percent, category: "1 to 19 percent").

Question version 2 used an alternative closed-response format that did not include a "0 percent" category and in which the correct answer was in the middle of an indicated probability range in each case (Question 1: "70 to 89 percent"; Question 2: "30 to 49 percent"; Question 3: "10 to 19 percent").

Question version 3 used an open numeric field in which numbers between 0 and 100 could be entered. An open format avoids respondents' answers being influenced by the predetermined answer categories.

Findings:

Detailed results on the experimental variation of the response format are presented on pages 12 to 21 in the results report (see downloads).

In summary, respondents generally overestimated the likelihood that a woman aged 40 to 45 would become pregnant after infertility treatments. In addition, it appeared that respondents were strongly oriented toward the answer categories, presumably because they did not know the correct answer. In version 1 with the few response categories, respondents tended to choose the middle categories. Version 2 with the higher number of response categories resulted in a multimodal distribution, and Version 3 had respondents overestimating the probability the most. In addition, respondents differed in whether they thought the probability of a woman aged 40 to 45 becoming pregnant was higher in the general formulation of question 2 or after fertility treatments (question 3). This difference in response behavior could indicate that the knowledge question is too difficult to answer, or that the question has other cognitive hurdles. For example, in question 3, some respondents might refer to the population of all women aged 40 to 45, while others might refer only to those who did not become pregnant naturally.
Recommendations for Answer Categories: Although answering the open question (question version 3) was perceived as more difficult than answering the closed question versions, this did not lead to item nonresponse in the cognitive pretest. Since an open question offers more analysis possibilities due to the metric scaling, we recommend asking the questions openly. If other questionnaire-related considerations speak against an open question, we recommend the closed question of version 2, since here the correct answer does not lie at the border between two answer options.
1. Recommendations: The results of the cognitive pretest show that the subject matter of the questions is complex and not all respondents have the appropriate knowledge to answer the questions. Nevertheless, respondents were able to select (versions 1 and 2) or fill in (version 3) an answer and justify it in a meaningful way. The questions can therefore be left in their current form.

Cognitive Techniques: Comprehension Probing, Category Selection Probing, Difficulty Probing
Findings for Question:
Response explanation:
Cognitive follow-up questions were used to examine how respondents proceeded in answering the questions and how they justified their answers. The test persons’ explanations were quite similar across all three questions and did not differ between the three question versions.

About half of the respondents justified their answers exclusively on the basis of the age of the woman mentioned in the question (30 to 35 years), in which the probability of pregnancy was (still) quite high:
- "Fertility is still very high at that age." (TP133, response to version 3: 95 percent)
- "What would be wrong with it at that age? The woman is still young." (TP202, response to version 2: 70 to 89 percent)
- "I believe that most 30- to 35-year-old women are fertile and very likely to become pregnant." (TP230, response to version 1: 80 to 100 percent).
In some cases, the respondents referred exclusively to the period of unprotected sexual intercourse mentioned in the question or to both the age of the woman and the period mentioned:
- "I have read that within one year, in any case, the majority of women become pregnant. Only after one year do we talk about unfulfilled desire to have children." (TP121, response to version 2: 50 to 69 percent)
- "A year is a long time, so it has to happen at some point." (TP150, response to version 1: 60 to 79 percent)
- "Relatively young age and relatively long period of time. So I assumed that the probability is quite high." (TP126, response to version 1: 60 to 79 percent)
- "Women that age can get pregnant quickly if they have unprotected sex for a year." (TP238, response to version 2: 90 to 100 percent)
Just under 12% of respondents (n = 9) expressed irritation with the time period mentioned in the question or pointed out that not only the time period of sexual intercourse had an impact on the likelihood of becoming pregnant, but also (especially) the frequency and timing of sexual intercourse. In some cases, the answers of these respondents gave the impression that they would have chosen or given higher values without reference to a time period:
- "Depends on how often the woman has sex that year. The question is not well formulated." (TP185, response to version 3: 50 percent)
- "You have to catch the 3 fertile days." (TP377, response to version 2: 0 to 9 percent)
- "Of course, it also depends on when in the month the woman has sexual intercourse. It could be that she never has sexual intercourse at ovulation, in which case I think it would be less likely. That's why I took a concrete mean value." (TP672, response to version 2: 30 to 49 percent)
- "I think it depends on the right timing and frequency, coupled with age-related deteriorated receptivity." (TP718, response to version 1: 60 to 79 percent)
Perceived difficulty of questions
When asked how easy or difficult it was for test persons to answer Question 1, about a quarter (n = 21) responded that they found it "somewhat" or "very" difficult. Most of these respondents (n = 15) had received question version 3 (open-ended question). In their explanations to the follow-up question as to why they had found it difficult to answer the question, the majority of the respondents stated that they had not yet dealt with the topic and therefore found it difficult to assess the likelihood of pregnancy:
- "My answer is just a guess. I lack facts and knowledge about this." (TP317)
- "This is a topic I do not deal with." (TP395)
- "Because I haven't dealt with it that intensively yet." (TP718)
Sporadically, it was noted that it was difficult to make a judgment for all women in the age group because the likelihood of pregnancy was very individual and depended on several factors:
- "Because I don't think you can give a general answer. Every woman is individual." (TP443)
- "Because there are way too many parameters that influence that." (TP653)

Question Topic: Society & social affairs/ Family & parenting
Construct: Estimated probabilities of pregnancies