Project „PubBias“ – Questions on assessing the prevalence and extent of publication bias (English Version)

Question Text:

Answer Categories:

Alternative Answer Format Tested:

Recommendations:

As with Question 12 , we recommend using a combination of the two question versions or response scales and removing the term “main” from the question. In addition, we recommend adding a residual category that allows respondents to indicate that they had no expectations (e.g., due to an exploratory study design):  

Did you find support for your expectations? 
Yes, for all expectations 
Yes, for most expectations 
Yes, but only for some expectations 
No, for none of my/our expectations 
I/We did not have any expectations (e.g., explorative study design)

Cognitive Techniques:

Findings for Question:

Information on the aim and purpose of the test: 
Question 13 assesses whether the results of the data analysis confirmed the most important expectations. As with the previous question, the two versions of the question tested differ in the response scale used.  
The purpose of the test was to determine which of the two versions works better or is preferred by the test persons. 

Findings: 
As with the previous question, the test persons rated themselves at the upper end of the response scale for both versions and explained that the results of the data analysis had (largely) confirmed their expectations. 

• “Because our main assumptions have been confirmed.” (TP01, Answer Question 13, Version 1: Definitely yes)  
• “‘Mostly yes,’ because while some correlations were not statistically significant, that was actually what we expected based on the literature.” (TP10, Answer Question 13, Version 1: Mostly yes)  
• “‘Somewhat yes’ here, too. Since we did find support, ‘No’ would simply be wrong, and I think ‘Mostly’ is too strong now, because it was almost 50:50. So I don’t think ‘Mostly’ is quite right.” (TP09, Answer Question 13, Version 1: Somewhat yes) 

Similarly to Question 12, the majority of respondents (n = 9) expressed a preference for Question Version 2, although it was also noted here that the first version might yield more nuanced results. 

• “I would definitely prefer the second version. I find the ‘Somewhat yes’ and ‘Somewhat no’ in the first version too vague and too close together.” (TP06, Answer Question 13, Version 1: Somewhat yes; Answer Question 13, Version 2: Yes, partially) 
• “Well, the second one is much simpler, but the first one, of course, allows researchers to make a clearer distinction.” (TP07, Answer Question 13, Version 1: Mostly yes; Answer Question 13, Version 2: Yes, partially)

Question Topic:

Construct: