Question Text:
Version 1: What is the probability you will resubmit the paper(s) to a peer-reviewed journal or edited book?
Version 2: How likely is it that you will resubmit the paper(s) to a peer-reviewed journal or edited book?
Instruction:
Version 1: You can enter a percentage between 0 and 100.
Answer Categories:
__ %
Alternative Answer Format Tested:
Very likely
Likely
Rather likely
Rather unlikely
Unlikely
Very unlikely
Recommendations:
We recommend asking Question 19 only to those respondents who indicated in the previous question that they have not yet resubmitted the paper(s) (“No, I have not (yet) resubmitted the paper(s),” see recommendation for Question 18). Since the majority of respondents preferred the second version of the question, we recommend using the closed-ended response scale for this question.
Category-Selection Probing, Difficulty Probing, Specific Probing, General Probing
Findings for Question:
Information on the aim and purpose of the test:
Question 19 measures the likelihood that respondents will resubmit the research paper(s) to a scholarly journal or anthology. This question is also asked only of those respondents who indicated in Question 16 that all or some of the papers related to the study were rejected. The two versions of the question being tested differ in whether the response must be entered as a percentage in an open-ended text field or selected from a closed-ended response scale.
The goal of the test was to determine whether the open-ended response format yields more nuanced answers than the closed-ended format and whether test participants prefer one of the two versions.
Findings:
As Table 17 shows, the respondents’ answers exhibited no or only very little variation across both versions of the question. All ten respondents stated that, after a paper was rejected, they would always submit it to a different journal or anthology.
- “I’ve put a lot of work into this, so I don’t want to let it go to waste. I have a paper right now that’s been sitting around for a while because we still need to make one change. But I’ll definitely submit it.” (TP03, Answer Question 19, Version 1: 100%)
- “It doesn’t do me any good if I don’t submit it somewhere. So I would never let a finished paper sit unused on my hard drive; I would always keep submitting it until it’s accepted somewhere.” (TP04, Answer Question 19, Version 1: 100%)
- ”Well, I think that if I’ve gone to the trouble of writing something, then I had a reason for writing it. And I’d always try again somewhere else and stick with it for the time being.” (TP05, Answer Question 19, Version 1: 90%)
Three participants also pointed out here (as they had in Question 18) that it was inappropriate to phrase the question in the present tense, since it is possible that a paper has already been resubmitted.
- “Yeah, okay. It’s practically already done. I’d have liked to click something like ‘I’ve already done this.’ Then, just to cover my bases, I’d enter 100% here.” (TP06)
- “Here we go again: If you’ve already done that, then I don’t know what to write here. For me, for example, that’s in the past. So the question doesn’t really fit.” (TP10)
The majority of respondents (n = 7) expressed a preference for question version 2, explaining that question version 1 required too detailed an answer.
- “I think the second one here is better because I feel that the answer format for the first question is a bit too detailed for a question that, in my opinion, can’t be answered in such detail. And I also don’t think there’s that much variation there.” (TP04)
- “I think in Version 1, that’s a scale with 100 levels. I’m not sure if that makes sense—I mean, won’t people just end up saying 100, 50, or 70 anyway? Wouldn’t the second scale serve the same purpose? A percentage scale isn’t really necessary for this question.” (TP10)
Question Topic:
Job and career/ Job situation & professional activity
Construct:
Probability of resubmission
