1. Do the interviewers have difficulties in noting down the respondents’ answers?
In two cases (TP 04, 08) the interviewers had difficulties in noting down the answers of the test persons. Both test persons first indicated which activities in list 3 they already had difficulties with, without indicating a probability of future limitations. Although this is correct in terms of the intention of the question and the interviewers noted the answer "I already have one or more of these limitations" for both test subjects, it was not clear to the interviewers whether the test subjects had understood the question correctly. They therefore asked the respondents to indicate the likelihood of developing limitations in the other activities listed, and then noted the percentages as an additional answer:
- TP 04: When I am older than now, you mean? Or now at my age? You mean later than now? [...] Well, I actually only have difficulty bathing. I noticed the other day that I don't get out of the bathtub as well as before. Maybe it's because of the weight. I used to be slim, but over the years I've gained weight. But that's the only thing. I don't bathe anymore because I'm afraid I won't be able to get out of the tub. I have no problems with the other things.
- INT: What do you think the probability is that you will ...
- TP 04: Later [...], when I get older, maybe just the bathing. And maybe difficulties later with the map [using a map to find your way around in a foreign environment]. That you might have some difficulties in a foreign environment. I have no problems with shopping: I go out of the house and there is a bakery next door and the supermarket directly opposite.
- INT: What is the probability between 0 and 100 that one day you will develop limitations on one or more of these activities? You mentioned earlier that you already have one of these limitations. But what is the probability in the ageing process ...
- TP 04: In percent? Let's say 20 to 30 percent.
- TP 08: Shall I say the things where ...
- INT: First of all, just read through it so that you can get an overview.
- TP 08: Well, with the "work on the house and garden" it's getting harder and harder because I'm no longer fit enough for such strenuous physical activities. I would still be able to do some laundry, but my wife does that. So, actually only this work on house and garden.
- INT: And if you had to state a probability that you would develop limitations in one or more of these activities?
- TP 08: Beyond this "work on the house and garden"?
- INT: Yes, well, generally in the future, in the ageing process.
- TP 08: Well, you never know what is coming. But maybe 20% to 30%.
In order to avoid such misunderstandings (between interviewer and respondent), it should first be asked whether the respondents already have limitations in one or more of these activities. Only if they answer in the negative they should then be asked about the probability of developing one or more of these restrictions in the future.
2. Are the respondents aware that they are supposed to answer in a general context rather than for each activity listed on the card separately?
None of the test persons gave different answers for the individual activities. Test person 01 merely expressed the wish to be able to give more differentiated answers but had no difficulty in making a general judgement on all the activities mentioned:
"I would rather like to answer this in more detail. I tend to be 'under 20', but I do not have a good sense of direction. I think the probability would be higher [for using a map]. I hope that when I'm old I'm still healthy and clear enough to manage on my own. Let's keep it ‘under 20%’.”
3. Is the question text too long?
Two test persons (TP 01, 09) commented on the length of the question or the amount of information that must be processed:
- "The question may be difficult for some people to answer. Firstly, there was a lot of text to follow. Then you had to focus on the diagrams, you forgot the first part again. I can't remember how many women in 100 can't do it.” (TP 01)
- "[The answer was] rather difficult. One must be careful not to get confused. Also with these diagrams." (TP 09)
In the cognitive interviews, the length of the question had no clear negative influence on the respondents' answering process. However, the interviews gave the impression that both the graphs and the indication to answer without rounding or approximation were ignored by most of the test subjects. One reason for this may be that the text of the question is too long, and respondents therefore only take into account information that they consider to be central and that they can remember.
4. Are the charts comprehensible?
Only two of the ten test persons (TP 05, 09) included the graphs in their answers. Testperson 05, however, found them more disturbing than helpful:
- "One is very much fixated on it. Without them, I think it would have been easier. Because now you have the pictures in front of your eyes and think, 'Hm ..., yes ...'. Then you are a bit distracted again and I would find it easier if you had asked me the question without them.” (TP 05)
- "If I look at the graph, what percentage there is. So I chose a slightly lower value. But of course, that's just an estimate. You can't calculate it." (TP 09)
The remaining eight test persons answered the question independently of the graphs. Four of these test persons commented or criticized the diagrams. Test person 01 found the graphs unnecessary, test person 02 doubted the correctness of the distributions shown, test person 07 criticised the colouring and test person 10 recommended that the percentages of the distributions be included in the graphs:
- "No, [they had] no [influence on my evaluation]. They could have been left out." (TP 01)
- "Seriously, well, I don't believe that. The diagrams here. Men and ageing, they didn't answer honestly. The experience is quite different. The experience is that men degenerate more and faster than women. Either the women were more honest and more likely to admit it or, I think, the men don't admit it. I would have loved to have seen the interviews of the men; I can't imagine that. I think that men certainly don't like to admit such things. Especially when it starts with number 1 [the first activity on the list: 'Getting dressed yourself, including socks and shoes']. Women have given more honest answers." (TP 02)
- "I don't like the colors." (TP 07)
- "You might have added the percentages." (TP 10)
5. Do the respondents have difficulties to understand and/or answer the question?
In addition to the problems already described above (too long question text, ignoring, not understanding or criticizing the graphs), the answers of the three test persons who named the extreme values 0% and 100% (TP 06, 07, 10) indicated that they did not understand the question in the intended way.
The two test persons, who stated a probability of 100% (TP 06, 07), justified their answers by stating that everyone from a certain age on has to expect restrictions in one or more of these activities. Test person 06 therefore questioned the meaningfulness of the question and described it as "nonsense":
- "I cared for my mother for 8 years. I know the path one takes. These complaints more or less all come at some point. It must be said quite clearly. [...] The question is nonsense for me, I'm sorry. You should rather ask something like: 'Imagine you are 60 years old, 70 years old, 80 years old. What could possibly apply to you?' Especially the connection between the list and the diagrams does not make sense [in the case of the period 'one day'].” (TP 06)
- "Of course, as you get older, you have physical and mental limitations. One hopes, of course, that this will always come later. But at 75 years of age, the one or other limitation is definitely there." (TP 07)
Test person 10 expressed a wish rather than the subjective probability of having to reckon with restrictions one day:
"This goes towards 0. This is very optimistic, but you can wish for it. [...] I hope I will be 100 years old and then drop dead, physically still unharmed. That is wishful thinking. But of course, one of these things can happen to you every day, regardless of your age. Some people have strokes at 35 and are handicapped."
Test person 03 criticizes the question as too hypothetical:
"So, I should answer now what is in a few years? But that is a strange question. I can go out of here now and drop dead. I hope that I can do these things for many years to come. But who knows? [...] I'm going to be 76 soon and I think it's going to come to me sometime. But I don't know what's coming or when or how, it's impossible to predict." The figure she quotes (50%) therefore appears to be an estimate between 0% and 100% ('fifty-fifty').
Test person 05 initially interpreted the question as asking her to indicate which of the activities on the list she expected to be restricted in old age. Only after the interviewer had gone over the matter several times did she name a probability:
"If I assume that I will do something when I am that age, I think it will be shopping, because it might be difficult to walk when I think of myself.” [The interviewer points out that a general judgement should be given on all activities]. "At an advanced age, when I see the age indications [on the graph], I would say between 65 and 70%. If I assume myself, I would say 65% in any case. [...] That's me, because I have to do with my feet anyway and I'm a bit limited and it doesn't get better, it gets worse. I think at that age it might be about this percentage that I'm not really mobile anymore."