Appendix U — Author interview schedule

Below is a list of topics/questions to be discussed in this study. Following best practice, the qualitative work will remain flexible with respect to participants’ agendas but we will cover the broad topics/questions noted. It is common in qualitative work to iteratively develop topics and questions as new ideas emerge from early data collection. Therefore, we may add new topics as the interviews progress and data collection continues. However, the key topics will remain the same.

Interview 1

Introduction

Thank you for agreeing to participate today. I am interviewing you to get feedback on a website we are building for researchers.

Your participation is completely voluntary and you may decline to answer any question, stop the interview or withdraw from the study at any time and for any reason.

Today’s interview will take around 45 minutes. I’ll then ask you to read something in your own time, as you get on with your normal, day-to-day research work, before inviting you back to a second interview, sometime within the next month, which will also take around 45 minutes. If you complete both interviews and the short task at home, we’ll send you a 50 GBP (Amazon) voucher / $50 dollars.

There are no right or wrong answers to any of my questions. I’m interested in your opinions and experiences, and I’d love it if you spoke your mind freely – the more you say, the more we can improve the website. So don’t hold back, and please be honest. The site you’ll see was made by someone else, so you won’t offend me.

Do you have any questions at this stage?

With your permission, I’d like to record the interview. I’ll record the audio because I don’t want to miss any of your ideas. The audio recordings will be transcribed and anonymised. These anonymous transcripts will be stored as data, but you won’t be identifiable from them.

I’m also going to record the video so that I can remember what you were looking at whilst speaking. The video recordings will remain confidential – only I will see them, and I’ll delete them once I have finished analysing the data.

At times, I’m going to ask you to share your screen as you interact with the website so that I can understand what you are seeing. That screen share will also be recorded in the video, so can I please ask you to close any windows that you do not want to be on that recording.

[wait for participant]

Do you have any questions about what I have just explained?

May I start the recording?

Building rapport and demographic questions

Before we begin it would be nice if you could tell me a little bit about yourself.

What are your research interests?
Where do you work?
How many years have you done research?
Is English your first language?

Can you tell me a little about what you are working on?

What stage are you at?

5 Second Test

I’m going to share my screen now. Hold on a moment….

[Confirm participant can see website]

I’d like you to have a look at this website.

[Wait 5 seconds before stopping sharing].

I only gave you a few seconds there. Can you tell me what you think the website is about?

What do you think reporting guidelines are?
What do you think reporting guidelines can be used for?
How would you expect the website to impact your work?
How would you describe the look and feel of the website?

If the participant has no clue, give them another 10 seconds.

Once the participant understands that the website is about reporting guidelines, ask the following:

What experience do you have with writing-up research?
What aspects of writing do you find most difficult?
What parts of research do you find most difficult?
Have you ever heard of reporting guidelines before, or perhaps names like STROBE, COREQ, SRQR?
Have you ever heard of the EQUATOR Network before?
Have you ever used a reporting guideline?
Have you ever filled out a reporting checklist?
If yes to either of above 2 questions: what did you think of it?

In a moment, I’m going to ask you to explore the website on your own computer, and I’d like you to say everything that is going through your mind as you do. I’d like you to tell me what you notice, what you like, don’t like, how it makes you feel, what you expect to happen if you click on things. Everything. I’ll demonstrate what I mean quickly now [demonstrate think aloud].

[go back to home page]. OK so now you try. I’m going to show you the home page again and I’ll leave it up on the screen for longer this time. Just tell me what you notice, how you feel, and what you expect. Verbalise everything that goes through your head.

[allow participant to warm up into think aloud].

[give feedback]

OK, I’m going to stop sharing my screen now and I’d like you to share yours instead. Remember to close anything that you don’t want recorded.

[wait for screen share]

I’ll paste the link into the chat [paste into chat]. Go ahead and open it.

Thank you.

You’ve already told me how you feel about the top of this page, so I’d like you to go ahead and start to scroll down, whilst continuing to tell me everything that you are thinking as it happens.

User Protocols

Find guidance for qualitative research

OK wonderful. Please scroll back to the top of the page. I’d like you to find a reporting guideline that you think would apply to your work. Please can you tell me which guideline you would click on?

[wait]

Why have you chosen this guideline?

What kinds of research does this guideline apply to?

I’d like you to click on to SRQR guideline for qualitative research.

I’d like you to continue verbalising everything you notice and think on this page too.

[Once they get to the guidance]

Let’s pause there for now. What do you expect from this guideline?

How trustworthy do you find this guideline? Why?

Finding item information

Ok great. Now I know the guidance is quite long and I don’t expect you to read all of it now. But let’s try finding some specific information instead. According to the guidance, what information do other people need to know about the researchers involved in the study?

According to the guideline, why do readers need to know this?

[Consider doing another user protocol test if time permits]

Have a look at the word “approach” at the top of the item (item 6). Why do you think it is underlined?
Have a look at the number 4 at the end of the first sentence. What do you think it is?
Now look at the title of the item - “Researcher Characteristics and reflexivity”. Look at the icon to the right, what do you think that is?

We are nearing the end, so I’d like to talk about our next interview and your little bit of homework. Thinking about your own research, what do you expect to be working on over the next few weeks?

Do you expect to write anything / have you already written anything?

[Tailor to stage of work]

I think you will find items [specify items] are relevant to what you’re planning to work on. Can I ask you read through those items before we next talk? You can read through the rest of the guidance too if you like, but I expect those items to be most relevant to your work.

I’d like you to annotate any parts that you like – for any reason – with a plus sign. And any parts that you don’t like with a “-“. Let’s practice that now. Have a read through [specify] and try highlighting parts that you like or don’t like.

[give feedback].

Would you be happy to send me [portion of text they are working on] before we next speak so that we can talk about it? You can send it by email two days before we talk, so that I have enough time to read and understand it. Again, remember that all information will be kept confidential, and we won’t quote anything that you have written without your permission. Does that sound OK to you?

[Schedule a date for the next interview and an expected deadline for submitting writing sample].

Thank you so much for your time today, and for everything that you shared with me. Feel free to email me with any questions, and I’m looking forward to speaking with you again soon.

Interview 2

Introduction

Hello again. Thank you for joining this second interview. I’m looking forward to hearing how you got on.

As a reminder, there are no right or wrong answers. With your permission I’m going to record today’s session like I did last time. Am I ok to start recording now?

We will look at your plus and minus markings in a moment. Before we do, can you tell me, in your own words, how did you find [describe task that was set]?

Plus minus test

Let’s take a look at your plus and minus markings.

[For each marking] Can you tell me why you put a [plus/minus] here?

Writing sample

Now I’d like to talk about the writing that you sent me.

[For items that were reported well]

Can you tell me how you wrote this section?
Can you tell me how the guidance influenced your work, if at all?
Did you use any other resources? If so, what were they?

[For items not reported fully]

Can you tell me how you wrote this section?
What does this item mean to you? (referring to guidance)
[If they misunderstood] Can you tell me where you have described this item?
[If they understood correctly] The item recommends that you describe how you did [insert]. Why did you decide not to include that?

Overall, how do you feel about what you’ve written?

Do you feel confident that it’s complete?
Do you like it?

Would you continue using the guidance for your work? Why?

Closing thoughts

Before we finish, I’d like to talk again about the website more generally. How would you describe it to a colleague?

Do you have some other thoughts about the website?

Thank you so much for your time. Would you like to see my findings once my research is finished? If you have time, you are welcome to comment on them to make sure I’ve represented your views fairly.

Please feel free to contact me at any time if you have any questions.