Appendix I — BCW Step 4 – Consolidated list of influences needing to change for the target behaviour to occur
The following list of influences was generated from my thematic synthesis, review of survey questions, and my evaluation of EQUATOR’s website. I have labelled each influence with ideas to address them, generated in my workshops with reporting guideline experts, and my focus groups with publishers, guideline developers, and advocates.
I.1 Researchers may not know what reporting guidelines are
Researchers may have never heard the term “reporting guideline” or may misunderstand it. Researchers may more commonly use terms like “writing” or “writing up” and the word “reporting” may get interpreted as a formal task (such as reporting progress to a funder). The word “guideline” may be interpreted by some as rules (as per journal “author guidelines”) and others as recommendations. Some researchers may perceive reporting guidelines as a set of design requirements, especially if they only use checklists, which typically lack the instructions and nuances included in the full guidance.
Ideas to address this influence:
I.2 Researchers may not know what reporting guidelines exist
Researchers may not be aware of which reporting guidelines exist. Most guidelines on the EQUATOR site are hardly ever accessed
Ideas to address this influence:
I.3 Researchers may not know whether a reporting guideline applies to them
If the scope of a reporting guideline is undefined or unclear, then researchers won’t know whether the guidance applies to them. Researchers may not understand study designs, making it difficult for them to identify which guidance applies.
Ideas to address this influence:
I.4 Researchers may not know what reporting guideline is their best fit
Researchers may not know when more specific guidance exists. An author’s “perfect fit” guideline may not exist, in which case they may not know know when to stop searching, and they may try to use an “imperfect fit” guideline without understanding which items are applicable.
Ideas to address this influence:
I.5 Researchers may not know what resources exist for a reporting guideline
Resources include the guidance itself, checklists, E&E files, templates, and web tools (e.g., PRISMA flow chart maker). Not all resources exist for each reporting guideline and researchers may be unaware of the ones that do. Many researchers may only use the checklist. Sometimes this is purposeful, but other times it may be because researchers don’t know that full guidance and examples exist.
Ideas to address this influence:
I.6 Researchers may not know when reporting guidelines should be used
Researchers may not know when they should use reporting guidelines in their research workflow. Guideline developers may want researchers to use guidance as early as possible, but this is may not be obvious to researchers who may only ever receive instruction to complete a checklist as part of journal submission and may never discover the full guidance. Consequently, researchers may assume that reporting guidelines are supposed to be used by single authors as pre-submission checklists to demonstrate adherence. It may not occur to them that reporting guidelines can be used earlier, or by teams. Some researchers, having come to this realisation themselves, report wanting to be told to use reporting guidelines earlier in their research.
Ideas to address this influence:
I.7 Researchers may misunderstand
Researchers may not understand concepts, terms or words within the guidance, or they may understand them differently to how the developers intended. Some items (or entire guidelines) might be new concepts. e.g., SQUIRE guidelines written at a time where Quality Improvement was still a new concept to many people, and some items (e.g., Context, Study of the intervention) were less familiar than others. Researchers may have nowhere to turn for help should they not understand something.
Ideas to address this influence:
I.8 Researchers may not know what benefits to expect
Researchers may not know what benefits to expect from using a reporting guideline. These benefits may include:
- improved completeness of reporting which helps readers use research and reduces research waste.
- improved flow and less “waffle” in writing
- facilitated discussions between collaborators, especially at the design or protocol stage
- publishing and passing peer review more efficiently
- increased publisher acceptance rates
- efficient, confident writing
- increased impact of manuscript, as the article is easier to search for and information within the article is easier to find.
Ideas to address this influence:
I.9 Researchers may not know why items are important
Researchers may not know why an item is important, or who it is important to.
Ideas to address this influence:
I.10 Researchers may not know how to do an item
Researchers might not know how to do something (e.g., a sample size calculation)
Ideas to address this influence:
I.11 Researchers may not know how to report an item in practice
Researchers may not understand how to report a particular item in practice
Ideas to address this influence:
I.12 Researchers may not know what to write when they cannot report an item
Researchers may not know how to report an item that they did not do (deliberately or as an oversight), or an item that they are unable to report for external reasons (e.g., IP, or data was missing from primary studies).
Ideas to address this influence:
I.13 Researchers have limited time
Guidelines take time to find, read, understand, and apply. Sometimes they may require time and work from multiple co-authors. Researchers & guideline developers may underestimate the time required for writing, and time is often most limited at the point of submission as grant funding may have run out.
Checklists take time to complete, and completing them with page numbers or pasted content can be annoying if future edits necessitate updating the checklist too. Checklists also generate work for editors and peer-reviewers who must cross check page numbers or pasted content with manuscript content.
Ideas to address this influence:
I.14 Researchers may not encounter reporting guidelines early enough to act on them
Some reporting guideline items require work that has to be done within a certain time windows such as:
- during planning or designing
- before or during data collection
- when other colleagues are available
- during the duration of a grant
Ideas to address this influence:
I.15 Researchers may not understand the language
Researchers may not understand the language guidance is written in. A lot of research comes from countries where English is not the first language, as do a lot of EQUATOR website visitors. Even if a researcher speaks English as a second language, language may be an additional barrier.
Ideas to address this influence:
I.16 Researchers may struggle to keep writing concise
Following a guideline can result in lengthy, bloated reports which are unpleasant to read and breach journals’ word limits. Researchers may not know how to keep writing fluid and concise or where they can report an item (e.g., what section, in the text or in a table or figure, in the manuscript or in supplementary material).
Ideas to address this influence:
I.17 Researchers may not have tools for the job at hand
Researchers use reporting guidelines for different tasks and want tools to make that job easier. Researchers report using reporting guidelines for:
- Planning research
- Designing research
- Researchers report wanting items presented in the order in which decisions need to be made
- Researchers report wanting links to resources
- Whilst collecting data
- Researchers report wanting items ordered in the order they are done
- Resaerchers report wanting items embedded into data collection tools
- Drafting manuscript
- Researchers report wanting templates
- Checking manuscripts
- Demonstrating compliance
- Researchers report wanting checklists embedded into submission workflows
- Reviewing the reporting of other people’s manuscripts
- Appraising the quality of other people’s manuscripts
Ideas to address this influence:
I.18 Reporting guidelines can become outdated
Guidelines can become out of date compared to other guidance or compared to current research standards.
Ideas to address this influence:
I.19 Researchers may struggle to reconcile multiple sets of guidance
Researchers must adhere to journal guidelines, multiple reporting guidelines (e.g., PRISMA + PRISMA-Abstracts + PRISMA-S) and other best practice guidelines (like NIH principles). Using multiple guidelines increases complexity and costs, and guidelines can contradict each other.
Ideas to address this influence:
I.20 Researchers may be asked to remove reporting guideline content
Researchers may be asked to remove guideline content by co-researchers, editors or reviewers.
Ideas to address this influence:
I.21 Researchers may forget to use reporting guidelines at earlier research stages
Having been told to complete a checklist upon journal submission, researchers may forget to use a reporting guideline earlier next time.
NB forgetting is different to not realising that reporting guidelines can be used early.
Ideas to address this influence:
I.22 Guidance may be difficult to find
Researcher should be able to easily find guidance and resources that they believe to exist. However:
- search functions can be hard to find or use,
- researchers may not know which search terms to use,
- websites may be hard to navigate,
- guidance can be buried within articles,
- resources may not be optimised for search engines,
- and resources may not be in the same place.
Ideas to address this influence:
I.23 Reporting guidelines may be difficult to access
Researchers may be unable to access guidance published in subscription journals. Journal websites can feature broken links.
Ideas to address this influence:
I.24 Reporting guideline resources may not be in usable formats
Resources differ in how easy or readily usable they are. For example, some checklists are published as PDF tables that cannot be filled or copied. Some guidance can be dense, unstructured text that is hard to digest or navigate; whereas some researchers will read the guidance sequentially, others may dip in and out whilst writing, and unstructured text can make information harder to find.
Ideas to address this influence:
I.25 Researchers may feel afraid to report transparently
Researchers may feel afraid or uncertain when trying to report something that they didn’t (or couldn’t) do.
Ideas to address this influence:
I.26 Researchers may feel restricted if reporting guidelines prescribe design
Advice or assumptions about design choices narrow the scope of the guidance and can make checklists appear prescriptive. Sometimes design assumptions can be implicit. For example, in requiring authors to report the method used to assess risk of bias, PRISMA is implying that authors should have designed their review to assess risk of bias.
Ideas to address this influence:
I.27 Researchers may feel patronized
Researchers can feel patronized by checklists.
Ideas to address this influence:
I.28 Researchers may not believe stated benefits
Researchers may not believe that using a reporting guideline will affect their acceptance rate or publication speed, that using a reporting guideline will help them write, or improve the quality of their manuscript.
Ideas to address this influence:
I.29 Researchers may not care about the benefits of using a reporting guideline
Researchers may understand that reporting guidelines aim to reduce poor reporting, but may not feel that poor reporting matters. Instead of hypothetical benefits or benefits to others, researchers report caring more about personal, immediate benefits like feeling confident, efficiency, and job performance.
Ideas to address this influence:
I.30 Researchers may expect the costs to outweigh benefits
Researchers may feel that the costs of using a reporting guideline - the time and work required and the added manuscript length - outweigh the benefits.
Ideas to address this influence:
I.31 Researchers may feel that checking reporting is someone else’s job.
Researchers report feeling that completing a reporting checklist should be the job of the editor or peer reviewer, not the author. Editors and reviewers may also disagree about whose role it is.
(NB. researchers, editors and reviewers could all check for reporting quality, but this research focusses only on researchers).
Ideas to address this influence:
I.32 Researchers may not consider writing as reporting
Researchers may need to change their approach to writing or what they consider writing to be.Researchers differ in their writing process. Authors that follow a structured approach to writing may find it easier to incorporate reporting guidelines into their workflow. Some experienced researchers may be used to a way of working and reluctant to change, and some inexperienced researchers may be unaware of alternative writing processes.
Ideas to address this influence: