Appendix O — Ideas generated from workshops and focus groups
The following list of ideas were generated from my workshops with EQUATOR staff and focus groups with guideline developers, journal staff, and academics (chapters 7 and 8). Each bullet point is a code, and codes were grouped inductively into broader ideas that felt cohesive. Each group is labelled with the influence(s) the idea addresses (from Appendix I) and the stakeholders involved.
In chapter 9 I describe how I used this list to define intervention components, and that during this process additional ideas came to me. I’ve denoted these ideas with JH.
Before developing a reporting guideline
O.1 Create reporting guidance for early stages of research
Consider creating reporting guidance to help authors write protocols, funding applications, and ethics applications.
Who could do this: Guideline developers
Influences addressed:
Researchers may not encounter reporting guidelines early enough to act on them
O.2 Avoid confusing authors with too many reporting guidelines
- To avoid duplicating resources, before commencing a new reporting guideline:
- consult EQUATOR’s register of reporting guidelines under development;
- EQUATOR could make this register easier to find and search;
- contact the developers of related reporting guidelines;
- journals could ask reporting guideline developers to prove that they have registered their guideline with EQUATOR (like they do for clinical trials).
- consult EQUATOR’s register of reporting guidelines under development;
- When a new reporting guideline is justified, build upon existing reporting guidelines instead of starting from scratch. This could mean extending or replacing subsets of items instead of publishing a totally new reporting guideline.
- Consider making modular guidance. For instance, the Journal Article Reporting Standards (JARS) are a set of reporting guidelines for psychology. JARS has a main guideline for quantitative studies. The Design item can be extended by one of three modules depending on whether the study involved experimental manipulation or was conducted on a single individual. the experimental manipulation module can be further extended by modules for random assignment, non-random assignment, and clinical trials. Other extension modules exist for longitudinal and replication studies.
Who could do this: Guideline developers, EQUATOR Network, Publishers
Influences addressed:
Researchers may struggle to reconcile multiple sets of guidance
Researchers may not know what reporting guideline is their best fit
When developing a reporting guideline
O.3 Avoid prescribing structure
- Avoid prescribing structure of a journal article as it may clash with journal requirements or other reporting guidelines.
- Instead, give options for where items can be reported.
- Include options beyond the article body where authors can report information, like tables, figures, or appendices be.
Who could do this: Guideline developers
Influences addressed:
Researchers may struggle to reconcile multiple sets of guidance
O.4 Keep reporting guidelines agnostic to design choices
Ask authors to describe methods transparently without making assumptions about, or prescribing, methods or design choices. For example, an instruction to “describe how you determined your sample size” may be more helpful than “report your sample size calculation” for authors who encounter checklists at submission and did not perform a sample size calculation before collecting data.
Avoid recommending or admonishing design choices within the reporting guidance because:
- doing so may make authors feel nervous or ashamed, and therefore less likely to report transparently;
- design advice elongates reporting guidelines;
- including design advice may give the impression that the reporting guideline is for designing or appraising design.
Consider linking to external design or appraisal tools instead.
Who could do this: Guideline developers
Influences addressed:
Researchers may feel restricted if reporting guidelines prescribe design
O.5 Describe reporting items fully
For each item, authors may need to know the following:
- What needs to be reported – a brief description could go in all resources (checklists, templates etc.) with a longer description in the full guideline document.
- Why the information is important, and to whom
- Any circumstances where the item is not applicable and what to write
- Indicate priority, and any circumstances that modify importance
- Where the item can be reported, including beyond the main article body (e.g., section, table, figure, appendix)
- What to write if an item wasn’t, or couldn’t be done
- What to write if an item cannot be reported for external reasons. For example, if items cannot be reported because of intellectual property restrictions.
- Examples, which could be real or generated, including:
- examples of good and bad reporting with explanations.
- examples of concise or word-count-friendly reporting, perhaps in alternative formats like tables and figures.JH
- examples of well reported “imperfect” items (items that were not done)
- examples from different research contexts
- Links to external design or appraisal advice
Who could do this: Guideline developers
Influences addressed:
Researchers may not know whether a reporting guideline applies to them
Researchers may not know what to write when they cannot report an item
Researchers may not care about the benefits of using a reporting guideline
Researchers may be asked to remove reporting guideline content
O.6 Describe each reporting guideline fully
For each reporting guideline, authors may need the following information:
- A clear definition of the reporting guideline’s intended scope in plain language.
- If-then rules to direct authors to other, more appropriate reporting guidelines. For example, CONSORT could point authors writing protocols to SPIRIT.
- If no better guidance exists then indicate which items do/do not apply. For example, no guideline exists for authors writing protocols for observational epidemiology. Their best option currently is to use STROBE, but only some items will be required in a protocol.
- What tasks the reporting guideline can and cannot be used for
- How long the resource will take to use
- Why the guidance should be trusted and link to how it was developed
Who could do this: Guideline developers, EQUATOR Network
Influences addressed:
Researchers may not know whether a reporting guideline applies to them
Researchers may not know what reporting guideline is their best fit
Researchers may not know when reporting guidelines should be used
Researchers may feel that checking reporting is someone else’s job.
O.7 Keep guidance short
Keep guidance as a short as possible:
- Be concise but clear.
- Be realistic about what to expect from authors as each additional item increases the chances an author will be put off
- Link to other guidance elsewhere if desired.
- Consider splitting broad guidance that tries to cater for different options into shorter, modular guidance (modularity avoids duplication).
Who could do this: Guideline developers
Influences addressed:
When writing guidance down and creating resources
O.8 Make resources ready-to-use
Ensure resources are ready-to-use e.g., checklists as Word files, not as tables within published articles.
Who could do this: Guideline developers, EQUATOR Network
Influences addressed:
O.9 Make reporting guidelines easy to understand
- Use plain language.
- Define key terms.
- Use consistent terms across related resources.
- Provide translations.
- Update guidance in response to user feedback.
Who could do this: Guideline developers
Influences addressed:
O.10 Use persuasive language and design
- Use language and design to communicate confidence and simplicity as opposed to judgement and complexity.
- Encourage explanation even when choices were unusual or sub-optimal.
- Reassure authors that most research has limitations that can be addressed in Discussion sections.
- Reassure authors that reporting guidelines are just guidelines.
- Avoid patronizing authors.
- Consider wording instructions directly at the intended user.JH
Who could do this: Guideline developers, EQUATOR Network, Publishers, Funders, Ethics committees, Institutions, Conference organisers, Registries, Preprint servers
Influences addressed:
Researchers may feel that checking reporting is someone else’s job.
O.11 Create additional tools
Create tools for different tasks:
- discussion points for planning research in the order decisions are made
- to-do lists for conducting research in the order data is collected
- templates for drafting
- writing assistance tools (e.g., COBWEB)
- checklists for checking manuscripts that are easy to fill out, update, and cross-check
- tools for co-researchers to check each other’s work
- tools for generating tables and figures
- resources for peer reviewers who wish to review reporting quality including:
- guidance specifically for peer reviewers.
- commonly-used words that reviewers can search for to quickly find relevant text.JH
- suggested text that peer reviewers can copy to request information
- tools to generate feedback reports
- journal articles where reporting guideline items are annotated/highlighted
Who could do this: Guideline developers, EQUATOR Network, Funders, Ethics committees, Publishers
Influences addressed:
Researchers may not encounter reporting guidelines early enough to act on them
O.12 Make resources easy to discover and find
Link resources:
- Ensure all resources link to each other. For example, checklists should link to explanation and elaboration documents and vice versa.
- Related reporting guidelines should link to each other.
- Reporting guidelines and resources should link to translations
- Links should be permanent (e.g., DOIs) where possible and old links should be maintained or redirected. Broken links should be replaced.
Make searching easy:
- Host resources somewhere consistent, like the EQUATOR Network website and database.
- Provide easy-to-use website search functions
- Web pages should be optimized for search engines JH
- Created curated collections for study types
- Create decision tools for identifying reporting guidelines
Names reporting guidelines to make them easy to discover and find:
- Reporting guideline names could be descriptive, as acronyms may be meaningless to novice users.
- Related reporting guidelines should use consistent names to show relationships (e.g., PRISMA and PRISMA-P appear more related than CONSORT and SPRIT).
Who could do this: Guideline developers, EQUATOR Network
Influences addressed:
Researchers may not know what resources exist for a reporting guideline
Researchers may not know what reporting guideline is their best fit
O.13 Make information digestible
Organise information so it is easy to navigate and not overwhelming.
- Cater to users that read from start to finish, and those that dip in and out.
- Structure text with headings.
- Use section URLs to send authors directly to relevant parts of guidance.
- Consider hyperlinking related resources
- Consider embedding reporting guidelines that “fit together”, like PRISMA and PRISMA-Abstracts
- For information presented online, consider showing/hiding information as required. For example, if PRISMA-Abstracts were embedded into PRISMA, users could choose to expand or collapse it. Or you could show/hide guidance depending on whether the author is writing a funding application, protocol, manuscript.
Who could do this: Guideline developers, EQUATOR Network
Influences addressed:
When disseminating resources
O.14 Describe reporting guidelines where they are encountered
- When authors first encounter reporting guidelines they may need to know:
- what reporting guidelines are
- how and when to use them
- why authors should use them, including:
- what personal benefits to expect
- the importance to others.
- Descriptions could be succinct (e.g., on journal instruction pages) or long (e.g., in publications) JH
- A generalised description can go where authors first encounter reporting guidelines e.g., journal author guidelines, EQUATOR’s home page.
- A reporting guideline-specific description could go at the top of guidance documents, checklists, and templates.
- Consider specifying whether the reporting guideline is also a design guideline.
- Specify whether the reporting guidelines are just guidelines, or whether they are intended to be requirements. Name the resource appropriately - words like guideline, standards, criteria, recommended, preferred, and templates, have different meanings.
Who could do this: Publishers, EQUATOR Network, Guideline developers, Funders, Ethics committees, Institutions, Registries, Preprint servers, Conference organisers
Influences addressed:
Researchers may not know when reporting guidelines should be used
Researchers may feel that checking reporting is someone else’s job.
O.15 Make resources accessible
Ensure resources are open access. This allows access to authors without journal subscriptions and allows others to build upon the guidance.
Who could do this: Guideline developers, EQUATOR Network
Influences addressed:
O.16 Show and encourage citations
- Display usage data (like citations or downloads) alongside the guidelines as a form of social proof.
- Encourage authors to cite the reporting guideline so readers discover it.
Who could do this: Guideline developers, EQUATOR Network, Publishers, Conference organisers, Preprint servers
Influences addressed:
O.17 Provide testimonials
Testimonials can be short quotes or longer case studies. They could come from:
- researchers who have had positive experiences using reporting guidelines, including researchers that were nervous about transparency,
- decision makers (e.g., editors/grant managers) that value good reporting and/or check for reporting as part of their evaluation,
- peer reviewers that use reporting guidelines to check for good reporting,
- patients who are affected by research waste,
- and researchers who need to understand, synthesise, or apply research articles.
Who could do this: Guideline developers, EQUATOR Network
Influences addressed:
Researchers may not care about the benefits of using a reporting guideline
On an ongoing basis
O.18 Budget for reporting
Funders and research supervisors could encourage researchers to allocate sufficient time and money for documenting and reporting results of their research.
Who could do this: Funders, Institutions
Influences addressed:
O.19 Create rewards
Stakeholders could create new rewards:
- journals could fast-track submissions or review for papers that followed a reporting guideline,
- journals could offer discounts on article processing charges for papers that followed a reporting guideline,
- journals, preprint servers, or peer review platforms could badge well reported articles,
- EQUATOR could offer a certification service,
- funders could reward good reporting financially,
- institutions could offer prizes for good reporting.
Who could do this: Guideline developers, EQUATOR Network, Publishers, Funders, Institutions, Preprint servers, Registries
Influences addressed:
Researchers may not care about the benefits of using a reporting guideline
O.20 Create discussion spaces
Create spaces for authors to discuss reporting and reporting guidelines. These could be:
- online (forums, social media, email),
- or offline (meet-ups, clubs).
Try to make spaces accessible to researchers from all nationalities, professional disciplines and other demographics. Spaces will allow authors to:
- solicit help,
- share experiences,
- provide feedback to guideline developers,
- and cultivate a feeling of inclusivity and community ownership.
Who could do this: EQUATOR Network, Guideline developers, Institutions
Influences addressed:
O.21 Create ways to catch authors earlier
- Consider creating email campaigns to prompt researchers at early stages.
- The EQUATOR website could encourage visitors to use reporting guidelines for planning and drafting research.
- Websites could be optimised for search terms like “how to write [study type]”, “protocol”, “research plan” or “funding application”. For example, reporting guideline pages on EQUATOR’s website rank highly in Google searches for “STROBE checklist” but not “How to write an observational epidemiology study”.JH
- Writing clubs and writing training could flag reporting guidelines.
Who could do this: EQUATOR Network, Guideline developers, Publishers, Funders, Ethics committees, Institutions, Conference organisers, Preprint servers, Registries
Influences addressed:
Researchers may forget to use reporting guidelines at earlier research stages
Researchers may not encounter reporting guidelines early enough to act on them
Researchers may not know when reporting guidelines should be used
O.22 Endorse and enforce reporting guidelines
Stakeholders could:
- endorse reporting guidelines
- enforce their use by mandating checklists or (preferably) checking adherence to items.
- Funders could ask about reporting guidelines or checklists when collecting updates from grant recipients.
Who could do this: Publishers, Institutions, Ethics committees, Funders, Registries, Conference organisers, Preprint servers
Influences addressed:
O.23 Evidence the benefits
Evidence any stated benefits:
- Quantifiable benefits could be evidenced with data (e.g., acceptance rates, publishing speed, writing speed).
- Experiential benefits could be evidenced by collecting case studies from authors who find that reporting guidelines help them feel confident and write more easily, and from readers who value well-reported research.
Who could do this: Guideline developers, EQUATOR Network, Publishers
Influences addressed:
O.24 Make reporting guidelines appear as a priority
Journals, funders and ethics committees could make reporting guidelines appear as a priority:
- Make them prominent in author instructions.
- Placing checklists earlier in the PDFs that are automatically created by journal submission systems.
- Publicize when reporting guidelines are used by reviewers.
Who could do this: Publishers, Funders, Ethics committees, Institutions, Preprint servers, Conference organisers, Registries
Influences addressed:
Researchers may not care about the benefits of using a reporting guideline
O.25 Promote reporting guidelines
- Promote reporting guidelines on and offline.
- Online may include websites, email campaigns, social media, and blogs.
- Offline may include appearing at conferences, seminars, and workshops.
- Institutions could promote reporting guidelines in their curricula, learning materials, or through reporting champions. Reporting guideline developers or EQUATOR could push for reporting guidelines to be included in text books.
- Promotion can begin before a reporting guideline has been published so that researchers know about guidelines being developed.
NB. Promotion is different to endorsement; a journal could run an email campaign to promote reporting guidelines without having an endorsement policy.
Who could do this: Institutions, Publishers, Guideline developers, EQUATOR Network, Ethics committees, Funders, Societies, Registries, Conference organisers, Preprint servers
Influences addressed:
O.26 Install reporting champions
All stakeholders could have members to promote and facilitate the usage of reporting guidelines.
- This could follow a local network model with EQUATOR as the central organiser.
- Could make use of existing networks, like regional reproducibility networks.
Who could do this: EQUATOR Network, Guideline developers, Institutions, Funders, Ethics committees, Publishers, Conference organisers, Preprint servers, Registries
Influences addressed:
Researchers may not know when reporting guidelines should be used
O.27 Provide additional teaching
Provide education or training (e.g., courses, videos) specific to particular reporting guidelines.
More generally, students could:
- learn about writing as a process and workflows for documenting and communicating research,
- learn about research waste from poor reporting,JH
- attempt a replication to learn about the importance of complete reporting,
- and use a reporting guideline as part of their studies.
Who could do this: Guideline developers, EQUATOR Network, Publishers, Institutions, Funders, Ethics committees
Influences addressed:
Researchers may not encounter reporting guidelines early enough to act on them
Researchers may not care about the benefits of using a reporting guideline
O.28 Make updating guidelines easier
Update guidance in response to user feedback or changes in the field. This would be easier if:
- reporting guideline developers could easily collect feedback from authors.
- small updates or refinements could be made without publishing a new article.
- reporting guideline developers had funding to evaluate, refine, and update their resources.JH
Who could do this: EQUATOR Network, Funders
Influences addressed: