Appendix P — Intervention Components Table
In chapter 9 I described how I combined the outputs of previous chapters to create the intervention components listed below.
| INTERVENTION INGREDIENT | BCT | INTERVENTION FUNCTION | BEFORE | NOW |
|---|---|---|---|---|
| Key Behaviour: Engage with (read) appropriate reporting guidance as early as possible | ||||
Targeted barrier: Researchers may not know what reporting guidelines are Behavioural driver: Capability |
||||
| Describe what reporting guidelines are where they are first encountered | Instruction on how to perform the behaviour | Education | No prominent description of what reporting guidelines are on EQUATOR home page or in reporting guidelines resources. Example: See Figure 9.3, Figure 9.6 |
Prominent definition on home page and guideline page. Example: See Figure 9.4, Figure 9.7 |
| Clarify what tasks (e.g., writing, designing, or appraising research) guidelines and resources are designed for | Instruction on how to perform the behaviour | Education | No clear instruction on what tasks reporting guidelines or their resources can and cannot be used for. Example: See Figure 9.3, Figure 9.5, Figure 9.6 |
Clear instruction and differentiation of resources Example: See Figure 9.4, Figure 9.7 |
Targeted barrier: Researchers may not know what reporting guidelines exist Behavioural driver: Capability |
||||
| Instruct authors to cite reporting guidelines so readers may learn about them | Instruction on how to perform the behaviour | Education | No consistent instruction to cite reporting guidelines | Consistent instruction to cite reporting guidelines |
| Decision tools for discovering appropriate resources* | Instruction on how to perform the behaviour | Enablement | We previously made a “reporting guideline wizard” but it was difficult to find. Example: see Figure 9.3 |
Not included yet |
| Collections of related reporting guidelines* | Adding objects to the environment | Environmental Restructuring | Collections exist on EQUATOR site but are difficult to find. | Not included yet |
| Links between related guidelines | Restructuring the physical environment | Environmental Restructuring | Guideline publications may cite guidelines published previously, but these can be buried in text and are not updated. EQUATOR website guideline pages feature links to extensions, but these may be hard to find. Checklists do not link to related resources. Example: See Figure 9.5, Figure 9.6 |
Guidelines prominently link to other relevant guidelines and explain when they should be used. Example: See Figure 9.7 |
| Embed reporting guidelines that “fit together”* | Instruction on how to perform the behaviour | Enablement | Checklists and their extensions are published separately. The best example of modular guidance is perhaps the JARS guidelines, but even these are published as separate documents. | No change |
Targeted barrier: Guidance may be difficult to find Behavioural driver: Opportunity |
||||
| Centralised hosting | Restructuring the physical environment | Enablement | EQUATOR maintains a database of reporting guideline meta-data, but the guidance and checklists were published and hosted in different locations and in different ways. | A core set of frequently accessed guidelines are now presented on a single website. |
| Search function on website | Restructuring the physical environment | Enablement | EQUATOR’s search function was difficult to find. Example: See Figure 9.3 |
Search function is easier to find as a recognizable icon in the navigation bar of every page. The home page includes additional ways to access search functionality. Example: See Figure 9.4 |
| Search Engine Optimization | Restructuring the physical environment | Enablement | EQUATOR’s website did not make use of some commonly used search optimization heuristics. It ranked well for guideline acronyms (like STROBE) but not for general terms that naive authors may use, like “observational epidemiology” or “how to write-up research”. The site wasn’t optimized for viewing on mobile devices, which will also harm google search rankings. Example: (Not visible) |
The site has additional meta-data. Each reporting guideline page has its own meta-data. The site is optimized for mobiles. Example: (Not visible) |
| Permanent document object identifiers (DOIs)* | Restructuring the physical environment | Enablement | Although guideline publications have DOIs, tools (commonly hosted on guideline developer’s websites) do not. EQUATOR’s website does not use document object identifiers. If resources move (e.g., a website is reorganised or depreciated) then links can “die”. | No change |
Targeted barrier: Reporting guidelines may be difficult to access Behavioural driver: Opportunity |
||||
| Ensure guidelines and tools are open access* | Restructuring the physical environment | Enablement | Some guidelines are published behind paywalls | No change |
Targeted barrier: Researchers may not know whether a reporting guideline applies to them Behavioural driver: Capability |
||||
| Describe the scope of a reporting guideline at the top of every resource | Instruction on how to perform the behaviour | Education | Some reporting guidelines may describe their scope within a publication. Others might not, or may only describe their scope broadly without fully explaining the design assumptions within the guidance. Guideline publications rarely explain circumstances where the guidance should not be used. Checklists rarely define intended scope beyond the title of the guideline. | The intended scope of a guideline is clearly & prominently described. This definition includes contexts in which the guidance should not be used. Example: See Figure 9.7 |
Targeted barrier: Researchers may not know what reporting guideline is their best fit Behavioural driver: Capability |
||||
| Use if-then rules to direct authors to more appropriate and up-to-date guidance when available | Instruction on how to perform the behaviour | Education | Reporting guidelines do not consistently point authors towards related resources that might be better fits. Guidelines are not updated as-and-when other guidelines become available. | Reporting guidelines clearly and consistently point authors to more appropriate guidance when appropriate, using if-then rules. These links can be updated any time. Example: See Figure 9.7 |
| Explicitly state when no better guidance exists for a use case | Instruction on how to perform the behaviour | Education | Reporting guidelines rarely explain what to do when no better guidance exists for a use case. | Reporting guidelines warn authors when no better guidance exists for a use case, and how the current guidance can be adapted instead Example: See Figure 9.7 |
Targeted barrier: Researchers may not understand the language Behavioural driver: Opportunity |
||||
| Provide translations | Instruction on how to perform the behaviour | Enablement | Some guidelines have been translated, but many haven’t. Links to translations are present on reporting guideline database pages but these links may not be easy to find. The EQUATOR website has an automatic translation tool which will translate content on web pages, but this doesn’t cover the guidance itself. Example: SRQR was available in French but the translation wasn’t advertised on the main guidance or database page. See Figure 9.6 and Figure 9.5 |
Translations are prominently listed above the guidance Example: See Figure 9.7 |
Targeted barrier: Researchers may expect the costs to outweigh benefits Behavioural driver: Motivation |
||||
| Make guidance appear shorter by removing superfluous information, hiding optional content, splitting long guidelines, using concise language, and separating design advice | Restructuring the physical environment | Environmental restructuring | reporting guideline publications may include lengthy explanation of development, verbose language, and can be bloated by design advice Example: See Figure 9.8 |
SRQR has been edited. The only text presented immediately is instruction on what the author needs to describe. Additional information is hidden at first and can be expanded. Text is shortened through editing and by using active voice. In the case of SRQR, this reduced the text length by 60%. Example: See Figure 9.9 |
| Cater to different kinds of user (readers vs dippers) by structuring guidance with headings, itemisation, hyperlinking to particular sections, and with optional content | Restructuring the physical environment | Environmental restructuring | Besides being split into items, reporting guidance is largely unstructured and different items can be organised in different ways. Checklist items do not link to items within an elaboration document. Example: See Figure 9.8 |
SRQR items are structured consistently, making information easier to find. Itemisation is used consistently, content is hyperlinked when useful. Example: See Figure 9.9 |
| Include testimonials from researchers who were nervous about being punished for reporting transparently | Demonstration of the behaviour | Persuasion | No such testimonials exist | Quotes included alongside guideline Example: see Figure 9.9 |
| Decrease fear of judgement by making reporting guidelines design agnostic | Remove aversive stimulus | Coercion (Removal of) | Reporting guidelines may conflate reporting advice with design advice or design assumptions. The justification for why an item is important to describe is frequently presented in terms of good and bad design. | SRQR explicitly states that it makes no assumptions about design. Inadvertent design assumptions were edited. |
| Remove branding and messaging that may invoke feelings of judgement, complexity, or administrative red-tape | Remove aversive stimulus | Coercion (Removal of) | EQUATOR’s website looked cluttered and visually unappealing. Guidance published in articles can look unappealing and dense. When justifying why authors should use reporting guidelines, guideline developers frequently referenced research waste, (lack of) transparency, bias, and poor design. Example: See Figure 9.3 |
A clean, simple interface for the home page and guidance pages. Text makes less use of judgemental phrases and fewer references to the negative consequences of poor reporting. Example: See Figure 9.4 |
| Reassure that all research has limitations to encourage explanation over perfect design | Social support (unspecified) | Persuasion | Few guidelines would include this kind of reassurance Example: See Figure 9.6 |
This reassurance appears on the home page and all guidance pages Example: See Figure 9.7 |
Targeted barrier: Researchers may feel that checking reporting is someone else’s job. Behavioural driver: Motivation |
||||
| Address communications to authors | Instruction on how to perform a behaviour | Persuasion | It wasn’t always clear whether reporting guidelines and the EQUATOR Network website were aimed at authors, editors, reviewers, or all. | All resources and website copy are directed predominantly at authors. |
| Communicate why reporting is primarily the responsibility of the author* | Instruction on how to perform a behaviour | Education | Because it wasn’t clear how reporting guidelines and checklists should be used, they (especially checklists) could appear as administrative tasks that should be the responsibility of the editor or reviewer. | Clear explanation of why guidelines and tools should be used by authors primarily, although can also be used by others. |
Targeted barrier: Researchers may not consider writing as reporting Behavioural driver: Motivation |
||||
| Educate authors about writing as a process | Instruction on how to perform a behaviour | Education | Many researchers don’t get trained on writing as a process, they just do it. EQUATOR provided education about how to write but this wasn’t advertised on guidelines. | Some SRQR items now link to relevant EQUATOR materials and courses. Example: See Figure 9.9 |
| Key Behaviour: Apply reporting guidance to writing | ||||
Targeted barrier: Researchers may not know what resources exist for a reporting guideline Behavioural driver: Capability |
||||
| Link all resources to each other | Restructuring the physical environment | Environmental restructuring | Reporting guideline development articles, explanation and elaboration articles, and checklist files may not link to each other. | Guidance links to all tools and development article Example: See Figure 9.7 |
Targeted barrier: Researchers may not know what benefits to expect Behavioural driver: Capability |
||||
| Describe personal benefits and benefits to others where reporting guidelines are introduced (home page, on resources, in communications) | Information about emotional consequences, Information about others’ approval, Information about social and environmental consequences | Education | Benefits are not prominently described on EQUATOR’s home page, nor within the guideline publications. Benefits that are described may by hard to find, and often focus on hypothetical benefits to the research community, but not personal benefits to the author. Example: See Figure 9.3 |
Benefits are prominently and consistently displayed across the home page and guidance pages. Descriptions prioritize personal benefits to the authors above hypothetical benefits to others. Example: See Figure 9.7 |
Targeted barrier: Researchers may not believe stated benefits Behavioural driver: Motivation |
||||
| Gather and communicate evidence for benefits | Information about emotional consequences, Information about others’ approval, Information about social and environmental consequences | Persuasion | Benefits often presented without evidence (if at all) | Dummy quotes provides evidence for experienced benefits. Example: See Figure 9.9 |
| Include design, features, and language to foster trust | Credible source, Social comparison | Persuasion | Website design looked amateur. Citation metrics are available for guideline publications, but are not displayed on the EQUATOR website or within the guidance publications or checklists themselves. Guidelines were often preceeded by lengthy explanations of development. Example: See Figure 9.6 |
Professional design. EQUATOR’s Logo remains prominent. Citation metrics are presented at the top the reporting guidance. Information about who developed the guidelines, how they developed it, and why the guidance is credible is still provided, and easily findable from the top of the guidance. Example: See Figure 9.7 |
| Create spaces for authors to discuss reporting guidelines with others | Social comparison, Credible source, Adding objects to the environment | Persuasion | There were no official on- or offline spaces for authors to discuss guidelines. | Each reporting item has its own discussion board. Example: See Figure 9.10 |
| Use tone of voice and design to communicate personal benefits; confidence and simplicity | Framing/reframing | Persuasion | Guidance text made little use of a reassuring tone or words. The EQUATOR website and guideline articles looked dense and complex. Example: See Figure 9.3 and Figure 9.6 |
A clean, simple interface for the home page and guidance pages. Text uses phrases like “confidence”, “quick”, “maximum impact”. Example: See Figure 9.4 and Figure 9.7 |
Targeted barrier: Researchers may not care about the benefits of using a reporting guideline Behavioural driver: Motivation |
||||
| Include testimonials from research users who benefit from complete reporting | Salience of consequence | Persuasion | Testimonials not included in reporting guidelines. | SRQR includes dummy testimonials and quotes from research users Example: See Figure 9.9 |
Targeted barrier: Researchers may misunderstand Behavioural driver: Capability |
||||
| Use plain language | Instruction on how to perform the behaviour | Enablement | Although developers aspire to write clearly, authors may misinterpret guidance or fail to understand it completely. | SRQR is edited to use plainer language. |
| Define key terms | Instruction on how to perform the behaviour | Education | Few guidelines came with a glossary. Some key terms may be defined within the guideline text. Including definitions this way makes them hard to find and elongates the guidance. | SRQR now has a glossary, and text is marked-up with definitions that appear upon click. Example: See Figure 9.9 |
| Use consistent terms | Instruction on how to perform the behaviour | Enablement | Guidelines may use different terms to refer to the same thing (or the converse - use the same term to refer to different things). A single guideline may do this too, as can websites. | The website uses terms consistently |
| Provide translations | None | See above | ||
| Create spaces for authors to discuss reporting guidelines with others (see above) | None | |||
Targeted barrier: Researchers may not know why items are important Behavioural driver: Capability |
||||
| For each item, explain why the information is important and to whom (not just what constitutes “good” design) | Information about social and environmental consequences | Education | Sometimes there was no explanation as to why an item should be reported. Other times the justification would be about why a particular design choice was important. Example: See Figure 9.8 |
Information added when necessary Example: See Figure 9.9 |
| Explain importance of complete reporting to the scientific community | Information about social and environmental consequences | Education | EQUATOR and most reporting guidelines do this already | Continue to do this |
Targeted barrier: Researchers may not know how to do an item Behavioural driver: Capability |
||||
| Provide links to other resources that explain how an item can be done | Instruction on how to perform the behaviour | Education | Some reporting guideline publications (or elaboration articles) include instruction in text but many don’t. SRQR did not. | Links included when relevant. Example: See Figure 9.9 |
Targeted barrier: Researchers may not know how to report an item in practice Behavioural driver: Capability |
||||
| For each item, provide clear instruction of what needs to be described | Instruction on how to perform the behaviour | Education | Writing instructions are often mixed in with other explanation and context. Example: See Figure 9.8 |
Writing instruction occurs first for each item. Example: See Figure 9.9 |
| For each item, provide examples of reporting in different contexts | Demonstration of the behaviour | Modelling | Not all reporting guidelines provide examples. Examples may not cover different contexts. Example: See Figure 9.8 |
SRQR already had some examples. No more examples added Example: See Figure 9.9 |
| Create spaces for authors to discuss reporting guidelines with others (see above) | None | |||
Targeted barrier: Researchers may not know what to write when they cannot report an item Behavioural driver: Capability |
||||
| Provide clear instruction of what needs to be described when an item was not done, could not be done, or does not apply | Instruction on how to perform the behaviour | Education | Rarely instructed Example: See Figure 9.8 |
Instructed where relevant Example: See Figure 9.9 |
| Provide examples of reporting “imperfect” items* | Demonstration of the behaviour | Modelling | Examples not provided | No changes made |
Targeted barrier: Researchers have limited time Behavioural driver: Opportunity |
||||
| Ensure all resources and tools (e.g., checklists and templates) are in ready-to-use formats* | Adding objects to the environment | Enablement | Some checklists not in immediately usable formats e.g., PDFs | No changes made |
| Structure guideline items to make them quicker to digest | Restructuring the physical environment | Enablement | E&E documents not structured below the item level Example: See Figure 9.8 |
Items have consistent structure and use bullet points consistently Example: See Figure 9.9 |
| Tell authors how long the guidance will take to read | Instruction on how to perform the behaviour | Education | Estimated reading time not given | Estimated reading time given Example: See Figure 9.7 |
| Tell authors how long guidance will take to apply* | Instruction on how to perform the behaviour | Education | No advice given | No changes made Example: No changes made |
Targeted barrier: Researchers may not know when reporting guidelines should be used Behavioural driver: Capability |
||||
| Tell authors when to use reporting guidelines, or that reporting guidelines are best used as early as possible | Instruction on how to perform the behaviour | Education | Rarely stated prominently | Stated prominently Example: See Figure 9.7 |
| Clarify what tasks (e.g., writing, designing, or appraising research) guidelines and resources are designed for (see above) | None | |||
Targeted barrier: Researchers may not encounter reporting guidelines early enough to act on them Behavioural driver: Opportunity |
||||
| Optimize websites for search terms aimed at early use like “how to write”, or “funding application”. (See Search Engine Optimization above) | None | |||
| Create prompts / communication campaigns to target authors early in their research* | Prompts/cues | Enablement | EQUATOR had no way to do this | No changes made |
| Create tools to be used for early writing tasks* | Adding objects to the environment | Enablement | Most reporting guidelines come with a checklist but none come with a template, or tools/guidance specific to protocols or funding applications. | No changes made |
Targeted barrier: Researchers may struggle to keep writing concise Behavioural driver: Opportunity |
||||
| Provide instruction as to how and where information can be reported without breaching word count limits or making articles bloated. | Instruction on how to perform the behaviour | Education | Few reporting guidelines include this information | Added instruction at top of reporting guideline and in some items where most useful Example: See Figure 9.7 |
| Provide examples of concise reporting* | Demonstration of the behaviour | Modelling | No examples specifically to display concise reporting | No changes made |
Targeted barrier: Researchers may not have tools for the job at hand Behavioural driver: Opportunity |
||||
| Create guidance for planning research, or for writing protocols/funding applications (see Create tools to be used for early writing tasks)* | ||||
| Create to-do lists in the order research is conducted, to help authors collect information they will need to report (see Create tools to be used for early writing tasks)* | ||||
| Create templates for drafting (see Create tools to be used for early writing tasks)* | ||||
| Create tools to facilitate checklist completion* | Adding objects to the environment | Enablement | Updating page numbers in a checklist is time consuming. It takes editors time to double check page numbers and content. Checklists may not include instructions of how to complete them. | No changes made |
| Create tools to facilitate particular reporting items* | Adding objects to the environment | Enablement | Some tools exist (e.g., PRISMA flow chart diagram maker, COBWEB) | No changes made |
| Create tools to help collaborators check each other’s work* | Adding objects to the environment | Enablement | Checklists exist but aren’t specifically designed for collaborators | No changes made |
| Create tools to help peer reviewers check reporting and request missing information* | Adding objects to the environment | Enablement | Checklists are reporting guidelines are not specifically aimed at peer reviewers | No changes made |
Targeted barrier: Reporting guidelines can become outdated Behavioural driver: Opportunity |
||||
| Provide feedback channels to help developers keep guidance updated (see Create spaces for authors to discuss reporting guidelines with others) | None | |||
Targeted barrier: Researchers may struggle to reconcile multiple sets of guidance Behavioural driver: Opportunity |
||||
| Explain when reporting guidelines do not intended to prescribe structure | Instruction on how to perform the behaviour | Education | Not always stated. Not always prominent Example: see Figure 9.6 |
Explained at top of guidance Example: see Figure 9.7 |
| Provide instruction as to how and where information can be reported without breaching word count limits or making articles bloated (see above) | None | |||
| Embed reporting guidelines that “fit together” (see above)* | None | |||
Targeted barrier: Researchers may be asked to remove reporting guideline content Behavioural driver: Opportunity |
||||
| Provide advice regarding how to respond if asked to remove reporting guideline content by a colleague, editor, or reviewer | Problem solving | Education | No advice given | Advice given in FAQ Example: See Figure 9.7 |
Targeted barrier: Reporting guideline resources may not be in usable formats Behavioural driver: Opportunity |
||||
| Ensure all resources and tools (e.g., checklists and templates) are in ready-to-use formats (see above)* | None | |||
Targeted barrier: Researchers may feel afraid to report transparently Behavioural driver: Motivation |
||||
| Present design advice separately to reporting advice | Restructuring the physical environment | Coercion (removal of) | Some reporting guideline E&Es include design advice Example: SRQR did not include design advice |
No changes made |
| Make reporting guidelines agnostic to design choices (see Decrease fear of judgement by making reporting guidelines design agnostic) | None | |||
| Encourage explanation even when choices are unusual or not optimal | Instruction on how to perform the behaviour | Education | Not always present Example: See Figure 9.8 |
Added to items Example: See Figure 9.9 |
| Reassure authors that all research has limitations (see Reassure that all research has limitations to encourage explanation over perfect design) | None | |||
| Include testimonials from researchers who were nervous about being punished for reporting transparently (see Include testimonials from researchers who were nervous about being punished for reporting transparently) | None | |||
Targeted barrier: Researchers may feel restricted if reporting guidelines prescribe design Behavioural driver: Motivation |
||||
| Present design advice separately and remain design agnostic (see Decrease fear of judgement by making reporting guidelines design agnostic) | None | |||
| Reassure when guidelines are just guidelines | Social support | Persuasion | Not always present or prominent Example: See Figure 9.6 |
Prominently displayed at top of reporting guideline Example: See Figure 9.7 |
Targeted barrier: Researchers may feel patronized Behavioural driver: Motivation |
||||
| Create spaces for authors to discuss reporting guidelines with others (see above) | None | |||
| Avoid patronizing language | Remove aversive stimulus | Persuasion | Although authors may feel patronized when asked to adhere to a reporting guideline, reporting guidelines themselves rarely use patronizing language | Continue to avoid using patronizing language |
| Explain how the guidance was developed and why it can be trusted | Credible source | Education | Most reporting guidelines explain this in a published article. Checklists do not Example: See Figure 9.6 |
Brief description included on home page and at top of reporting guideline, links to full to development information Example: See Figure 9.7 |
| Key Behaviour: Repeat engagement with reporting guidelines for subsequent studies | ||||
Targeted barrier: Researchers may forget to use reporting guidelines at earlier research stages Behavioural driver: Opportunity |
||||
| Create prompts / communication campaigns to target authors early in their research (see above)* | None |