Improving the Reporting of Primary Care Research: An International Survey of Researchers ======================================================================================== * William R. Phillips * Elizabeth Sturgiss * Liesbeth Hunik * Paul Glasziou * Tim olde Hartman * Aaron Orkin * Joanne Reeve * Grant M. Russell * Chris van Weel ## Abstract *Purpose:* To assess opportunities to improve reporting of primary care (PC) research to better meet the needs of its varied users. *Methods:* International, interprofessional online survey of PC researchers and users, 2018 to 2019. Respondents used Likert scales to rate frequency of difficulties in interpreting, synthesizing, and applying PC research reports. Free-text short answers were categorized by template analysis to record experiences, concerns, and suggestions. Areas of need were checked across existing reporting guidelines. *Results:* Survey yielded 255 respondents across 24 nations, including 138 women (54.1%), 169 physicians (60%), 32 scientists (11%), 20 educators (7%), and 18 public health professionals (6%). Overall, 37.4% indicated difficulties using PC research reports “50% or more of the time.” The most common problems were synthesizing findings (58%) and assessing generalizability (42%). Difficulty was reported by 49% for qualitative, 46% for mixed methods, and 38% for observational research. Most users wanted richer reporting of theoretical foundation (53.7%); teams, roles, and organization of care (53.4%); and patient involvement in the research process (52.7%). Few reported difficulties with ethics or disclosure of funding or conflicts. Free-text answers described special challenges in reporting PC research: context of clinical care and setting; practical details of interventions; patient-clinician and team relationships; and generalizability, applicability and impact in the great variety of PC settings. Cross-check showed that few current reporting guidelines focus on these needs. *Conclusions:* Opportunities exist to improve the reporting of PC research to make it more useful for its many users, suggesting a role for a PC research reporting guideline. * Evidence-Based Medicine * Faculty * Family Medicine * General Practice * Health Communication * Primary Health Care. Research Design * Research Report * Scholarly Communication * Surveys and Questionnaires * Translational Medical Research ## Introduction Primary care (PC) research is a growing discipline with great potential to improve patient care and population health.1 It is a broad enterprise, including work done by PC investigators, studies conducted in PC settings, and research about PC done by those in other specialties and disciplines. As with PC practice, PC research has developed its own perspectives and methods to meet its special challenges.2 It uses a broad array of research methods to study the universe of health problems across a wide-ranging variety of clinical and community settings, emphasizing patient-centered, problem-oriented, relationship-based approaches. PC research addresses not only direct clinical care, but also diverse subjects, including communication, health systems, implementation, evaluation, public health sciences, education, public policy, and the biopsychosocial model. PC research embraces many partners and serves many users. Investigators across many fields recognize opportunities to improve the planning, conduct, and reporting of research.3 The EQUATOR network4 ([https://www.equator-network.org](https://www.equator-network.org)) catalogs the growing array of research reporting guidelines aimed to improve the planning, conduct, dissemination, implementation, synthesis, and evaluation of research; increase the translation, adoption, and evaluation of new knowledge and improvements in patient care and health care systems; reduce delays from bench to bedside to patients and communities; reduce research waste; and enhance the impact and value.5 Reporting guidelines increasingly influence editorial policies of peer-reviewed journals.6⇓–8 The EQUATOR Network provides a core set of reporting guidelines that focus on key research methods, but the bulk of the 432 guidelines are specific to disciplines or subjects. Yet, none focuses on PC. PC is a distinct discipline, with specific needs for knowledge and research, and effective dissemination of findings is necessary to improve practice, patient outcomes and population health. However, little is known about the quality of PC research reporting or how well it meets the specific needs of its varied users: clinicians, patients and families, researchers, educators, policy makers, and communities. We can find in the published literature no reporting guidelines focused on PC research and very limited research on the quality or content of reports of PC research. Our long-term goal is to formulate guidance to help improve the reporting of PC research, recognizing the distinct contribution of PC to patient care and health care systems and optimizing the quality and impact of research as a core component of effective PC. This initial stage in our work has 2 specific aims. First, to assess the usefulness of, and characterize problem areas with, the current reporting of PC research. Second, to gather suggestions, topics, and elements for possible inclusion in such guidance. Informed by these findings, we plan a broad-based Delphi study to identify and prioritize items for PC research reports. To fill these knowledge gaps, we conducted an international, interprofessional, interspecialty, online survey of PC researchers, educators, and leaders to better understand how often current PC research reports are problematic and to explore opportunities to improve the reporting of PC research. We also reviewed existing reporting EQUATOR network guidelines to assess coverage of the needs reported by PC researchers. ## Methods We conducted an online survey using Qualtrics XM software (Qualtrics, Seattle, WA), October 2018 to March 2019. The questionnaire recorded demographic information, research training and experience, profession and specialty, years since completion of training, and research role. For this survey, we offered the following working definitions. “Primary care research” is original scholarly work on, in, or about PC. “Research reporting” is final reports published in peer-reviewed professional and scientific journals. We drafted survey questions to assess experiences, problems, limitations, concerns, unmet user needs, and opportunities for improving the reporting of PC research. Questions came from our international group of experienced PC investigators, authors, reviewers, editors, readers, and clinicians. Iterative drafts aimed to capture the most common and important functions of research reports. We field-tested several drafts with a multidisciplinary, international group of PC academics and clinicians, to improve readability, construct validity, and comprehension of scale items. This test group included a variety of potential survey respondents from 7 nations, including physicians, nurses, mental health clinicians and public health professionals; family medicine or general practice academics, researchers, practitioners, and educators; PC researchers, PhDs, social scientists, and other research team members. We asked respondents to estimate the frequency of encountering problems reading reports of PC research, using a 5-point Likert scale (always, most of the time, about half the time, sometimes, never, or not applicable/not sure). (See questionnaire, Appendix 1.) The questions covered potential problems with application and translation of study findings; study designs; research methods; and the reporting of conflicts of interest, funding, and bias. After each section, we invited open-ended short-text comments on all aspects of PC research reporting, asked for specific examples of concerns, and invited suggestions for improvement. We distributed the link to the online survey widely, starting with e-mails and Web site posts on many national and international PC research organizations, plus social media. We sent E-mail invitations with the survey link to individuals and/or organizations in over 54 nations on 6 continents. To increase and broaden the study population, we used a snowball sampling method,9 asking respondents to forward the survey link to PC colleagues, emphasizing the recruiting of those outside of North America, nonclinician researchers, and clinicians from diverse PC disciplines. For data analysis, we used descriptive statistics to summarize respondent characteristics and their Likert scale responses. To describe the short free-form comments, we used a template analysis approach10 and word-processing software. Our initial template was based on the traditional components of the research report. The coding team included an experienced family physician-researcher (United States), an early career family physician with research PhD (Australia), a final-year medical student entering a clinical-PhD program in PC (Netherlands), and a final-year medical student (Australia). We each categorized all comments from the first open-ended question and refined the category list through discussion. Then 2 researchers independently categorized comments from each of the open-ended questions and the team met to discuss and resolve any differences. We added or coalesced categories as needed to include factors that emerged from the data. Using the revised category list and an iterative process, we repeated the process for each question and the comments of all respondents, with each comment reviewed by at least 2 team members and discussions to resolve any differences. The entire investigator group offered final feedback on the analyses. After we identified the needs voiced by survey respondents, we reviewed published reporting guidelines to check if they address similar areas of concern. Focusing on EQUATOR Network guidelines on research methods most commonly used in PC research, 2 reviewers identified components that seem to address concerns about research reporting similar to those voiced by our respondents. (See Appendix 2 for details of methods and results.) This study was granted a waiver by the Human Subjects Division of the University of Washington, Seattle, WA. The survey was anonymous, and participants gave informed consent when they proceeded with the online survey. ## Results Our survey yielded 255 respondents from 24 nations, including: 54.1% (138) women, 64% (159) with doctoral degrees, 60% (169) physicians, 11% (32) scientists, 7% (20) educators, and 6% (18) public health professionals. Just over half were from North America, 55.6% (132), with 20% (47) from Australia, 15.6% (37) from Europe, and 28.7% (68) from other countries. (See Table 1.) View this table: [Table 1.](http://www.jabfm.org/content/34/1/12/T1) Table 1. Characteristics of Survey Respondents ### Difficulties with Research Reports Respondents reported the frequency of experiencing difficulty when using reports of both general health research and PC research (see Table 2 and Appendix 3 for details). Here, we focus on areas in which respondents reported it is “difficult at least half the time.” Overall, 74 of respondents who answered the question (37.4%) said that PC research reports caused problems for their work. Fully 58% (n = 109) found difficulty synthesizing findings across studies, and 83 respondents (41.9%) found difficulty assessing generalizability. Many found reporting to be insufficient for specific methods: 49% (n = 84) for qualitative research, 46% (n = 75) for mixed methods, and 38% (n = 65) for cohort/observational research. The elements that were most commonly reported as missing were the theoretical basis of research (54%, n = 87); description of teams, roles and organizations of care (53%, n = 86); and how patients were involved in the research process (53%, n = 78). View this table: [Table 2.](http://www.jabfm.org/content/34/1/12/T2) Table 2. Areas of Primary Care Research Reports Where Respondents Encounter Problems “about Half or More of the Time” Lower percentages of respondents cited problems with other aspects of PC research reports but over 20% of respondents noted problems with most aspects of PC research reporting “about half or more than half of the time” (Table 2.) Fewer respondents indicated insufficiencies in the reporting of the role of funders (21%), potential conflicts of interest (18%), ethical conduct of research, and institutional approval (7%). ### Respondent Comments Respondent comments about the reporting of PC research are organized into categories and subcategories, summarized in Table 3, with exemplar quotations. They generally followed the stages of the research process and the conventional format of research reports. View this table: [Table 3.](http://www.jabfm.org/content/34/1/12/T3) Table 3. Categories of Comments on Reporting of PC Research One observation ran through the comments of many respondents across the questions: PC is different. Many respondents emphasized that PC—practice, research and research reporting—is different from other health care and medical practice. > “There is a tendency for PC research to be more likely to involve multimorbidity, multiple disciplines, social determinants of health, and community-based sampling. (FP; clinician, editor, reviewer, manager, researcher; Australia; M) > “Purely because it is setting specific and refers to a much broader population than specialty care.” (Health services researcher; community member/patient, reviewer, methodologist, researcher; UK; F) > “PC has many contexts, types of practitioners and also takes patients into account-patient-centered care and factors in multimorbidities and preventative medicine. It is much more complex than hospital care which mostly is single health issue with a fairly passive patient.” (FP; advanced researcher/educator; Australia; F) A few respondents did not see much difference between the reporting of PC research and medical research in general. > “I don’t really think the reporting is much different to equivalent research designs in other settings. It’s just that there are few randomized controlled trials (RCTs) in PC settings, so often the findings are descriptive.” (FP; advanced researcher/educator; Australia; F) A few questioned the need for a new reporting guideline for PC research. > “None of the above seem unique to PC research in any way and are covered already in standards and journal requirements.” (FP; advanced researcher/educator; USA; M) One respondent worried that PC research was too broad to lend itself to a reporting guideline. > “I am not sure of the value of looking for basic consistencies across PC research when the field is so big, eclectic and covering a huge range of topics, methods and contexts. Sometimes reporting will be good sometimes not.” (FP; advanced researcher/educator; New Zealand; M) However, some highlighted the need for specific guidance for PC research. > “It is not much different now but needs to be. Given the complexity of the intervention and of the patients, we need to know far more details of the research than are usually reported.” (FP; advanced researcher/educator, reviewer; Canada; M) One respondent called for PC research to lead the way in improving the conduct and reporting of medical research. > “There is a more fundamental problem in medical publication than PC. As Ionnaidis has pointed out, most published medical research findings are most likely false. The poor study designs, misinterpreted analyses, small or unrepresentative sample sizes, bias due to industry or academic reputation, and outright fraud to achieve publication are some of the reasons that “the evidence (for most of medicine) sucks.” Shame on us if we perpetuate these inadequacies in PC.” (FP; advanced researcher/educator, policymaker; USA; M) ### Other Reporting Guidelines Using these comment categories summarized in Table 3, we scanned the EQUATOR Network reporting guidelines most relevant to PC research and found that many of the concerns voiced by our respondents are not adequately addressed by currently published guidelines (Appendix 2). ## Discussion This is the first survey published on user experience with PC research reports. We identified opportunities for improvement, some specific to PC and others applicable to health research more generally. The PC researchers we surveyed reported concerns about the ways medical research is reported and they identified areas where PC research deserves special attention to issues often not well reported in medical research. These included theoretical foundations, the context of interventions and care, and patient-clinician and team relationships. Respondents recommended changes for improving the reports of PC research to make them more valid, useful, generalizable, and applicable in practice. Our findings suggest that changes in reporting format and dissemination strategies will be needed to meet these needs. Most respondents (52%) want better description and documentation of the involvement of patients and communities in the studies reported, citing problems half or more of the time. This may reflect a commitment to participation and partnership in the research process among the PC research community. Optimizing PC research reports—their use, translation and application—is essential if we are to realize the potential of PC research to empower the translation of new knowledge into improved patient care and health outcomes through more effective application of findings into routine PC practice.1 These findings add to the growing literature on deficiencies with the reporting of research across a variety of research fields.11 Our findings should not be interpreted to suggest that PC research reporting is more or less problematic than research in other fields. We are seeking to understand how to help investigators, reviewers, and editors improve the reporting of PC research for all its many users working in diverse settings.12 Guidance that focuses on the issues of particular concern in PC—context, relationships, theory, and applicability—may also offer insights to help improve the reporting of health research more broadly. Our survey focused on identifying potential difficulties and did not document the strengths of current research and reporting practices in PC. Our study also focused on the content of published reports of PC research. Further research can explore the best alternative formats and dissemination strategies to make research findings most accessible to the full range of users, including practitioners, patients, and policy makers. This study has limitations associated with online surveys, informal sampling methods, Likert scales and free-text responses. We specified definitions for general health and PC research, but some respondents may have used other designations or had difficulty differentiating these 2 categories. Likert scales may lead to blunting of answers, but we did not observe ceiling effects. Our questions about the frequency of encountering problems may not be the most sensitive way to measure users’ satisfaction and experience with research reports. The short comment format did not allow for deep questioning of participants about the topic. However, we had more than 300 free-text responses with many participants writing in detail about their experiences and concerns. The long questionnaire risked survey fatigue, as a few respondents noted in their comments. We observed some fall-off in response to later questions, but we calculated all answer percentages using the denominators of responses to each question (see Table 2 and Appendix 3). We successfully engaged an expert group of producers and users of PC research. Respondents were mostly doctorally qualified researchers, so their responses may not be representative of the broader population using reports of PC research, including clinicians, policy makers and patients. Our respondents cannot be considered representative of all individuals and groups involved in PC research. With our purposeful and snowball sampling methods, we intentionally sought broad participation and inclusive numerators at the expense of defined populations and precise denominators. This approach served our purpose of capturing diverse experiences and wide-ranging suggestions to inform our Delphi process. We are currently analyzing a companion survey more focused on an international, interprofessional community of practicing PC clinicians. Having identified areas for improvement, we believe there is a role for additional guidance for researchers, authors, and journals to improve the usefulness and applicability of PC research reports. Although existing reporting guidelines help with specific methods used in PC research, none adequately addresses the concerns specific to PC patients, problems, and settings, or the rich context of both research and patient care. Specific areas for improvements in reporting and for new guidance tailored to the needs of PC research are suggested by the categories listed in Table 3. Using data from the current survey and our planned practitioner survey, we plan to conduct a Delphi study to help distill these concerns and suggestions into a priority list of consensus items to help optimize the reporting of PC research. ## Conclusions The findings of this international, interprofessional survey of PC researchers highlight the challenges encountered in interpreting, synthesizing and applying findings in the complex world of PC. Our findings suggest there is a role for added guidance to make reports more valuable to the many users of PC research. ## Acknowledgments We thank our colleagues around the world who completed and helped disseminate this survey. We thank Ms. Trudy Hong (Monash University, Melbourne, Australia) for her help with data analysis. ## Appendix 1. Online Questionnaire on Reporting of Primary Care Research, 2018 to 2019 CONSENSUS REPORTING ITEMS FOR STUDIES IN PRIMARY CARE Needs Assessment Survey Thank you for contributing to the Consensus Reporting Items for Studies in Primary Care (CRISP) through this Needs Assessment Survey. Our goal is to identify common and important shortcomings in the reporting of primary care research to help improve the quality, reporting and application of research to improve primary care. At this stage, we seek your expertise and opinions about the ways researchers could improve the way they report the research they do in, on and about primary care. The results of this survey will be collated and analyzed by the CRISP team to determine the most common and most important issues with current reporting of primary care research. These results will be used to inform our Delphi study to develop consensus guidelines for reporting primary care research. Your responses to this survey will be anonymous. Your participation is entirely voluntary and you can skip any questions or quit at any time. This study has been reviewed and exempted by the Human Subjects Division of the University of Washington, Seattle, WA, USA. After completing this short questionnaire, you will have the opportunity to volunteer to be an important part of our Delphi Group which will work to develop a consensus list of reporting items in primary care. Thank you. ![Figure1](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/34/1/12/F1.medium.gif) [Figure1](http://www.jabfm.org/content/34/1/12/F1) ![Figure2](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/34/1/12/F2.medium.gif) [Figure2](http://www.jabfm.org/content/34/1/12/F2) ![Figure3](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/34/1/12/F3.medium.gif) [Figure3](http://www.jabfm.org/content/34/1/12/F3) ![Figure4](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/34/1/12/F4.medium.gif) [Figure4](http://www.jabfm.org/content/34/1/12/F4) ![Figure5](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/34/1/12/F5.medium.gif) [Figure5](http://www.jabfm.org/content/34/1/12/F5) ![Appendix 2.](http://www.jabfm.org/https://www.jabfm.org/content/jabfp/34/1/12/F6.medium.gif) [Appendix 2.](http://www.jabfm.org/content/34/1/12/F6) Appendix 2. Table A. Comparison of published reporting guidelines with the categories of concern about primary care research reports expressed by survey respondents* Dr William R. Phillips, MD, MPH, FAAFP University of Washington, Seattle, WA. USA wphllps@uw.edu Dr Liz Sturgiss, BMed, PhD, FRACGP, MPH, MForensMed The Australian National University, Canberra, AUST Co-Conveners, CRISP elizabeth.sturgiss@anu.edu.au. ## Appendix View this table: [Appendix 2. Table B.](http://www.jabfm.org/content/34/1/12/T4) Appendix 2. Table B. EQUATOR Guideline references. ## Appendix 3 View this table: [Table5](http://www.jabfm.org/content/34/1/12/T5) Respondent Ratings of Frequency of Encountering Problems with the Reporting of Primary Care Research ## Notes * This article was externally peer reviewed. * *Conflict of interest:* None. * To see this article online, please go to: [http://jabfm.org/content/34/1/12.full](http://jabfm.org/content/34/1/12.full). * *Funding:* None. * To see this article online, please go to: [http://jabfm.org/content/34/1/000.full](http://jabfm.org/content/34/1/000.full). * Received for publication May 29, 2020. * Revision received July 30, 2020. * Accepted for publication July 31, 2020. ## References 1. 1.van Weel C, Rosser WW. Improving health care globally: a critical review of the necessity of family medicine research and recommendations to build research capacity. Ann Fam Med 2004;2: S5–S16. [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTI6IjIvc3VwcGxfMi9zNSI7czo0OiJhdG9tIjtzOjE5OiIvamFiZnAvMzQvMS8xMi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 2. 2.Kidd M. The importance of being different: inaugural Dr. Ian McWhinney lecture. Can Fam Physician 2015;61:1033–8. [FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiY2ZwIjtzOjU6InJlc2lkIjtzOjEwOiI2MS8xMi8xMDMzIjtzOjQ6ImF0b20iO3M6MTk6Ii9qYWJmcC8zNC8xLzEyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 3. 3.Simera I, Moher D, Hirst A, Hoey J, Schulz KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR network. BMC Med 2010;8:24. [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1186/1741-7015-8-24&link_type=DOI) [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=20420659&link_type=MED&atom=%2Fjabfp%2F34%2F1%2F12.atom) 4. 4.Altman DG, Simera I. A history of the evolution of guidelines for reporting medical research: the long road to the EQUATOR network. J R Soc Med 2016;109:67–77. [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1177/0141076815625599&link_type=DOI) [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=26880653&link_type=MED&atom=%2Fjabfp%2F34%2F1%2F12.atom) 5. 5.Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet 2009;374:86–89. [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1016/S0140-6736(09)60329-9&link_type=DOI) [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=19525005&link_type=MED&atom=%2Fjabfp%2F34%2F1%2F12.atom) [Web of Science](http://www.jabfm.org/lookup/external-ref?access_num=000267768800037&link_type=ISI) 6. 6.Shamseer L, Hopewell S, Altman DG, Moher D, Schulz KF. Update on the endorsement of CONSORT by high impact factor journals: a survey of journal “Instructions to authors” in 2014. Trials 2016;17:301. [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1186/s13063-016-1408-z&link_type=DOI) [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=27343072&link_type=MED&atom=%2Fjabfp%2F34%2F1%2F12.atom) 7. 7.Orkin AM, Phillips WR, Stange KC. Research reporting guidelines and the new annals instructions for authors. Ann Fam Med 2016;14:500–501. [FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTQvNi81MDAiO3M6NDoiYXRvbSI7czoxOToiL2phYmZwLzM0LzEvMTIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 8. 8.Moher D, Simera I, Schulz KF, Hoey J, Altman DG. Helping editors, peer reviewers and authors improve the clarity, completeness and transparency of reporting health research. BMC Med 2008;6:13. [CrossRef](http://www.jabfm.org/lookup/external-ref?access_num=10.1186/1741-7015-6-13&link_type=DOI) [PubMed](http://www.jabfm.org/lookup/external-ref?access_num=18558004&link_type=MED&atom=%2Fjabfp%2F34%2F1%2F12.atom) 9. 9.1. Lewis-Beck MS, 2. Bryman A, 3. Liao T Snowball sampling. In: Lewis-Beck MS, Bryman A, Liao T, eds. The SAGE encyclopedia of social science research methods. Thousand Oaks, CA: SAGE Publications; 2004. 10. 10.1. Crabtree BF, 2. Miller WL Using codes and code manuals: a template organizing style of interpretation. In: Crabtree BF, Miller WL, eds. Doing qualitative research. 2nd ed. Newbury Park, CA: SAGE Publications; 1999. 11. 11.Simera I, Kirtley S, Altman DG. Reporting clinical research: guidance to encourage accurate and transparent research reporting. Maturitas 2012;72:84–87. 12. 12.van Weel C, Tamblyn R, Turnbull D. Variation matters and should be included in health care research for comparison of outcomes. Prim Health Care Res Dev 2017;18:183–187.