Retracted papers originating from paper mills: cross sectional study - The BMJ

Study plan and information collection

We conducted a transverse sectional investigation of each papers retracted for being insubstantial mill papers, from 1 January 2004, the twelvemonth of work of the archetypal insubstantial mill insubstantial identified, until 26 June 2022, the day erstwhile we past accessed the database. These papers were identified via the Retraction Watch database,9 utilizing the filter “Reason for retraction” and choosing the enactment “Paper mill.” We included each papers retracted for this crushed and excluded papers bearing an look of concern, wherever technological misconduct had not been confirmed.

All the variables of involvement were collected and stored successful a purposely designed database. To behaviour this study, we utilized 3 main information sources: the Retraction Watch database, Web of Science, and Journal Citation Reports (both belonging to Clarivate Analytics). Additionally, we consulted the afloat substance of the papers included to grounds accusation related to the characteristics of the paper, specified arsenic the day of submission and publication, authors’ connection of funding, and competing interests.

Retraction Watch database

Retraction Watch tracks technological publications, careless of language, that person been retracted and aggregates them into a publically disposable database, including antithetic variables of involvement extracted by their staff. This database includes much than 30 000 retractions and expressions of concern.9

The Retraction Watch database has higher sum of retractions than PubMed and CrossRef due to the fact that these databases usage antithetic sources to observe retracted articles and notices of retraction. The main sources for the recognition of retractions are publishers’ and editors’ websites, but reports of technological integrity investigations, societal media sites, and tips from their blog followers are besides checked. Staff astatine Retraction Watch usage PubMed and Web of Science to treble cheque the retractions. To place retractions, unit astatine Retraction Watch tally protocolised manual searches regular utilizing keywords specified arsenic “retraction,” “withdrawn,” oregon “retracted paper.”

Retraction Watch uses chiefly the accusation included successful the announcement of retraction to classify retractions into antithetic reasons. Retraction Watch besides manually checks different sources for clarification of information, specified arsenic organization probe reports and US Office of Research Integrity reports.

In the circumstantial lawsuit of insubstantial mill products, Retraction Watch’s recognition is based connected respective indicators. One is the announcement of retraction, immoderate intelligibly authorities that the insubstantial is from a insubstantial mill, others usage a euphemism for insubstantial mill specified arsenic “third enactment editing service.” In different cases, journals and publishers retract a ample fig of articles accompanied by an editorial indicating that the retracted papers were from insubstantial mills. These editorials usually usage a akin language, stating that the insubstantial “resembles antithetic papers from antithetic authors.” Retraction Watch besides uses PubPeer and the database of probable insubstantial mill papers published by Elisabeth Bik and different investigators.11

We sourced the full fig of papers retracted for immoderate crushed per twelvemonth and the full fig retracted for originating from insubstantial mills per year. For each insubstantial retracted for the crushed of originating from a insubstantial mill, we collected the following: rubric of paper; fig of authors; authors’ affiliated country; archetypal author’s institution; benignant of instauration of archetypal writer (hospital, university, probe centre); and paper’s day of work and day of retraction. The past entree to the database was 26 June 2022.

Web of Science

We retrieved the full fig of papers published per twelvemonth crossed the survey play (1 January 2004 to 26 June 2022) with nary exceptions. For each insubstantial included, we collected the full fig of citations received, some earlier and aft retraction, from day of work until 26 June 2022.

Journal Citation Reports

We gathered information for the diary that published each insubstantial and its characteristics, specified arsenic its name, Journal Citation Reports interaction factor, Journal Citation Reports category, and comparative presumption by Journal Citation Reports quartile (ie, the highest interaction origin of journals belonging to the archetypal quartile (or Q1) and the lowest interaction origin journals to the 4th quartile (or Q4)), and work modality (open entree oregon subscription). Where the diary was included successful much than 1 category, we chose the highest presumption according to the diary interaction factor. We categorised hybrid journals arsenic non-Open Access journals.

Statistical analysis

We utilized a descriptive investigation of the characteristics of the retracted papers identified, by notation to the variables of interest, with continuous variables being expressed arsenic median and interquartile range, and categorical variables arsenic implicit fig and comparative frequency.

We calculated the work complaint of papers that were retracted due to the fact that of insubstantial mills per 100 000 papers published successful a fixed year, implicit the full fig of papers published for the aforesaid year. Therefore, we assessed the proportionality of papers mill publications regarding the full fig of publications successful each twelvemonth of the survey period. Additionally, we calculated the percent of insubstantial mill papers retracted per year, with respect to the full fig of retractions per year, to ascertain the proportionality of insubstantial mill retractions compared with retractions for immoderate different reason.

We described the organisation of this benignant of papers by Journal Citation Reports class of the diary successful which they were published. We created a ranking of journals and publishers based connected the fig of retracted insubstantial mill papers they published during the survey period. We person determined if the diary of work was reliable, that is, not suspected of being a predatory journal, successful the subsample of the technological journals that published the astir retracted insubstantial mill papers. To measure the reliability of a journal, we utilized the ThinkCheckSubmit checklist (https://thinkchecksubmit.org/journals).

We calculated the clip elapsed betwixt the paper’s submission and work and the clip elapsed betwixt the paper’s work and retraction, successful days. Analysis of the times elapsed betwixt submission and work and the times betwixt work and retraction were stratified by the Journal Citation Reports quartile of the diary successful which the insubstantial was published. Similarly, we analysed the full citations received by the papers, some wide and stratified by quartile. All statistical analyses were done utilizing the Stata mentation 17.0 machine bundle programme.

Patient and nationalist involvement

This probe was done connected statement with Retraction Watch, wherever we committed to usage a database nether circumstantial circumstances, including confidential uses, and to debar sharing the downloaded database with 3rd parties. We did not trial immoderate especific wellness involution oregon drug, therefore, we did not deliberation that diligent oregon nationalist engagement would beryllium adjuvant successful this research.

Principal findings

Our transverse sectional investigation of each papers retracted for originating from insubstantial mills until June 2022, identified from the Retraction Watch database, suggests that these insubstantial mill retractions are expanding successful frequency. Nearly each authors of these papers came from China and were predominantly affiliated with hospitals. The median clip for retraction of a insubstantial mill insubstantial was adjacent to 2 years and accrued with the ranking of the diary successful which it was published, truthful that the higher the Journal Citation Reports interaction factor, the shorter the play until retraction. These papers impact morganatic journals and does not look to beryllium exclusive to predatory journals. Furthermore, this survey showed the interaction and visibility of these retracted papers due to the fact that immoderate were highly cited, with the imaginable consequences that this entails. To our knowledge, this is the archetypal survey to analyse the increasing improvement of insubstantial mill retractions and their characteristics.

Our findings suggest that the work of insubstantial mill papers accrued betwixt 2017 and 2019, erstwhile astir 5 to 10 were published and yet retracted for this crushed per 100 000 publications. In 2020, the fig of identified insubstantial mill papers published successful the technological lit fell sharply. This alteration whitethorn person occurred for a fig of reasons. Firstly, papers published betwixt 2020 and 2022 that mightiness yet beryllium identified for retraction person not yet been identified oregon retracted. Retraction of a insubstantial takes a agelong time, and much retractions volition perchance look successful the future. Secondly, arsenic a effect of investigations initiated successful aboriginal 2020 by a fig of editors and researchers,12 the technological assemblage person go alert of the problem, and guidelines person been published to assistance editors place specified papers.4 Even though these guidelines bash not alteration a insubstantial mill insubstantial to beryllium unequivocally recognised, they bash marque screening and recognition of papers originating fom insubstantial mills possible. Hence, numbers mightiness beryllium smaller than would person been due to the fact that technological journals person improved methods for their identication during editorial reappraisal and adjacent review, thereby preventing their publication. Thirdly, the accrued attraction to this benignant of fraud mightiness besides person deterred authors from engaging the services of insubstantial mills, due to the fact that of the consequences of technological fraud, particularly successful immoderate countries specified arsenic China.13 Then again, an accrued vulnerability could person caused insubstantial mill organisations to alteration their mode of operation, frankincense hindering detection.14

Although this contented is comparatively new, peculiarly successful America and Europe, for immoderate years present the usage of these types of organisations has been wide successful different countries, specified arsenic China.1015 China encouraged its researchers to people papers successful instrumentality for wealth and vocation promotions.16 Furthermore, aesculapian students astatine Chinese universities are required to nutrient a technological insubstantial successful bid to graduate.15 In fact, these organisations openly advertise their services connected the Internet and support a beingness connected assemblage campuses, not lone successful China but besides successful different countries, specified arsenic Russia.815

Perhaps unsurprisingly, astir papers retracted for being insubstantial mill papers travel from that aforesaid country. These results are successful enactment with the findings of different researchers and editors of technological journals, though insubstantial mill papers person been reported successful different countries, specified arsenic Iran oregon Russia.81217 The enactment of the largest insubstantial mill organisation successful Russia named International Publisher has precocious been acknowledged.810 Although this insubstantial mill has published astir 1000 papers, its ain website announces that much than 5000 authors person bought the coauthorship of astatine slightest 1 paper.8

Also, we enactment that astir authors of identified insubstantial mill papers were infirmary affiliated, which is accordant with erstwhile research.15 The main crushed for this mightiness beryllium that Chinese doctors are not affiliated with aesculapian schools, but with hospitals. Of note, unit to people is greater successful biomedical sciences than different specialties and publications are usually needed to get a assemblage grade oregon a promotion successful China.15

Most insubstantial mill papers were published successful pharmacy and objective medicine journals, but galore of them were published successful basal subject journals arsenic well, specified arsenic cellular and molecular biology oregon biochemistry. Therefore, this occupation not lone affects objective medicine areas. This probe has not focused successful analyzing specifically if insubstantial mill papers are published much often connected objective medicine topics oregon basal research. We are of the sentiment that this facet should beryllium further analysed. According to our results, nary large variations implicit clip person been observed successful the topics covered by the insubstantial mill papers truthful far. However, the latest COPE study indicates that this signifier could change, for illustration successful taxable areas oregon types of journals, implicit time.18

The main occupation which insubstantial mill papers airs for editors and reviewers of technological journals is the trouble of identifying them done the adjacent reappraisal process due to the fact that the papers look to beryllium legitimate. Analysis of images successful a manuscript has been identified arsenic 1 of the imaginable strategies for detecting insubstantial mill papers due to the fact that astir images thin to beryllium manipulated oregon duplicated, oregon both.14 Although antithetic softwares are susceptible of detecting representation manipulation, insubstantial mill papers often usage duplicated images (or banal images)519 due to the fact that they are much hard to observe than manipulated ones. At present, nary bundle is susceptible of detecting representation duplication successful a reliable way, frankincense leaving this task to editors and reviewers. That said, however, not each papers incorporate images that let for scrutiny. Another strategy for screening questionable papers is the Problematic Paper Screener software. This bundle identifies alleged “tortured phrases,”—that is, antithetic phrases alternatively of established ones, which mightiness beryllium an indicator of suspected technological misconduct.20 Also, COPE has published a database of communal indicators for insubstantial mill papers that could service arsenic a screening instrumentality for suspicious articles.18

With the purpose of preventing and detecting technological misconduct, immoderate countries already person offices and circumstantial bodies that code aspects relating to technological integrity, but galore others bash not person structures of this type.21 Countries that person nary assemblage oregon policies governing technological misconduct incur a higher hazard of producing fraudulent papers.22 Countries specified arsenic Denmark, Sweden, and China, person passed laws against technological fraud. Ironically, China has the astir terrible penalties for probe fraud. The paucity of consequences that technological misconduct has historically had successful this state mightiness person played an important relation successful the summation successful unethical behaviour, including the usage of insubstantial mills.15 In 2018, aft a fig of scandals successful China, the instrumentality against technological fraud was strengthened by imposing sanctions that spell beyond the purely world and occupational sphere.23 This tougher attack appears to person started yielding results and, successful December 2021, much than 300 researchers were reportedly penalised for technological misconduct. Among different things, the penalties included revocation of world degrees and cancellation of promotions.24 Because practically each insubstantial mill papers travel from China, these caller penalties argumentation mightiness person contributed to the simplification successful the fig of insubstantial mills since 2020.

Strengths and limitations

This survey had limitations. Retractions of insubstantial mill papers proceed implicit time. Because of this, our probe volition request to beryllium updated implicit clip arsenic the conclusions could good alteration arsenic the database of retractions grows. The characteristics of retracted and non-retracted insubstantial mill papers tin differ, which could explicate wherefore immoderate papers were identified but not others, though each correspond fraudulent science. Another regulation was the trouble successful assigning the origin of retraction successful immoderate cases, hence misclassification is simply a risk. In this study, we person included formally retracted insubstantial mill papers, not taking into relationship suspicious papers (ie, those from the database elaborated by EB and others) and this mightiness beryllium a regulation of the contiguous research. However, the inclusion of papers not formally retracted mightiness incur successful a hazard of misclassification of those papers if they are not yet retracted arsenic insubstantial mill products. A regulation regarding the citation investigation is that citations earlier and aft retraction person not been differentiated successful this survey and this contented should beryllium considered successful aboriginal research.

The main spot of this survey was the usage of the Retraction Watch database to place retracted insubstantial mill papers due to the fact that this root is the main database connected retractions and should presently beryllium considered arsenic the golden modular for aggregated accusation connected retracted articles. The Retraction Watch database has 3 times the sum of PubMed and 5 times the sum of CrossRef (Retraction Watch, idiosyncratic communication, 2022). Taking this into account, we see that the fig of missing retractions should beryllium minimal.

Conclusions

The insubstantial mill papers that we person identified arsenic retracted to day perchance correspond lone a tiny fig of insubstantial mill papers successful full due to the fact that perchance thousands of these papers could person been published successful the technological lit and not yet identified nor retracted. Some editors of planetary technological journals person begun to systematically place and retract insubstantial mill papers, which has led to wide retractions.2526 The emergence of insubstantial mills is simply a caller ethical occupation successful probe and, much specifically, successful work ethics. Not lone does this contented entail the merchantability of authorship, but these types of papers person besides been observed to incorporate fabricated and manipulated information and images, frankincense disseminating mendacious results successful technological literature. Efforts indispensable beryllium accrued to forestall the usage of these insubstantial mill organisations, opening with improved acquisition successful morals and technological integrity for editorial committees of technological journals, students, and researchers.

Ethical approval

Because this survey utilized publically disposable materials and did not impact humans, morals committee support was not required.