Original Investigation

How ChatGPT writes scientific titles in medical research: structural and content differences compared to human authors

Paul Sebo¹, Bing Nie², Ting Wang³

doi: http://dx.doi.org/10.5195/jmla.2026.2266

Volume 114, Number 3: 222-237
Received 06 2025; Accepted 04 2026

ABSTRACT

Objective:

Scientific article titles play a central role in shaping research visibility, interpretation, and discoverability. With the rise of large language models like ChatGPT, there is growing interest in using AI tools to support title generation, yet little is known about how AI-generated titles differ from those written by human authors. This study compared titles written by original authors to those generated by ChatGPT-4.0 from the same abstracts, focusing on structural and content-level differences.

Methods:

Fifty research articles published in 2000, before the advent of generative AI, were randomly selected from ten high-impact general internal medicine journals. For each, the structured abstract was submitted to ChatGPT-4.0 using a standardized prompt to generate a title. Human-written and AI-generated titles were then compared using quantitative measures (word and character counts, punctuation marks) and descriptive content analysis (study design, population descriptors, outcome emphasis, public health or clinical framing, temporal context). Paired statistical tests were applied to assess differences.

Results:

ChatGPT-generated titles were significantly longer than human-written titles (median 16 vs. 12.5 words, p-value<0.001) and included more characters and punctuation marks (colons in 100% vs. 30%; p-value<0.001). AI titles more often specified or clarified study design, detailed populations, emphasized outcomes, framed findings in public health or clinical terms, and incorporated temporal context.

Conclusion:

ChatGPT-4.0 produces more explicit and structured titles than human authors, emphasizing methodological clarity and content completeness. These findings raise important questions about norms in scientific communication and the need for further research and ethical guidance on AI-assisted writing.

Keywords: Article Title; Artificial Intelligence; AI; ChatGPT; Language Model; Medical Publishing; Scientific Writing.

INTRODUCTION

The title of a scientific article plays a critical role in shaping how research is discovered, interpreted, and disseminated. It is often the first, and sometimes the only, element of a paper that is read, and its structure may affect visibility in indexing databases, search engines, and citation rates [1–8]. An effective title must balance clarity, informativeness, and conciseness [4, 5, 9]. Yet despite its importance, there is no definitive consensus on what constitutes a “good” scientific title, nor on which elements, such as study design, population, outcomes, or rhetorical framing, should be prioritized. Preferences vary by discipline, audience, journal policy, and cultural norms, leaving considerable ambiguity around best practices in title construction [4, 5, 9–12]. In addition, journal editors and reviewers often provide inconsistent guidance [13], and empirical evidence on title effectiveness remains limited.

For health sciences librarians and information professionals, titles are central to how research is found and used. They affect how articles are retrieved in bibliographic databases, discovered in institutional repositories, and interpreted during literature searching. In academic and health sciences settings, librarians help users navigate these variations by teaching how article title wording can influence database retrieval and relevance ranking and by helping refine search strategies when results are overly broad, narrow, or inconsistent. As AI-assisted writing tools become increasingly integrated into scholarly workflows, systematic differences in how titles are constructed may influence indexing patterns, search performance, and research visibility. Understanding whether AI-generated titles differ structurally or linguistically from human-written titles is therefore relevant to librarians’ roles in supporting effective evidence retrieval and responsible AI use in scholarly communication.

Against this backdrop, the emergence of large language models (LLMs), including OpenAI’s ChatGPT, has introduced new tools that can support scientific writing tasks [14–17]. LLMs are trained on vast corpora of texts and can generate fluent, coherent, and contextually appropriate language. They have been studied in several domains of academic writing, including summarizing articles or drafting abstracts [14,18–21], and generating reviewer reports [22,23]. Existing studies show that LLMs can produce coherent scientific text, although concerns remain regarding factual reliability, methodological precision, and the risk of undetected errors or hallucinations [24–26]. Only a small number of studies have examined LLM-generated scientific titles [27,28]. Chen and Eger [27] explored abstract-to-title generation using fine-tuned transformer models on large-scale natural language processing and machine learning datasets, reporting that their best end-to-end systems performed comparably to human-authored titles in human evaluation, although humor-oriented title generation remained substantially more challenging. Rehman et al. [28] fine-tuned multiple pre-trained language models for title generation and found that PEGASUS-large outperformed both general-purpose LLMs and other fine-tuned models across automatic evaluation metrics, while zero-shot GPT-3.5-turbo produced generally accurate and creative titles but did not consistently surpass task-specific fine-tuned models. Together, these studies suggest that transformer-based models can generate fluent and contextually appropriate titles from abstracts, and that fine-tuned models may outperform general-purpose LLMs on automated evaluation metrics; however, performance varies depending on domain, evaluation criteria, and model specialization.

Titles are a distinct genre of scientific writing: unlike manuscripts or abstracts, they are short, highly compressed forms of communication that function both as human-readable summaries and as elements that influence search and retrieval. As previously noted, their wording may affect keyword matching, indexing processes, and how effectively articles surface in bibliographic systems. Any systematic change in title construction, whether produced by humans or LLMs, may therefore affect how easily relevant studies are located or interpreted.

Given the growing interest in using LLMs for scientific writing, it is important to understand whether these models follow the same stylistic conventions as human authors or introduce systematic changes that may influence retrieval and discoverability. The existing literature on AI-generated manuscripts does not address this question, and results from longer free-text writing do not necessarily generalize to short, deliberately structured forms like titles.

To address this gap, the present study compares human-written titles of published scientific articles with titles generated by ChatGPT-4.0 from the same abstracts. Our aim is to identify systematic differences in structural, linguistic, and content-level features, without making normative judgments about which type of title is preferable.

Because titles influence how articles are located in bibliographic systems, these differences have practical implications for health sciences librarians who support evidence retrieval and research visibility. Understanding LLM titling patterns may help inform consultation practices, instructional strategies, and broader discussions about the responsible integration of AI-assisted writing in scientific communication.

METHODS

Study context and objectives

We conducted a paired comparative study evaluating structural, linguistic, and content features of titles written by original authors versus titles generated by ChatGPT-4.0 from the same abstracts. The objective was to identify systematic differences in format, style, and content emphasis between the two title types.

Dataset and article selection

To construct the dataset, we first identified the ten general internal medicine journals with the highest impact factors (IF) in the 2023 Journal Citation Reports (JCR). We focused on general internal medicine because it encompasses a broad range of clinical research topics and article types, making it representative of general medical publishing practices. Selecting ten journals provided sufficient diversity in editorial styles and article content while keeping the sample size manageable for detailed analysis. To ensure consistency and relevance across journals, only those fulfilling all of the following criteria were eligible for inclusion: (1) they publish original research articles and/or systematic reviews; (2) they use structured abstracts for both types of articles; and (3) they have been in continuous publication since at least January 2000. Based on these criteria, the following journals were selected: The Lancet (IF 98.4), The New England Journal of Medicine (IF 96.3), The BMJ (IF 93.7), JAMA (IF 63.5), Archives of Internal Medicine (IF 22.3), Annals of Internal Medicine (IF 19.6), CMAJ (IF 12.9), Journal of Travel Medicine (IF 9.1), Journal of Internal Medicine (IF 9.0), and Mayo Clinic Proceedings (IF 6.9).

From each journal, one researcher (PS) manually screened the table of contents and article metadata to identify eligible publications, defined as original research articles (reporting primary data and results) or systematic reviews published in the year 2000 with a structured abstract. Research letters, brief reports, and equivalent short-form formats were excluded. After identifying all eligible articles, we used computer-generated randomization to select five per journal, for a final sample of 50 articles. Because content analysis methodology does not prescribe fixed sample sizes for studies of short textual units such as titles [29,30], we selected 50 articles to balance diversity across journals with the feasibility of detailed manual analysis of title structure and content. The year 2000 was chosen to ensure that all original titles were written before the emergence of generative AI-assisted writing tools, eliminating any risk of contamination by machine-generated content. Random selection within each journal was conducted using a computer-generated random number algorithm applied to the list of all eligible publications. To avoid an unintended concentration of articles in a single topic area or study design, one researcher (PS) conducted an informal screening of abstracts and article metadata to verify that the selected articles covered a range of study types and subject areas. This screening was not a formal assessment of representativeness but rather a practical check to ensure basic diversity within the randomly drawn sample. Indeed, we did not further restrict inclusion by medical specialty or subject area, as our goal was to capture the broad range of research published across high-impact general medical journals rather than focus on specific clinical domains that might differ systematically in title conventions or terminology.

ChatGPT title generation protocol

For each selected article, the abstract was extracted and anonymized. All identifying metadata, including the original title, authors, affiliations, and journal, were removed. Each anonymized abstract was submitted in a new session to ChatGPT-4.0 using the same standardized prompt: “Write a title for this scientific article based on the abstract below”. Immediately after entering the prompt, we pasted the full abstract of the selected article.

Each submission was independent (no chat memory), and the outputs were not altered, except for standardizing capitalization: words were converted to lowercase when uppercase was not required (e.g., unless referring to names, countries, or other proper nouns). This setup aimed to reflect a realistic use case where a researcher inputs an abstract and retrieves a title without prompt engineering. The resulting human-written and ChatGPT-generated titles are presented in Table 1.

Structural, linguistic, and content-level comparison

For each abstract, the original title and the ChatGPT-generated title were compared on a set of predefined structural features: total number of words, total number of characters, and punctuation marks (colons, semicolons, question marks). These features were selected because they represent widely studied and quantifiable structural elements linked to variation in scientific title and impact, making them an appropriate initial basis for comparing human- and AI-generated title construction. Specifically, while longer titles are not inherently more visible in search systems, including more specific search terms can improve indexing and keyword matching in academic databases [31]. Similarly, punctuation in academic titles has le potential to attract specific groups of readers, but evidence regarding its impact on retrieval and citation is mixed [4,32].

In parallel, a descriptive coding framework was developed inductively to capture recurrent linguistic and content-level differences between the two types of titles. This framework was informed by prior studies describing common features of scientific titles, such as the inclusion of study design, outcomes, population descriptors, and framing elements [33–36]. However, as noted in the Introduction, there is no consensus on what constitutes a ‘good’ title or which elements should be prioritized. Against this background, we focused on the following thematic elements, which are frequently discussed in the literature and observed in practice: addition or clarification of study design, outcome emphasis, specification of the study population (population descriptors), terminology simplification, inclusion of specific intervention or diagnostic terms, public health or clinical framing, and temporal framing [34–37]. Each adjustment identified in the ChatGPT-generated title was cross-checked against the corresponding abstract to verify that it accurately reflected the study’s content. Each title was independently coded and verified by two reviewers (PS and TW) to ensure consistency and accuracy. Disagreements were resolved by discussion and consensus. The full text of the articles was not consulted for this study; all assessments were limited to information available in the abstract, as this was the source on which ChatGPT generated the titles.

Statistical analysis

All extracted data were first entered into Microsoft Excel (Microsoft Corporation, Redmond, WA, USA) and then exported to Stata for analysis. Structural variables were defined as total word count, total character count, and the presence of specific punctuation marks (colons, semicolons, question marks). Linguistic and content-level variables were defined according to the thematic framework described above (e.g., mention or clarification of study design, outcome emphasis, population descriptors, terminology simplification, intervention or diagnostic specification, public health or clinical framing, and temporal framing).

Descriptive statistics were reported as medians with interquartile ranges (IQRs) for continuous variables (word count, character count) and as frequencies with percentages for categorical variables (presence of colon, semicolon, or question format). Paired comparisons between human-written and ChatGPT-generated titles were conducted. The Wilcoxon signed-rank test was applied to continuous variables, while McNemar’s test was applied to categorical variables. Linguistic and content-level variables were summarized descriptively. All analyses were conducted using Stata version 15.1 (StataCorp, College Station, TX, USA). A two-sided p-value <0.05 was considered statistically significant.

Ethical considerations

This study used only publicly available research articles and did not involve human participants, patient data, or confidential information; therefore, ethics committee approval was not required. The use of ChatGPT complied with OpenAI’s terms of service.

RESULTS

The main results of the study are summarized in four tables. Tables 1 and 2 support the descriptive thematic analysis: Table 1 presents the original and ChatGPT-generated titles for each abstract, along with brief summaries of their key differences; Table 2 synthesizes the main thematic patterns observed across abstracts, corresponding to recurrent linguistic, content-level, and framing differences. Tables 3 and 4 support the quantitative analysis: Table 3 displays the raw structural data extracted from each pair of titles, while Table 4 reports descriptive statistics and p-values comparing human- and AI-generated titles.

Quantitative comparison of title structure

ChatGPT-generated titles were generally longer than those created by humans. The median word count for AI-generated titles was 16 (IQR: 14–18), compared to 12.5 (IQR: 11–15) for human-written titles (p-value <0.001). Similarly, the median number of characters was higher for ChatGPT titles (118; IQR: 109–126) than for human-written ones (94; IQR: 78–113; p-value <0.001). All ChatGPT-generated titles included a colon, compared to only 30% of human-written titles (p-value <0.001). Only two human-written titles were phrased as questions, compared to none among ChatGPT titles, which was not statistically significant.

Descriptive comparison of content and framing

Systematic differences were identified in the way ChatGPT-4.0 generated titles from abstracts, as summarized in Table 2. A prominent pattern was the frequent addition or clarification of methodological information, particularly through more explicit identification of study design (17 titles, 34%). Titles produced by ChatGPT also tended to emphasize outcomes or key findings more explicitly than their human-written counterparts (21 titles, 42%). For example, these titles included terms such as “increased incidence” (abstract 1) or detailed findings like “body mass index cut off points” (abstract 12). Another recurring modification was the clarification or expansion of the study population, with ChatGPT specifying age groups, geographic settings, or cohort names (e.g., “a defined UK region” in abstract 2 or “EURODIAB register” in abstract 3) (17 titles, 34%). Some AI-generated titles underwent terminology simplification, replacing technical or compound phrasing with simpler and more common wording (e.g., abstract 13: “single blind, randomized trial of efficacy and acceptability of oral picolax versus self-administered phosphate enema in bowel preparation for flexible sigmoidoscopy screening” simplified to “bowel preparation for flexible sigmoidoscopy: randomized trial of enema versus Picolax”) (2 titles, 4%). This pattern reflects ChatGPT’s general tendency to favor fluency and accessibility, consistent with prior observations that LLMs often prioritize readability over maintaining domain-specific precision [38–40].

Other AI-generated titles included the addition or clarification of specific interventions, drugs, or diagnostic elements, such as naming a drug like “xemilofiban” (abstract 9) or specifying “muscarinic receptors” (abstract 45) (6 titles, 12%). In several cases, the AI titles framed findings in terms of public health or clinical framing, introducing expressions like “implications for prevention” (abstract 27) or “missed opportunities for intervention” (abstract 47) (3 titles, 6%). Finally, the temporal framing was occasionally added or clarified, as seen in examples such as “1989–94” (abstract 3) or “one year follow up study” (abstract 14) (3 titles, 6%).

Together, these patterns illustrate ChatGPT’s tendency to produce more structured, explicit, and information-dense titles. All thematic additions and modifications introduced by ChatGPT accurately reflected the information contained in the corresponding abstracts. No instances of factual inaccuracy or misrepresentation were identified during reviewer verification.

DISCUSSION

Summary of key findings

This study revealed systematic differences between titles written by human authors and those generated by ChatGPT-4.0 from the same scientific abstracts. Across a matched dataset of 50 original medical research articles, ChatGPT-generated titles were consistently longer, more structured, and more explicit in content than their human-written counterparts.

Quantitatively, AI-generated titles included significantly more words and characters and systematically used colons to structure information. Descriptively, ChatGPT displayed consistent tendencies to specify or clarify the study design, emphasize key outcomes, clarify or expand population descriptors, frame results in terms of public health or clinical framing, and incorporate temporal context. Terminological simplifications and the addition of contextual details were also frequently observed. These two tendencies, adding contextual specificity while simplifying technical phrasing, suggest that ChatGPT favors titles that are simultaneously more explicit and more accessible.

However, the greater explicitness and structural regularity observed in ChatGPT titles came with trade-offs. Human-written titles often used concise phrasing, disciplinary shorthand, or more narrative or question-based structures that conveyed tone, emphasis, or conceptual stance, elements largely absent from AI-generated versions. Several ChatGPT titles also introduced more detail than may be appropriate under journal title-length constraints, and the consistent use of a two-part colon structure produced titles that were more formulaic and uniform. Thus, while AI-generated titles were more standardized and explicit, they may be less aligned with journal-specific style expectations and disciplinary conventions regarding conciseness, framing, and authorial voice.

Comparison with the literature

To date, a growing number of empirical studies have examined how LLMs perform in the task of scientific text generation [14–23,27,28]. Most published work has focused on broader applications of LLMs in academic writing, including abstract generation [14,18–21], literature review synthesis [41,42], and peer review drafting [22,23]. Within the more specific domain of title generation, very few studies have directly addressed this task [27,28]. To our knowledge, our work is the first to conduct a paired, descriptive, and quantitative comparison between human-written titles and ChatGPT-generated titles derived from the same scientific abstracts. By combining structural features with thematic analysis, this study offers a novel empirical contribution to understanding how LLMs address a critical yet underexplored component of scientific communication.

Chen and Eger (2023) evaluated the ability of transformer-based models, including ChatGPT, to generate scientific titles from abstracts in the fields of natural language processing and machine learning [27]. Their study emphasized stylistic dimensions, such as humor and novelty, and introduced the first large-scale dataset of humorous scientific titles. While some models (e.g., BARTxsum) approached human-level quality, capturing genuine humor remained a significant challenge. This contrasts with our findings: we did not evaluate creativity or humor but instead observed that ChatGPT produced more explicit, structured, and information-dense titles, often prioritizing methodological transparency over stylistic variation.

Rehman et al. (2024) assessed the capacity of LLMs to generate titles from abstracts, comparing the performance of fine-tuned models with large general-purpose models in zero-shot settings [28]. They report that a fine-tuned PEGASUS-large model outperformed GPT-3.5-turbo and LLaMA-3-8B on standard automated metrics, and human annotators often favored its outputs. While they noted that ChatGPT could generate creative titles in different stylistic modes, our study extends their findings by providing a paired, article-level comparison with human-written titles, highlighting not only structural features but also systematic differences in framing and emphasis.

Implications for practice and research

Our findings have implications for health sciences librarians, particularly in settings where they support researchers in developing effective search strategies and understanding how title wording may influence indexing and retrieval in bibliographic databases. Because titles are key elements used in bibliographic indexing and search algorithms, changes in title structure or terminology may directly affect how articles are classified and retrieved. In particular, AI-generated titles may increase retrieval performance in keyword-based systems by explicitly naming study designs, populations, and outcomes. Such information may facilitate controlled vocabulary indexing (e.g., MeSH in PubMed) and improve keyword matching in searchable title fields in bibliographic databases and platforms such as PubMed, Scopus, and Web of Science. Moreover, this approach aligns with the PICO framework commonly used in medical information retrieval [43,44]. However, broader or more inclusive labels may also lead to less precise retrieval. According to Manning et al., precision is defined as the proportion of retrieved documents that are relevant [45]. From an information retrieval perspective, titles containing a larger number of keywords may generate a greater number of indexed terms, which can broaden retrieval and increase the likelihood of spurious matches. In some cases, this may introduce additional noise and potentially reduce precision in search results. Given these possibilities, librarians can help researchers understand how AI-generated titles may influence indexing consistency and retrieval accuracy, offer targeted instruction on how title wording shapes search strategies and retrieval results, and contribute to institutional discussions about policies on AI-assisted writing.

Librarians are also well positioned to advise on disclosure practices and to monitor how AI-generated titles interact with bibliographic systems over time. As indexing increasingly incorporates algorithmic or AI-assisted components, including machine learning and LLM–based approaches to subject assignment and semantic analysis, the interaction between AI-generated titles and AI-driven indexing systems warrants attention. If titles are produced using models that systematically foreground study design, population descriptors, or outcome language, and if indexing systems likewise rely on automated term extraction or semantic pattern recognition, these features may be reinforced during the indexing process. Conversely, shifts in titling conventions may influence how AI-based indexing models interpret and classify content. Understanding this evolving interaction between AI-assisted writing and AI-assisted indexing will be important for maintaining accurate, transparent, and effective information retrieval in bibliographic systems.

The observed differences highlight the need for clearer and more explicit guidance as AI becomes integrated into scholarly writing practices. While many journals may implicitly favor brevity and certain stylistic conventions in titles, such as concise phrasing, rhetorical variation, or avoidance of overly formulaic structures, these expectations are often shaped by disciplinary convention rather than by formal policy. For health sciences librarians who support researchers in understanding publication norms, greater transparency around criteria for title length, punctuation, and descriptive structure would facilitate more consistent and informed guidance, particularly when AI-assisted tools are used in the writing process. Although titles are unlikely to determine acceptance on their own, they are highly visible elements of scholarly communication that influence how research is presented, interpreted, and retrieved. Clearer articulation of expectations surrounding AI-assisted titling would therefore assist librarians in their instructional roles.

Ethical considerations are also central. Current authorship and AI-use guidelines apply broadly to the writing process and rarely address titles explicitly [46–50]. However, because titles function as core bibliographic metadata and are used directly in indexing, retrieval, and search ranking, transparency about AI involvement is particularly relevant at the title level. Librarians increasingly engage in institutional conversations about research integrity, AI literacy, and transparency in scholarly communication. Awareness of how AI-generated elements appear in highly visible metadata fields such as titles may therefore inform local guidance, training initiatives, and policy discussions about responsible AI integration in academic writing.

Our findings highlight several areas where further evidence is needed to support librarians’ instructional and consultative work. While completeness and structure may improve information retrieval, they do not necessarily align with how titles influence impact. Several studies have shown that shorter titles and titles without colons are associated with higher citation rates and visibility [2–5,51], although some findings, such as those reported by Jacques and Sebire [1], as well as by Rostami et al. [52], suggest the opposite. Our findings therefore raise an important and unresolved question: since ChatGPT-generated titles are consistently longer, more structured, and more likely to contain colons, will they enhance visibility through improved indexing and keyword matching, or reduce citation potential due to lower readability and engagement? This paradox between completeness and conciseness underscores the open question of whether AI-generated titles will improve discoverability at the cost of readability or reader engagement. In addition, foregrounding design, outcomes, or populations may predispose readers to particular interpretations, narrowing perceived generalizability or overstating certainty. These framing effects reflect defaults of the model rather than deliberate authorial choices and may shape how research is received and cited. Addressing such risks will require both empirical study and clear editorial guidance.

For researchers, LLMs such as ChatGPT may serve as useful co-writing tools to enhance the completeness, precision, or discoverability of titles, particularly for early-career researchers or non-native English speakers [53]. ChatGPT-generated titles consistently included key structural features such as study design, population descriptors, and outcome emphasis. These features reflect established conventions in scientific titling that aim to convey clarity and specificity, but their inclusion does not necessarily imply greater effectiveness or appropriateness. However, such structural patterns may improve visibility by aligning with keyword-based retrieval in academic databases and general search engines [54]. At the same time, longer or more formulaic titles could introduce framing bias that narrows interpretation, and the tendency to generate longer, more structured titles raises concerns about readability while enhancing searchability [2,13,52]. This paradox between discoverability and accessibility highlights an important intersection between AI-assisted title generation, information retrieval, and search engine optimization (SEO) practices. Against this backdrop, researchers might consider using LLMs as first draft generators while applying their own judgment to refine titles for clarity, tone, and balance.

While ChatGPT occasionally simplified technical or compound terminology, these changes were not always neutral. For example, simplification could result in the omission of methodological or outcome details, thereby reducing informational completeness. This tension between clarity and precision illustrates the need for human oversight in evaluating whether AI-generated adjustments accurately reflect the study’s content. Future research should further explore how such simplifications affect reader understanding, perceived rigor, and information retrieval.

Limitations

This study has several limitations. First, it was conducted on a historical dataset of articles published in 2000, prior to the emergence of LLMs, to ensure that original titles were entirely human-written. While this provides a clean comparison, some editorial conventions may have evolved over the past two decades, potentially limiting generalizability to contemporary writing. Second, title conventions in our dataset may have been shaped by journal-specific editorial policies, such as restrictions on title length or discouragement of colons. At the time, the ICMJE Uniform Requirements recommended only that titles be ‘concise but informative’, without prescribing detailed structural rules [55,56]. The observed differences between ChatGPT- and human-written titles may therefore partly reflect contrasts between journal house styles and ChatGPT’s default titling patterns, rather than solely fundamental differences in emphasis. Third, we used only structured abstracts as input and operated ChatGPT in a zero-shot mode without prompt optimization or chain-of-thought reasoning. As such, our findings reflect the model's baseline performance rather than its full capabilities when used by experienced prompt engineers. Fourth, while we employed rigorous manual coding, certain content judgments, such as what constitutes an “outcome mention” or a “population descriptor”, may involve interpretive nuance. Fifth, we focused exclusively on high-impact general medical journals; patterns may differ in specialty fields, lower-impact venues, or non-English publications. Sixth, the diversity check performed on the randomly selected articles was informal and not based on a standardized taxonomy of study designs or subject areas. As such, although the sample covered a range of topics, it cannot be considered formally representative of the full spectrum of research published in general medical journals. Seventh, this study did not include reader ratings or perceptions. A follow-up study is planned in which researchers will compare the accuracy and appeal of human-written versus ChatGPT-generated titles based on the same abstracts. Finally, article selection and the representativeness check were performed by a single researcher (PS), which may introduce bias despite the use of randomization.

Conclusion

This study provides empirical evidence that ChatGPT-4.0, when prompted in a zero-shot setting using only structured abstracts, produces article titles that are systematically longer, more detailed, and more likely to include key elements such as study design, population descriptors, and outcome emphasis compared to human-written titles. These results suggest that LLMs can support scientific writing by enhancing structural completeness and transparency, although human input may still be necessary to refine tone, style, and rhetorical nuance. At the same time, longer and more formulaic titles may reduce stylistic conciseness, and simplification of terminology may omit technical specificity or introduce unnecessary detail, making titles less aligned with journal-specific conventions. These trade-offs reinforce the need for careful human oversight when AI is used in title construction.

As generative AI tools become increasingly integrated into academic workflows, further research is warranted to explore their impact on readers’ perceptions, editorial standards, and the evolving norms of scientific communication. Ethical considerations, including transparency in authorship, acknowledgment of AI assistance, and adherence to journal guidelines, will also need to be addressed as LLM-assisted writing becomes more widespread.

FUNDING & COMPETING INTERESTS

None.

ETHICAL APPROVAL

Since this study did not involve the collection of personal health-related data it did not require ethical review, according to current Swiss law.

DATA AVAILABILITY STATEMENT

The data associated with this article are available in Table 1.

AUTHOR CONTRIBUTIONS

Conceptualization: Paul Sebo; Project administration: Paul Sebo, Bing Nie, and Ting Wang; Formal analysis: Paul Sebo; Writing-original draft: Paul Sebo.

REFERENCES

1. Jacques TS, Sebire NJ. The Impact of Article Titles on Citation Hits: An Analysis of General and Specialist Medical Journals. JRSM Short Rep SAGE Publications; 2010 June 1;1(1):1–5. DOI: http://dx.doi.org/10.1258/shorts.2009.100020
.

2. Paiva CE, Lima JP da SN, Paiva BSR. Articles with short titles describing the results are cited more often. Clinics 2012 May 1;67(5):509–513. DOI: http://dx.doi.org/10.6061/clinics/2012(05)17
.

3. Subotic S, Mukherjee B. Short and amusing: The relationship between title characteristics, downloads, and citations in psychology articles. J Inf Sci SAGE Publications Ltd; 2014 Feb 1;40(1):115–124. DOI: http://dx.doi.org/10.1177/0165551513511393
.

4. Jamali HR, Nikzad M. Article title type and its relation with the number of downloads and citations. Scientometrics 2011 Aug 1;88(2):653–661. DOI: http://dx.doi.org/10.1007/s11192-011-0412-z
.

5. Nair LB, Gibbert M. What makes a ‘good’ title and (how) does it matter for citations? A review and general model of article title attributes in management science. Scientometrics 2016 June 1;107(3):1331–1359. DOI: http://dx.doi.org/10.1007/s11192-016-1937-y
.

6. Harmon JE, Gross AG. The Structure of Scientific Titles. J Tech Writ Commun SAGE Publications Inc; 2009 Oct 1;39(4):455–465. DOI: http://dx.doi.org/10.2190/TW.39.4.g
.

7. Guo F, Ma C, Shi Q, Zong Q. Succinct effect or informative effect: the relationship between title length and the number of citations. Scientometrics 2018 Sept 1;116(3):1531–1539. DOI: http://dx.doi.org/10.1007/s11192-018-2805-8
.

8. Braticevic MN, Babic I, Abramovic I, Jokic A, Horvat M. Title does matter: a cross-sectional study of 30 journals in the Medical Laboratory Technology category. Biochem Medica Croatian Society of Medical Biochemistry and Laboratory Medicine; 2020 Feb 15;30(1):0–0. DOI: http://dx.doi.org/10.11613/BM.2020.010708
.

9. Hartley J. To Attract or to Inform: What are Titles for? J Tech Writ Commun SAGE Publications Inc; 2005 Apr 1;35(2):203–213. DOI: http://dx.doi.org/10.2190/NV6E-FN3N-7NGN-TWQT
.

10. Yitzhaki M. Variation in informativity of titles of research papers in selected humanities journals: A comparative study. Scientometrics Akadémiai Kiadó, co-published with Springer Science+Business Media B.V., Formerly Kluwer Academic Publishers B.V.; 2006 July 26;38(2):219–229. DOI: http://dx.doi.org/10.1007/bf02457410
.

11. Medical Writing 4th Edition. Camb Univ Press Assess. Available from: https://www.cambridge.org/universitypress/subjects/medicine/medicine-general-interest/medical-writing-prescription-clarity-4th-edition [accessed June 7, 2025].

12. Haggan M. Research paper titles in literature, linguistics and science: dimensions of attraction. J Pragmat 2004 Feb 1;36(2):293–317. DOI: http://dx.doi.org/10.1016/S0378-2166(03)00090-0
.

13. Habibzadeh F, Yadollahie M. Are shorter article titles more attractive for citations? Cross-sectional study of 22 scientific journals. Croat Med J 2010 Apr;51(2):165–170. PMID: 20401960.

14. Salvagno M, Taccone FS, Gerli AG. Can artificial intelligence help for scientific writing? Crit Care 2023 Feb 25;27(1):75. DOI: http://dx.doi.org/10.1186/s13054-023-04380-2
.

15. Korinek A. Language Models and Cognitive Automation for Economic Research. National Bureau of Economic Research; 2023. DOI: http://dx.doi.org/10.3386/w30957
.

16. Stokel-Walker C, Van Noorden R. What ChatGPT and generative AI mean for science. Nature 2023 Feb;614(7947):214–216. PMID: 36747115.

17. Lund B, Ting W. Chatting about ChatGPT: How May AI and GPT Impact Academia and Libraries? Rochester, NY: Social Science Research Network; 2023. DOI: http://dx.doi.org/10.2139/ssrn.4333415
.

18. Gao CA, Howard FM, Markov NS, Dyer EC, Ramesh S, Luo Y, Pearson AT. Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. NPJ Digit Med 2023 Apr 26;6(1):75. PMID: 37100871.

19. Else H. Abstracts written by ChatGPT fool scientists. Nature 2023 Jan;613(7944):423. PMID: 36635510.

20. Biswas S. ChatGPT and the Future of Medical Writing. Radiology 2023 Apr;307(2):e223312. PMID: 36728748.

21. Sebo P. Can ChatGPT Recognize Its Own Writing in Scientific Abstracts? Cureus 2025 July;17(7):e88774. PMID: 40861680.

22. Hosseini M, Horbach SPJM. Fighting reviewer fatigue or amplifying bias? Considerations and recommendations for use of ChatGPT and other Large Language Models in scholarly peer review. Res Sq 2023 Feb 20;rs.3.rs-2587766. PMID: 36865238.

23. Robertson Z. GPT4 is Slightly Helpful for Peer-Review Assistance: A Pilot Study. arXiv; 2023. DOI: http://dx.doi.org/10.48550/arXiv.2307.05492
.

24. Stokel-Walker C, Van Noorden R. What ChatGPT and generative AI mean for science. Nature 2023 Feb;614(7947):214–216. PMID: 36747115.

25. Thorp HH. ChatGPT is fun, but not an author. Science 2023 Jan 27;379(6630):313. PMID: 36701446.

26. Sebo P. How accurate are the references generated by ChatGPT in internal medicine? Intern Emerg Med 2024 Jan;19(1):247–249. PMID: 37979017.

27. Chen Y, Eger S. Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-End. arXiv; 2023. DOI: http://dx.doi.org/10.48550/arXiv.2212.10522
.

28. Rehman T, Sanyal DK, Chattopadhyay S. Can pre-trained language models generate titles for research papers? arXiv; 2024. DOI: http://dx.doi.org/10.48550/arXiv.2409.14602
.

29. Krippendorff K. Content analysis: an introduction to its methodology. Fourth edition. Los Angeles London New Delhi Singapore Washington DC Melbourne: SAGE; 2019. ISBN:978-1-5063-9566-1.

30. Neuendorf KA. The content analysis guidebook. Second edition. Los Angeles London New Delhi Singapore Washington, DC Melbourne: SAGE; 2017. ISBN:978-1-4129-7947-4.

31. Schilhan L, Kaier C, Lackner K. Increasing visibility and discoverability of scholarly publications with academic search engine optimization. Insights 2021; Available from: http://unipub.uni-graz.at/obvugrfodok/5992511 [accessed Nov 11, 2025].

32. Pottier P, Lagisz M, Burke S, Drobniak SM, Downing PA, Macartney EL, Martinig AR, Mizuno A, Morrison K, Pollo P, Ricolfi L, Tam J, Williams C, Yang Y, Nakagawa S. Title, abstract and keywords: a practical guide to maximize the visibility and impact of academic papers. Proc R Soc B Biol Sci Royal Society; 2024 July 31;291(2027):20241222. DOI: http://dx.doi.org/10.1098/rspb.2024.1222
.

33. Busse C, August E. How to Write and Publish a Research Paper for a Peer-Reviewed Journal. J Cancer Educ Off J Am Assoc Cancer Educ 2021 Oct;36(5):909–913. PMID: 32356250.

34. Lang TA. Writing a better research article. J Public Health Emerg AME Publishing Company; 2017 Dec 18;1(12). DOI: http://dx.doi.org/10.21037/jphe.2017.11.06
.

35. Milojević S. The Length and Semantic Structure of Article Titles—Evolving Disciplinary Practices and Correlations with Impact. Front Res Metr Anal Frontiers; 2017 Apr 6;2. DOI: http://dx.doi.org/10.3389/frma.2017.00002
.

36. Ecarnot F, Seronde M-F, Chopard R, Schiele F, Meneveau N. Writing a scientific article: A step-by-step guide for beginners. Eur Geriatr Med 2015 Dec 1;6(6):573–579. DOI: http://dx.doi.org/10.1016/j.eurger.2015.08.005
.

37. Busse CE, Anderson EW, Endale T, Smith YR, Kaniecki M, Shannon C, August ET. Strengthening research capacity: a systematic review of manuscript writing and publishing interventions for researchers in low-income and middle-income countries. BMJ Glob Health BMJ Specialist Journals; 2022 Feb 1;7(2):e008059. PMID: 35165096.

38. Rust P, Frings J, Meister S, Fehring L. Evaluation of a large language model to simplify discharge summaries and provide cardiological lifestyle recommendations. Commun Med Nature Publishing Group; 2025 May 29;5(1):208. DOI: http://dx.doi.org/10.1038/s43856-025-00927-2
.

39. Roster K, Kann RB, Farabi B, Gronbeck C, Brownstone N, Lipner SR. Readability and Health Literacy Scores for ChatGPT-Generated Dermatology Public Education Materials: Cross-Sectional Analysis of Sunscreen and Melanoma Questions. JMIR Dermatol 2024 Mar 6;7(1):e50163. DOI: http://dx.doi.org/10.2196/50163
.

40. Patel EA, Fleischer L, Filip P, Eggerstedt M, Hutz M, Michaelides E, Batra PS, Tajudeen BA. The Use of Artificial Intelligence to Improve Readability of Otolaryngology Patient Education Materials. Otolaryngol–Head Neck Surg Off J Am Acad Otolaryngol-Head Neck Surg 2024 Aug;171(2):603–608. PMID: 38751109.

41. Luo X, Chen F, Zhu D, Wang L, Wang Z, Liu H, Lyu M, Wang Y, Wang Q, Chen Y. Potential Roles of Large Language Models in the Production of Systematic Reviews and Meta-Analyses. J Med Internet Res 2024 June 25;26(1):e56780. DOI: http://dx.doi.org/10.2196/56780
.

42. Wu S, Ma X, Luo D, Li L, Shi X, Chang X, Lin X, Luo R, Pei C, Du C, Zhao Z-J, Gong J. Automated literature research and review-generation method based on large language models. Natl Sci Rev 2025 June 1;12(6):nwaf169. DOI: http://dx.doi.org/10.1093/nsr/nwaf169
.

43. Robinson K, Bontekoe K, Muellenbach J. Integrating PICO principles into generative artificial intelligence prompt engineering to enhance information retrieval for medical librarians. J Med Libr Assoc JMLA 2025 Apr 18;113(2):184–188. PMID: 40342302.

44. Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. J Med Libr Assoc JMLA 2018 Oct;106(4):420–431. PMID: 30271283.

45. Manning CD, Raghavan P, Schütze H. Introduction to information retrieval. Cambridge: Cambridge university press; 2008. ISBN:978-0-521-86571-5.

46. Hosseini M, Rasmussen LM, Resnik DB. Using AI to write scholarly publications. Account Res 2024 Oct;31(7):715–723. PMID: 36697395.

47. Science journals set new authorship guidelines for AI-generated text. Natl Inst Environ Health Sci. Available from: https://factor.niehs.nih.gov/2023/3/feature/2-artificial-intelligence-ethics [accessed June 13, 2025].

48. Resnik DB, and Hosseini M. Disclosing artificial intelligence use in scientific research and publication: When should disclosure be mandatory, optional, or unnecessary? Account Res Taylor & Francis; 0(0):1–13. PMID: 40126451.

49. Lund BD, Naheem K t. Can ChatGPT be an author? A study of artificial intelligence authorship policies in top academic journals. Learn Publ 2024;37(1):13–21. DOI: http://dx.doi.org/10.1002/leap.1582
.

50. Lund B, Ting W, Mannuru NR, Nie B, Shimray S, Wang Z. ChatGPT and a New Academic Reality: Artificial Intelligence-Written Research Papers and the Ethics of the Large Language Models in Scholarly Publishing. Rochester, NY: Social Science Research Network; 2023. DOI: http://dx.doi.org/10.2139/ssrn.4389887
.

51. Letchford A, Moat HS, Preis T. The advantage of short paper titles. R Soc Open Sci Royal Society; 2015 Aug;2(8):150266. DOI: http://dx.doi.org/10.1098/rsos.150266
.

52. Rostami F, Mohammadpoorasl A, Hajizadeh M. The effect of characteristics of title on citation rates of articles. Scientometrics 2014 Mar 1;98(3):2007–2010. DOI: http://dx.doi.org/10.1007/s11192-013-1118-1
.

53. Del Giglio A, Pereira da Costa MU. The use of artificial intelligence to improve the scientific writing of non-native English speakers. Rev Assoc Med Bras (1992). 2023;69(9):e20230560. DOI: http://dx.doi.org/10.1590/1806-9282.20230560
.

54. Schilhan L, Kaier C, Lackner K. Increasing visibility and discoverability of scholarly publications with academic search engine optimization. Insights. 2021;34:6. DOI: http://dx.doi.org/10.1629/uksg.534
.

55. International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. N Engl J Med 1997 Jan 23;336(4):309–315. PMID: 8995096.

56. Uniform requirements for manuscripts submitted to biomedical journals. International Committee of Medical Journal Editors. BMJ 1991 Feb 9;302(6772):338–341. PMID: 2001512. https://doi.org/10.1177/096100062096773.

Paul Sebo, 1 paul.seboe@unige.ch, Researcher and Family Physician, University Institute for Primary Care (IuMFE), Faculty of Medicine, University of Geneva, Geneva, Switzerland

Bing Nie, 2 icynb@foxmail.com, Researcher, Zhejiang Tongji Vocational College of Science and Technology, Hangzhou, Zhejiang, China

Ting Wang, 3 twang2@g.emporia.edu, Researcher, School of Library and Information Management, Emporia State University, Emporia, KS, USA

© 2026 Paul Sebo, Bing Nie, Ting Wang

This work is licensed under a Creative Commons Attribution 4.0 International License.

Journal of the Medical Library Association, VOLUME 114, NUMBER 3, July 2026