INTRODUCTION
The tobacco industry has long been a dominant threat to global public health, with conventional cigarette smoking historically representing the leading preventable cause of disease and death worldwide1,2. In recent decades, this landscape has been disrupted by the emergence and rapid proliferation of electronic cigarettes (e-cigarettes)3. Initially developed and marketed as a harm-reduction tool for smokers seeking to quit combustible tobacco, the e-cigarettes have since evolved into a multifaceted phenomenon, attracting not only existing smokers but also a new generation of nicotine users who might never have smoked conventional cigarettes4,5. In fact, the e-cigarettes also have high nicotine content and are easily inhaled in aerosol form, which can lead to respiratory diseases, cardiovascular damage, and nicotine dependence, and affect the physical and mental health of adolescents1,4,6.
These days, the global e-cigarette market has expanded exponentially, driven by aggressive marketing strategies, continuous product innovation, and the normalization of vaping culture through digital platforms7-9. Within this global trend, China occupies a uniquely significant position. As the world’s largest producer and consumer of conventional tobacco products, China has a longstanding and complex relationship with the tobacco industry, housing over 300 million smokers and bearing a substantial burden of tobacco-related disease10. In recent years, this landscape has been further transformed by the rapid proliferation of e-cigarettes within the Chinese market11. Initially emerging as a niche product in major metropolitan areas, e-cigarettes have experienced explosive growth across the country, with numerous domestic brands capturing substantial market share through both online and offline retail channels11,12. Industry reports suggest that China may now constitute or is poised to become the world’s largest e-cigarette market13.
Despite a legal ban on the sale to youth under 18 years, there is a growing concern of illegal selling to adolescents and young adults in China14. This concern is particularly prominent for online sales, given the highly developed e-commerce ecosystem, coupled with the intensive online activity of its youth population15. These have created an environment where adolescents and young adults can readily access unregulated e-cigarette-related information and purchasing channels through social media platforms14,15. Consequently, e-cigarettes have found fertile ground among youth populations who may perceive them as modern, less harmful, and socially desirable alternatives to conventional cigarettes8,10. Crucially, this demographic also represents the most active user segment on social media platforms. Adolescents and young adults increasingly turn to digital spaces for health information, social connection, and entertainment, making them particularly susceptible to the influential content circulating on platforms such as Bilibili and TikTok16,17. These platforms host a vast and largely unregulated repository of e-cigarette-related content, ranging from product reviews and user testimonials to lifestyle portrayals that may glamorize vaping, all of which may shape perceptions and behaviors among impressionable viewers.
Previous research by Li et al.18 collected and analyzed posts on Weibo (China’s equivalent to Twitter) related to e-cigarettes. They identified that multi-stakeholder governance mechanisms involving health authorities, media institutions, and digital platforms were lacking18. Therefore, a systematic evaluation of e-cigarette-related videos on Chinese social media platforms is needed. By systematically analyzing the quality, reliability, and thematic content of videos on Bilibili and TikTok, this study aims to provide policymakers and healthcare providers with a comprehensive understanding of the current information landscape shaping youth perceptions of e-cigarettes.
METHOD
Study design and data collection
A cross-sectional study was conducted to identify and analyze short videos related to e-cigarettes on the two major Chinese social media platforms: Douyin (The Chinese version of TikTok) and Bilibili. The Chinese term for e-cigarettes was used as the sole search term on both websites. To mitigate the influence of personalized algorithms, all searches were performed using newly created accounts and the incognito mode of web browsers. On each platform, the default sorting algorithm (‘Comprehensive Ranking’ on TikTok and ‘Most Relevant’ on Bilibili) was used to simulate the experience of a typical user. All relevant videos from each platform were collected for further screening. Two reviewers independently screened all retrieved videos for eligibility and extracted data, both of whom are healthcare professionals with relevant working experience. Disagreements were resolved by discussion or by a third reviewer. Data collection was carried out on 11 January 2026, to maintain consistency and avoid dynamic fluctuations in platform algorithms.
Inclusion and exclusion criteria
Videos were included if their primary content was directly related to e-cigarettes. Exclusion criteria were: 1) duplicate videos; 2) videos identified as irrelevant or mainly promotional; and 3) videos lasting less than 10 seconds or published within one week prior to data collection.
Data extraction
For each video included, the following data were extracted into a standardized Microsoft Excel spreadsheet: platform (Bilibili/TikTok), unique video URL, date of upload, video duration (in seconds), and quantitative engagement metrics, including the number of likes, comments, shares and collections. All data were sourced from publicly available information.
Video source analysis
Uploaders were classified into three categories: ‘Specialist’ (healthcare providers were regarded as ‘Specialists’), ‘Institution’ (medical or educational institutions) and ‘Individual’ (general users). The classification process was also independently evaluated by two reviewers. Video source analysis compared uploader type distributions between platforms using the chi-squared test and assessed quality score differences across uploader groups with the Kruskal-Wallis test.
Video quality assessment
The quality of the included videos was assessed using four standard tools: the Global Quality Scale (GQS), the modified Decision-making Information Support Criteria for Evaluating the Reliability of Non-randomized Studies (mDISCERN), the Journal of the American Medical Association (JAMA) Benchmark Criteria, and the Video Information Quality Index (VIQI). Overall informational quality was evaluated using the GQS, which assigns a score from 1 (poor quality) to 5 (excellent quality) (Supplementary file Table 1)19. The reliability of the health information was measured with the mDISCERN, which evaluates five key items: 1) clarity of aims; 2) use of reliable sources; 3) balance of information; 4) additional information; and 5) area of uncertainty (Supplementary file Table 2). Each scored as 0 or 1, yielding a total possible score of 0 to 520,21. Transparency and accountability were assessed via the JAMA Benchmark Criteria, which cover four domains: authorship, attribution, disclosure and currency; each domain is scored 0 or 1, resulting in a total score ranging from 0 to 422. VIQI is a platform-adapted scoring system based on four key domains: information flow (VIQI 1), clarity (VIQI 2), video quality (VIQI 3) and consistency (VIQI 4). Each domain ranges from 0 to 5, yielding a total score between 0 and 20. A higher score indicates higher quality23. All assessments were conducted independently by two reviewers. Any disagreement in scores was resolved by thorough discussion or a third reviewer for the final decision. Inter-rater reliability for the quality scores was calculated using Cohen’s kappa coefficient, with κ>0.60 suggesting good agreement (Supplementary file Table 3)24,25.
Statistical analysis
Descriptive statistics were used to analyze video characteristics and quality metrics. Medians and interquartile ranges (IQR) were reported due to the non-parametric distribution of the data. The Mann-Whitney U test was used to compare non-normal distribution continuous variables among two groups. The Kruskal-Wallis test was employed to assess differences in GQS, mDISCERN, JAMA, and VIQI across different uploader groups. The relationship between non-normally distributed continuous variables was assessed using Spearman correlation analysis. A p<0.05 was considered statistically significant. All statistical computations were conducted using IBM SPSS 26.0 and the online statistical platforms26,27.
RESULTS
Video baseline characteristics
In all, 291 videos were obtained from the initial search. After applying exclusion criteria, 218 videos (102 on Bilibili and 116 on TikTok) were finally included for analysis (Figure 1). The baseline feature analysis showed that there were significant differences between the two platforms in terms of video duration, engagement metrics, and quality scores. The median video duration on Bilibili was significantly longer (261.00 seconds vs 77.50 seconds, Z= -8.98, p<0.001). Regarding social engagement metrics, the number of likes (median 495.00 vs 101.00, Z= -3.36) and shares (median 277.50 vs 20.00, Z= -4.61) were significantly higher on TikTok (both p<0.001). There was no significant difference in the collection volume between the two platforms (p=0.979) (Table 1).
Table 1
Baseline information of e-cigarettes-related videos from Bilibili and TikTok, 11 January 2026
| Variables | Bilibili (N=102) Median (IQR) | TikTok (N=116) Median (IQR) | Effect size Z* | p |
|---|---|---|---|---|
| Video length (seconds) | 261.00 (140.25–636.25) | 77.50 (37.00–119.00) | -8.98 | <0.001 |
| Comments | 14.00 (1.00–140.25) | 69.00 (8.00–307.00) | -2.71 | 0.007 |
| Shares | 277.50 (17.25–3950.75) | 20.00 (1.00–314.25) | -4.61 | <0.001 |
| Likes | 495.00 (55.00–5776.50) | 101.00 (10.75–1057.00) | -3.36 | <0.001 |
| Collections | 82.50 (7.25–759.00) | 80.50 (4.75–720.75) | -0.03 | 0.979 |
| GQS | 4.00 (3.00–4.00) | 1.00 (1.00–3.00) | -9.50 | <0.001 |
| mDISCERN | 2.00 (2.00–2.00) | 1.00 (1.00–2.00) | -10.19 | <0.001 |
| JAMA | 2.00 (2.00–2.00) | 1.00 (1.00–1.00) | -8.80 | < 0.001 |
| VIQI | 16.00 (15.00–18.00) | 13.00 (12.00–13.25) | -7.81 | <0.001 |
| Uploader, n (%) | χ2=129.47 | <0.001 | ||
| Individual | 102 (87.93) | 11 (10.78) | ||
| Institution | 13 (11.21) | 81 (79.41) | ||
| Professional | 1 (0.86) | 10 (9.80) |
Video quality metrics between the two platforms
Regarding content quality, Bilibili videos demonstrated significantly higher scores across all assessments. The Global Quality Scale (GQS) scores were significantly higher on Bilibili (median=4.00) than TikTok (median=1.00; Z= -9.50, p<0.001) (Table 1, Figure 2). In addition, the score distribution was also contrastingly different between the two platforms. Similarly, the Video Information Quality Index (VIQI) was significantly higher on Bilibili (median=16.00) than on TikTok (median=13.00; Z= -7.81, p<0.001). The mDISCERN also showed higher median scores of 2.00 for Bilibili versus 1.00 for TikTok (Z= -10.19, p<0.001), and both platforms were dominated by low-to-middle scores. JAMA Benchmark Criteria were higher on Bilibili (median=2.00) compared to TikTok (median=1.00; Z= -8.80, p<0.001).
Video source analysis
A significant variation in video source between platforms was identified (Supplementary file Figure 1). On Bilibili, the majority of uploaders were individual users (87.93%), whereas on TikTok, institutional uploaders predominated (79.41%). Healthcare providers represented only a small proportion on both platforms (overall 5.05%). The distribution characteristics and differences across three different uploader groups are presented in Figure 3. GQS score showed that professionals uploaded videos with the highest median score (4.00), which is significantly higher than that of institutional (3.00) and individual (1.00) uploaders. The score of institutional uploaders was significantly higher than that of individual uploaders. The median score for videos uploaded by professionals and institutions was 2.00, significantly higher than that of individual uploaders (1.00, both p<0.001). There was no significant difference between professionals and institutional uploaders. In terms of JAMA Benchmark Criteria, professionals uploaded videos with the highest median score (2.00), significantly higher than institutional (1.50) and individual (1.00) uploaders (both p<0.001).
Correlation analysis
The Spearman correlation heatmap showed video social engagement metrics (likes, favorites, comments, shares, video duration) and video quality scores (GQS, mDISCERN, JAMA, VIQI) of the two platforms (Figure 4). There is a significant positive correlation within the interaction social engagement metrics on both platforms. In contrast, the association between the social interaction indicators and video quality metrics was weak on both social media platforms. On Bilibili, in particular, the favorites and mDISCERN score is negatively associated (r= -0.05), whereas comments and JAMA Benchmark Criteria were -0.01 on TikTok.
DISCUSSION
This cross-sectional analysis of e-cigarette-related videos on Bilibili and TikTok revealed significant disparities between the two platforms in terms of video characteristics, audience engagement, and content quality. Bilibili videos demonstrated substantially higher quality scores across all assessment instruments, despite having fewer engagement metrics such as likes and comments. Moreover, the distribution of uploader types differed markedly between platforms, with individual users predominating on Bilibili and institutional uploaders on TikTok. Furthermore, videos uploaded by professionals and institutions demonstrated significantly higher quality scores compared to those uploaded by individual users, suggesting that uploader expertise and organizational backing are critical determinants of content reliability. Importantly, Spearman’s correlation analysis revealed no significant positive association between social engagement metrics and content quality on either platform, indicating that popular videos are not necessarily reliable sources of health information.
Our findings align with some existing research examining e-cigarette-related content on social media platforms. During screening, we identified that a range of advertisements and promotional content was created on both platforms. This was consistent with the Dashtian et al.7 findings. However, the predominance of institutional uploaders on TikTok contrasts with findings from Dashtian et al.7, as they observed that individual users generated the majority of e-cigarette-related content on YouTube. This discrepancy may reflect platform-specific user demographics and content creation cultures, as well as the evolving landscape of social media marketing. Furthermore, we also found that professional uploaders produced the highest quality videos, which corroborated research by Ye et al.28 who demonstrated that healthcare professionals and media organizations generated more credible smoking cessation content on Bilibili compared to individual users. The weak association between engagement metrics and quality scores in our study echoes concerns raised by Smith et al.29 regarding the absence of age and health warnings in popular vaping content, highlighting the public health risks associated with algorithm-driven content exposure among youth.
This cross-sectional study identifies several areas for future investigation. First, the observation that adolescents and young adults, who are also active social media users, are exposed to e-cigarette content of highly variable quality. This calls for research into how different health communication formats might engage this specific population to mitigate the consumption of e-cigarettes among them. Second, the marked heterogeneity in content quality and uploader characteristics between Bilibili and TikTok suggests that platform-specific regulatory scenarios warrant exploration through policy analysis and longitudinal research, especially given the digital enforcement gap identified despite existing age-based sale prohibitions. Third, the lack of association between engagement metrics and content reliability implies that current recommendation algorithms may amplify low-quality information; experimental or quasi-experimental studies that manipulate algorithmic signals to prioritize quality over popularity could clarify whether such adjustments alter user exposure to unreliable content.
Strengths and limitations
This study has several strengths. First, this is a comprehensive evaluation of e-cigarette-related video content on China’s two leading social media platforms, Bilibili and TikTok. This provides novel insights into the information landscape shaping youth perceptions in the world’s largest tobacco market. Second, we employed multiple validated quality assessment instruments, GQS, mDISCERN, JAMA Benchmark Criteria, and VIQI, to allow for multidimensional evaluation of content quality, reliability, transparency and information presentation. This multi-instrument approach enhances the robustness and comparability of our findings. Third, our methodological rigor included independent assessment by reviewers with good inter-rater reliability, systematic searches using newly created accounts to minimize algorithmic bias, and appropriate statistical analyses that accounted for non-parametric data distributions. These methodological choices strengthen the validity and reproducibility of our results.
However, several limitations should be acknowledged when interpreting our findings. Firstly, this is a crosssectional study, and our findings only represent a snapshot rather than a longitudinal trend, given the dynamic nature of social media platforms and the rapid evolution of e-cigarette products and marketing strategies. Future research should examine how e-cigarette-related content evolves over time and in response to regulatory changes. Moreover, the nature of cross-sectional studies precludes causal inferences regarding the relationship between platform characteristics, uploader type, and content quality. Secondly, our analysis was limited to two platforms, Bilibili and TikTok and may not be generalizable to other social media platforms popular among Chinese youth, such as WeChat and Weibo. Each platform may possess unique user demographics, content formats, and algorithmic characteristics that may influence the nature and quality of e-cigarette-related content. Finally, we employed several standardized quality assessment instruments, but there will still be inherent subjectivity.
CONCLUSIONS
This study reveals significant heterogeneity in the quality and reliability of e-cigarette-related videos on Chinese social media platforms. Bilibili hosted substantially higher-quality content than TikTok. Uploader expertise and institutional affiliation emerged as key determinants of content quality, while social engagement metrics failed to associate with reliability. These findings highlight substantial heterogeneity in e-cigarette video quality across Chinese social media platforms and uploader types. Further longitudinal studies are needed to explore the impact of platform-specific features and to evaluate the effectiveness of potential content oversight measures.
