Non-Native and Mobile Readers of a Teaching Article: Evidence on English-Medium Instruction, Subtitles, Bilingual Subtitles, Machine Translation, and Screen Reading

🌏 中文版

This is Part 24 of the "Writing Articles That Teach" series, and the last part of section five, "Who you learn with, and who can't read it." The earlier parts covered learning together versus alone, comment sections and communities, writing for others, and accessibility. This part handles three more reader-side conditions an author cannot see: the reader is not reading in their first language, is reading a translation, or is reading on a phone.

This site runs into all three. Its course guides take English-language university lectures and write them up in Chinese, one lecture at a time, with the original recording attached; readers read Chinese and watch English video. Most course guides do not turn on subtitles for the reader, but the guides to Stanford CS229 and CS329A turn on English captions by default. The vast majority of Chinese posts also have an English version, and the vast majority of those English versions were translated by AI following this site's translation spec; the spec defaults to rewriting rather than sentence-by-sentence translation. The production site loads at least Cloudflare Web Analytics, which Cloudflare injects at the edge rather than through the site's own code. According to the dimensions listed in Cloudflare's documentation, it records aggregate device type, so this site does have aggregate data on the share of phone readers; I did not look it up for this post. Writing for these readers raises three questions. What do readers lose when the language is not their first? How much evidence is there for translations, bilingual text, and subtitles? How does reading a long article on a phone differ from paper or a computer?

What earlier parts already covered is only touched on here. Whether to add the English original to technical terms, what non-native English speakers themselves say gets in the way of English materials, and glosses in foreign-language reading are in Part 4. Where to put the video in an article and how to lay it beside the text are in Part 5 and Part 6. The first meta-analysis of screens versus paper is in Part 9. The completion gap for learners from less developed countries in MOOCs, and what happened when "value relevance" writing was scaled to 247 courses, are in Part 19.

First, how these studies were chosen and how they lean. The scope is three things: the reader's language, translation and subtitles, and the reading device. Sources were found by scanning the literature first and then picking studies to read closely, with at least one study per topic that found the opposite result or no detected effect. Every source in the references was read in full in the version I could obtain, with an extraction record. The lean is obvious. Nearly all of the language half is English as a foreign language: the MOOCs are MITx science and engineering courses, the subtitle experiments used French speakers and Chinese students studying in the UK and Australia, and the meta-analyses mostly involve university students. The device half is mostly university students reading expository text in English, with very few direct comparisons involving phones. Studies from Taiwan are not covered here. No study measured non-native readers reading a teaching article on the web, and no study measured how much readers learn from a long web article read on a phone.

The short version

"No difference detected" means not statistically significant. It does not mean the two were shown to be the same.

What non-native readers lose

Online courses: getting to the finish

Sa'ar Karp Gershon, José Ruipérez-Valiente, and Giora Alexandron, in a 2021 study, analyzed 158 runs of MITx courses on edX from 2013 to 2018, covering about 1.59 million learners who viewed course material. Both language and country development were inferred from the country of the IP address each learner used most, and development was split into high and low by the Human Development Index (HDI).

The authors state a design limitation in their methods: every English-native country is a high-HDI country. So "English-native versus not" can only be compared among high-HDI countries, and countries such as India and Nigeria, where English is widely used but HDI is low, are not in that comparison.

• Within high-HDI countries, no difference was detected in the odds of completion between learners from English-native and non-English-native countries.
• Within non-English-native countries, learners from low-HDI countries had about 55% of the completion odds of learners from high-HDI countries.
• Looking only at certificate earners, five of six courses showed no detected difference in estimated ability by country development; in four of them, completers from developing countries actually had higher averages (none significant).

Put together: in these science and engineering courses, learners from developing countries struggled to reach the end, and those who reached it did no worse than others; whether learners were English-native showed no detected difference, within the range that could be compared. These are observational data measuring completion and assignment performance, not comprehension. The authors also note that "non-native" was inferred from country, so non-native speakers living in English-speaking countries and VPN users are misclassified.

English-medium university teaching: subject results

English Medium Instruction (EMI) means teaching subject content in English where English is not most people's first language. Hansol Lee, Heath Rose, Ernesto Macaro, and Jang Ho Lee, in a 2025 meta-analysis, included only higher education; I read the author accepted manuscript held in Oxford's repository.

Comparing groups taught in English with groups taught in their first language, subject-content results came to d = 0.13, with no difference detected. Counting only the 12 samples that confirmed the two groups started out comparable, d = 0.04; the 4 samples that did not confirm this came to 0.41 and pulled the overall figure up. The authors' conclusion is worded as: English-medium teaching does not, on the whole, harm subject learning, and lets students perform comparably to those learning in their first language.

Two things in this analysis belong together. First, in the same analysis, the pre-post gain of the EMI students alone was d = 1.57; the authors say explicitly that within-group comparisons like this cannot be attributed to EMI itself. Second, the authors list the non-comparability of the two groups' characteristics as a major threat to the between-group studies. These are whole programs or whole courses, with students studying for a semester or more; "reading one teaching article in a foreign language" is something else, and carrying the result over is my analogy.

What readers say gets in the way

Part 4 covered Philip Guo's 2018 survey of 656 non-native English speakers on a programming-education website. Among those who answered the question about how their English level made learning programming harder, the most commonly mentioned difficulty was reading instructional materials (35%). Among those who answered the question about how materials could be improved, a small group (7%) preferred reading the English original over a translated version, complaining that terms get translated away and translations go out of date. Of the non-native respondents, 83% rated their English reading as proficient or fully fluent; all of this is self-report, not a measured effect. Putting it next to the two studies above is my reading: Karp Gershon (among high-HDI countries, with first language inferred from country) and Lee (English-medium university teaching) both found no effect of language itself, and neither selected people by English level; Guo's respondents mostly had decent English, and the difficulties they described were reading slowly and getting stuck on terms. People with weak English are invisible in all three datasets.

Can culture and language be separated?

Language is often tied to other things about a country. René Kizilcec and Geoffrey Cohen, in two 2017 experiments, put a few minutes of self-regulation writing into two MOOCs: learners wrote down the benefits of and obstacles to finishing the course, then wrote "if…then…" plans (median time spent 6.6 to 8 minutes). Results were analyzed by how individualist each country is. Neither experiment detected an overall effect; in individualist countries completion rose by a relative 32% and 15%, which works out to absolute differences of about 1.8 and 3.8 percentage points (my calculation from the original tables); no effect was detected in collectivist or culturally balanced countries.

What matters for this part is a further check the authors ran: a country-level English proficiency index was strongly correlated with individualism, but it was not a consistent moderator. In the authors' words, "Moreover, our pattern of results does not appear to be related to English proficiency per se." Among the obstacles learners wrote down, one category the authors call "practical barriers" has representative words including Internet and computer, and also language, English, and understand; learners in collectivist countries wrote this kind of obstacle more often, and the writing exercise had no detected effect for it. This category mixes in Internet and computer problems, so it cannot be read as language barriers alone.

The same line of work was later scaled to 247 courses; Part 19 has the full account. One detail to add here: for the study-plan prompt, the original effect appeared in the subgroup of learners who intended to complete and were fluent in English, and after scaling up no difference was detected in the corresponding subgroup (the subgroup definition was adjusted across waves: first intending to complete and fluent in English, then English fluency only, and the last wave dropped the English condition); about half of the focal sample reported being fluent in English.

What this section can say: in MOOCs, there is no consistent evidence of an effect of language itself, while country development and culture do show up. What non-native readers of a web article lose has not been measured directly by any study so far.

Subtitles, bilingual subtitles, and translation

This site's guides attach the original English lectures. Whether readers turn on subtitles while watching, and which kind, is exactly what the following experiments manipulated.

Subtitles on foreign-language video, or none

Maria Pannatier and Mireille Bétrancourt, in a 2024 experiment, recruited 131 French-speaking students to watch a 5-minute English academic talk (an excerpt of a public lecture Richard Mayer gave at Harvard). A placement test first sorted them into elementary, intermediate, and advanced English, and within each level they were randomly assigned to English subtitles, French subtitles, or no subtitles. The video could be watched only once, with no pausing or rewinding, and participants answered in French right afterward.

None of the three subtitle formats showed a detected difference on any of the three tests, and there was no interaction between subtitle format and English level. The authors had predicted that elementary learners would do best with first-language subtitles, intermediate learners with English subtitles, and advanced learners with none; none of the three predictions held. English level itself affected the questionnaire and transfer items, and self-rated prior knowledge and habitual subtitle use each had some effect too. The authors concluded:

the subtitles cannot be considered harmful and, if designers have the option of enabling subtitles in whatever language, they should probably do it – even though the research provides no definitive answer regarding the choice of language (native or same language as narration).

The groups in this experiment were small, about 7 people per elementary cell, without enough power to detect small effects. Other limitations the authors list: there were no eye-tracking data, so it is unknown whether participants read the subtitles; and English and French are close languages, while earlier studies that found English subtitles helped had Chinese- and Korean-speaking participants.

Bilingual subtitles

Bilingual subtitles show two lines at once, one in the video's language and one in the viewer's first language. In both experiments below, participants were Chinese students learning English as a foreign language, and the bilingual subtitles put Chinese on top and English below.

Andi Wang and Ana Pellicer-Sánchez, 2023 experiment: I read an earlier version of the manuscript in University College London's repository; its first page says the published version is authoritative, so the numbers may differ from the published article. 112 Chinese students studying in the UK, at English level B2 to C1, watched a 23-minute BBC documentary in one of four groups (bilingual, English captions, Chinese subtitles, no subtitles) and then answered 34 questions in Chinese.

• The bilingual and Chinese-subtitle groups both scored around 80%, with no difference detected between them; both beat the English-caption group (about 69%, my conversion from the mean scores). The difference between English captions and no subtitles was not significant (p = .06).
• Eye tracking: while subtitles were on screen, the bilingual group spent about 42% of the time on the Chinese line and about 20% on the English line; the English line was more often skipped entirely.
• The authors say bilingual subtitles did not cause cognitive overload, but the study did not measure cognitive load; that statement is inferred from eye tracking and scores. The authors also say relying on the Chinese line may not help with learning English itself.

These participants already used bilingual subtitles more than any other kind, and the authors list this as their first limitation.

Sixin Liao, Jan-Louis Kruger, and Stephen Doherty, 2020 experiment, was smaller still: 20 Chinese postgraduate students in Australia were recruited, each watched four 5-minute BBC documentary clips, each clip with a different subtitle condition, and wrote down in English what they remembered. The individual analyses actually used 15 to 18 people.

• Free-recall scores showed no detected difference across the four conditions, including no subtitles.
• Self-reported cognitive load: no difference detected between bilingual and single-language subtitles; bilingual subtitles were lower than no subtitles.
• Individually, 9 of 16 looked more at the Chinese line and 7 more at the English line; the authors read this as each person choosing one language as the main source.

The abstract of this study says bilingual subtitles were more beneficial for comprehension than no subtitles. The recall scores themselves do not support that; what supports it is participants reporting that it felt easier. The authors themselves caution that writing recall in a second language may have suppressed differences.

Do subtitles help with language or with content?

The three experiments above all measured understanding of the content. Another line of research measures language learning. Satsuki Kurokawa, Aung Myo Hein, and Takumi Uchihara, in a 2024 meta-analysis, pooled 49 experiments in which foreign-language learners watched foreign-language video, one group with same-language captions and one without, without knowing they would be tested on vocabulary. The caption groups remembered more target words, g = 0.56.

In terms that are easier to picture: in the 18 experiments where absolute figures could be computed, caption groups learned 13.75% of target words and no-caption groups 10.64%, a gap of about 3 percentage points. The video type closest to this site's was the weakest: lectures and talks (half of them TED talks) came to g = 0.14, with no effect detected; video type was not a significant moderator overall, though, and this category had only 6 experiments. Studies with a pretest showed clearly smaller effects, and the authors say they cannot tell which is closer to the true value.

"Captions" in this analysis means foreign-language video with captions in the same foreign language (English in over 80% of the experiments), not subtitles translated into the viewer's first language, so it cannot be used to support providing translations. Taken together with the three experiments above: same-language captions help with remembering foreign words, and showed no detected help with understanding content.

Machine-translated course material

Do readers learn as well from translated teaching content? I did not find an experiment on this. The closest is a 2025 case study by Yerin Kwak, Nora Dunder, Olga Viberg, and Zachary Pardos: for a calculus course in the university preparatory year at KTH Royal Institute of Technology, the English open-source adaptive tutoring problem bank OATutor was translated into Swedish and aligned to the Swedish curriculum. The authors say in their limitations that effects on student learning are future work; the paper has no student-side data at all.

What it measured is the translation itself:

• On problem sentences, every model's BLEU (a metric of word overlap with a reference translation) was between 84 and 94; on terminology-dense sentences such as section titles, BLEU was only 18 to 61. Another, more meaning-oriented metric, xCOMET, was actually slightly higher on titles; the authors explain that it is more forgiving of small terminology errors. In the authors' words, "The models tested generally performed well in translating mathematical problems but struggled with terminology dense sentences like section titles". The fix was to use Swedish Wikipedia math articles as terminology data the model could consult while translating; in one example GPT-4o rendered scientific notation with a literal phrase rarely used in Swedish mathematics, and after consulting the terminology data switched to the standard term. The reference translations were made by one of the authors, a native Swedish speaker.
• After translation, the course instructor edited the output, changing 30 of 2,544 words (1.18%).
• Localization is more than translation. Of 16 skill topics recommended by AI, the instructor judged 6 irrelevant to the course; one topic's problems had to be rewritten to fit the Swedish curriculum's framing; and Sweden uses a comma as the decimal mark, so short-answer items would originally have marked "1,2" wrong.

This site's English versions go the other way (Chinese to English), and the content is not math problems, but the finding that terminology-dense sentences are the easiest to mistranslate points to the same place as the complaint from Guo's respondents in Part 4 that terms get translated away.

For an entire interface translated into the first language, Part 4 covered the 2017 Scratch data from Sayamindu Dasgupta and Benjamin Mako Hill: people using a first-language interface started lower and then grew slightly faster, the gap was small, and it is correlation, not causation.

How reading long text on a phone differs

Paper and screens

Part 9 covered the 2018 meta-analysis by Pablo Delgado and colleagues: reading the same text, comprehension on paper was slightly better than on screen, g = 0.21, and the authors consider the size educationally meaningful. Two moderator results relevant here were not spelled out then:

• In comparisons using computers, the paper advantage was significant; in comparisons using handheld devices (tablets, e-readers, and phones together) it was 0.12, with no difference detected. The difference between the two was itself not significant.
• In comparisons that needed scrolling, the paper advantage was 0.25; without scrolling it was 0.13 (p = .06), not significant. The difference between those two was also not significant, and the authors list it for future research.

Separating the devices

Virginia Clinton-Lisell and Christine Litzinger, in a 2026 meta-analysis using network meta-analysis, put paper, computers, tablets, e-readers, and phones into one model, drawing on 56 studies from 2010 onward in which participants read expository text in their first language.

• Overall, paper beat computers (g = 0.17) and phones (0.26); no difference was detected for tablets or e-readers. Paper ranked first and phones last.
• The phone result is the least stable. There are only 3 direct effect sizes comparing phones with paper; the direct estimate is 0.12, and half of the model estimate of 0.26 comes from indirect inference. The authors say it is hard to be confident in the results for phones and e-readers.
• The authors split the data into two groups and ran them separately. Among the 25 studies that needed scrolling, paper beat computers (0.35) and tablets (0.38); the phone point estimate was 0.48, but its confidence interval ran from −0.14 to 1.10, with no difference detected. Among the 31 studies that did not need scrolling, no difference from paper was detected for any device.
• Because the two groups were run separately, the paper does not test whether the difference between the scrolling and non-scrolling groups is itself significant. The abstract includes the phone's 0.48 in the range where paper was "notably better," which the tables in the body do not support.

On this basis the authors recommend:

For college students reading expository texts in English, developing e-texts that may be read through pagination rather than scrolling may be useful.

The authors also caution that scrolling was forced into a yes-or-no variable, without accounting for how much or how fast readers scroll; about half the studies did not report the reliability of their tests.

Ladislao Salmerón and colleagues' 2024 meta-analysis is often cited as "reading on a phone is worse," but its "handheld devices" include only tablets and e-readers, and it explicitly excludes studies that used only phones. I read the author manuscript held in the University of Valencia's repository. The between-participants analysis came to g = 0.11 and the within-participants analysis to 0.10, both slightly favoring paper. The 21 within-participants effect sizes pointed the same way, but not a single original study was significant on its own; the between-participants studies varied widely, and with two outliers put back in, no difference was detected. The gap was larger for university students (0.23) and not detected for school students.

Reading fiction on a phone

Experiments that actually compare phones with paper on a full sample are rare. Motoyasu Honma and eight co-authors, in a 2022 experiment, had 34 Japanese university students each read two excerpts of Haruki Murakami novels (about three thousand characters each), one on a 5-inch phone and one on paper, with no time limit, and answer 10 questions afterward. The paper was deliberately made the same size and weight as the phone, with the same type size.

Reading test scores were higher on paper than on the phone, the main effect of medium was significant, and paper scored higher for each novel separately; no difference was detected in reading time. The paper reports only the statistical tests; the body gives no figures for each group's mean or the size of the gap, so I cannot say how large the difference was. The authors also measured breathing and prefrontal activity and say the phone reduced sighing and caused prefrontal overactivation; the main effect of medium on prefrontal activity was in fact not significant, and the authors frame the whole study as exploratory.

This study disagrees with the meta-analyses above on two points, set out under "Where the sources disagree" below.

Applying this to long web articles

The screen conditions in these studies mostly stripped out digital-only features (links, interactivity), and most measured multiple-choice answers under time limits. A teaching article on the web always scrolls, readers set their own pace, and they often read it over several sittings. So what carries over is only the direction: when reading expository text that requires scrolling, comprehension on screen is slightly worse than on paper; the evidence is not enough to say whether phones add a further gap. I found no study that directly compares reading the same long web article on a phone and on a computer screen.

Readers' devices, networks, and this site's pages

The 75th-percentile reader

The device and network a reader uses determine how long a page takes to appear and how much data it costs. Every year, Alex Russell estimates the 75th-percentile user on a personal blog: a quarter of devices and networks are worse than this baseline. This is the author's estimate and commentary built from third-party data, not peer-reviewed research.

The 2026 edition: 9 Mbps down, 3 Mbps up, 100 ms round-trip latency, with a low-to-mid-range Android phone such as the Samsung Galaxy A24 4G as the test device. According to the HTTP Archive data the author links to, the median mobile page weighs 2.6 MiB; low-end Android single-core performance is 9 times slower than a contemporary iPhone. The author says these conditions may be optimistic, and that sites serving readers worldwide should target the 90th to 95th percentile.

Nothing in this source is about learning. I found no study on whether slow pages or expensive data make readers learn less.

One of this site's posts on a phone

Loading the Chinese version of Part 1 (in Chinese) on the production site at a Pixel 5 screen size (393 × 727 CSS pixels), unthrottled and with no cache, on 2026-10-14 at 19:22 Taiwan time:

• About 1.88 MB transferred, of which about 1.40 MB is a single Chinese font file (a Noto Sans TC OTF file, preloaded on every post). The font is set to display text in a system font first and swap when the download finishes, so it does not block the text from appearing, but the data is still spent. JavaScript is about 289 KB, mostly React components such as the on-page chat assistant, and it also includes Cloudflare's analytics script (about 10 KB).
• The uncompressed HTML is about 458 KB, of which about 270 KB is the site-wide glossary data; every post carries the same copy, used to mark terms in the text and show definitions on tap.
• The page is about 26,000 pixels tall, about 36 screens. Body text is 16 px, paragraphs are 325 px wide, and a line holds about 20 Chinese characters (my conversion).
• Tables do not wrap. Table width is set to fit the longest line of content, so the summary table at the top is 1,682 px wide inside a 367 px frame, and readers have to swipe sideways to see all of it.

Against Russell's budgets (my calculation): the total of about 1.79 MiB is within Russell's 2.0 MiB budget for a page light on JavaScript to be usable within 3 seconds, but 1.34 MiB of it is a single font file. This measurement was not throttled, so these numbers say how much has to be downloaded, not how long it takes.

Where the sources disagree

The following results point in different directions, or look contradictory but actually measure different things. I set them side by side without picking a winner.

Is language related to MOOC completion? Within high-HDI countries, Karp Gershon and colleagues detected no relationship between being English-native and completion. They cite, secondhand, a 2017 result by Türkay and colleagues using HarvardX data that points the other way: English learners had about 60% of native speakers' odds of earning a certificate. The authors list five possible explanations, including different datasets, completion and certification not being the same thing, per-course versus pooled calculation, and Türkay asking about language in a survey whereas they inferred it from IP. I did not read Türkay's original paper.

Do first-language subtitles help understanding? In Wang and Pellicer-Sánchez's experiment, Chinese and bilingual subtitles beat English captions and no subtitles. In Pannatier and Bétrancourt's experiment, French subtitles showed no detected difference from English subtitles or none; in Liao and colleagues' experiment, none of the four conditions differed detectably. The differences among the three: 23-minute versus 5-minute video; how far the participants' first language is from English (Chinese versus French); how they were tested (first-language multiple choice, first-language questionnaire and open items, free recall written in English); and Pannatier's video could not be paused. No study has directly compared which of these is the main cause.

Which line of bilingual subtitles do eyes land on? In Wang and Pellicer-Sánchez's bilingual group, time on the Chinese line was about twice that on the English line. In Liao and colleagues' bilingual condition, no difference in dwell time between the two lines was detected, and individually about half the people favored Chinese and half English. Liao's design had each person watch all four conditions, and in three of the four orders bilingual subtitles came right after English captions; this is exactly the design Wang and Pellicer-Sánchez criticize.

Is there a paper advantage for narrative text? In Delgado and colleagues' meta-analysis, the 7 comparisons using only narrative text showed no paper advantage. In Salmerón and colleagues' meta-analysis, the point estimate for narrative text was actually larger than for expository text, but neither showed a detected difference. Honma and colleagues used fiction, and paper beat the phone. Honma's paper was made phone-sized, so the comparison is a screen versus paper of the same size, not a typical phone versus a book.

Does the gap shrink when readers control their own time? In Delgado and colleagues' meta-analysis, the gap was small when readers set their own pace and clear only under time limits. In Salmerón and colleagues' meta-analysis, time limits made no difference, which the authors themselves call a conflicting finding. Honma and colleagues had no time limit and still detected higher scores on paper.

Is scrolling the key? Clinton-Lisell and Litzinger split the data by whether scrolling was needed; only the scrolling group showed a paper advantage, and on that basis the authors recommend pagination. In Delgado and colleagues' meta-analysis, the difference between scrolling and non-scrolling comparisons was not significant. In Salmerón and colleagues' data there were only 5 scrolling comparisons, too few to judge.

This site's practice, checked against the evidence

What you can do

All of the following are my judgments, each with its basis.

1. When writing for readers whose English is adequate but not their first language, deal with terms and reading speed first. Basis: Karp Gershon (among high-HDI countries, with first language inferred from country) and Lee (English-medium university teaching) both found no effect of language itself; Guo's respondents were mostly proficient in English and said their difficulties were reading slowly and a single term blurring a whole sentence. How: give a one-sentence plain definition when a term first appears; put only one new term in a sentence.
2. When embedding foreign-language video, don't decide subtitles for the reader, but state the video's language and subtitle situation. Basis: Pannatier's authors recommend turning subtitles on where possible and leaving the language to the user; the three experiments disagree on first-language subtitles. How: next to the player, write one line such as "Taught in English; YouTube has English captions" or "No captions."
3. Turn English captions on by default only when the reason is helping readers learn English, not helping them follow the content. Basis: in Kurokawa's meta-analysis, same-language captions help with remembering words, with no effect detected for lecture videos; in the three comprehension experiments, same-language subtitles showed no detected help. Of the 31 posts on this site that turn English captions on by default, 28 are full lecture recordings in the Stanford CS229 and CS329A course guides, 1 is an English shadowing exercise, and 2 are general posts; the shadowing exercise fits this direction, and the 28 course guides need a different justification.
4. Have someone who knows the field check the terms in a translated version. Basis: in Kwak's case study, the terminology-dense title sentences were translated worst, and the instructor's small amount of editing concentrated there. How: after translating, list every term in the text with its translation and confirm each is the standard usage in the target-language community, not a literal rendering.
5. Avoid tables that need sideways swiping on a phone. Basis: no direct study; both scrolling meta-analyses deal with vertical scrolling. This one is my inference. How: write each table cell as a short phrase and move longer explanations into a paragraph below the table; when a table has more than three columns, consider a list instead.
6. Don't write "reading on a phone is worse" into an article or a design rationale as settled fact. Basis: there are only 3 direct effect sizes comparing phones with paper, and the frequently cited handheld-device meta-analysis does not include phones.

Three things to do tonight

1. Open your longest post on a phone, count how many screens you have to scroll, find the tables that need sideways swiping, and shorten the widest one or turn it into a list.
2. Next to every embedded video, add one line on the video's language and subtitle situation.
3. Open the Network tab in your browser's developer tools, clear the cache, reload one of your posts, and look at the three largest files transferred: does the reader actually need them to read this post?

Limits of this evidence

• No study has measured how much non-native readers learn from web teaching articles. The closest are MOOC completion records and whole-course university grades; applying them to a single article is my analogy.
• The language evidence is almost all English as a foreign language. Karp Gershon's "non-native" is inferred from country; subtitle experiment participants were only French speakers and Chinese students; most samples in the EMI meta-analysis are not in Chinese-language settings. Studies of Chinese as a foreign language, or of Chinese readers reading Chinese translations, are not in this post.
• Small samples. Liao's analyses used 15 to 18 people, Pannatier had about 7 per elementary cell, Honma had 34. Some of the "no difference detected" results may simply be undetectable at that size.
• The subtitle experiments used short videos or allowed only one viewing. Pannatier: 5 minutes, no pausing; Liao: 5 minutes per clip; Wang: 23 minutes. This site attaches full lectures of one to two hours that can be paused and rewound.
• Versions. For Wang and Pellicer-Sánchez I read an earlier manuscript version; for Lee and colleagues and Salmerón and colleagues, author manuscripts; the published numbers were not checked. Pannatier's tables were not checked against the PDF layout.
• Observational data are not causal. Karp Gershon (language and HDI both inferred at the country level), Dasgupta and Hill, and Guo's survey.
• The phone evidence. Clinton-Lisell has only 3 direct phone-versus-paper effect sizes; Salmerón excludes phones; Honma's paper was phone-sized, and the body gives no score difference.
• Page weight. Russell's figures are one person's annual estimate, not peer-reviewed; this site's measurement covers one post, one screen size, unthrottled. I found no study on how page weight or network speed affects learning.
• My inferences, eight in all: treating EMI and MOOC results as analogous to a single article; reading Guo's self-reports together with the Karp Gershon and Lee results; Kwak's terminology problem and Guo's respondents pointing to the same place; scrolling research not answering questions about horizontal scrolling; comparing this site's transfer size against Russell's budget; the six items under "What you can do"; the three things to do tonight; and the "partly supported" rating in the comparison table. Each is marked in the text or listed here.
• Numbers I calculated myself: about 69% for Wang's English-caption group; Kizilcec and Cohen's absolute differences of 1.8 and 3.8 percentage points; about 20 Chinese characters per line on this site; the 1.79 MiB and 1.34 MiB conversions.
• Secondhand citations: Türkay and colleagues' 2017 figure of 60% is as reported by Karp Gershon and colleagues; Russell's median page weight has no source in the body text, and the hyperlink points to HTTP Archive.
• Known to exist, but I could not read a legally public full text: Kizilcec, Saltarelli, Reich, and Cohen's 2017 short article in Science, "Closing global achievement gaps in MOOCs"; Montero Perez and colleagues' 2013 captioning meta-analysis; Clinton's 2019 paper-versus-screen meta-analysis; Lin and colleagues' 2024 meta-analysis of interest-based personalization.
• Not found: experiments on how translation quality (human, machine, AI) affects learning; experiments on how culturally specific examples affect comprehension (Kurokawa has one supplementary analysis comparing culturally specific with neutral video content, measuring vocabulary, with no difference detected); the share and outcomes of phone use for learning in MOOCs.

Appendix: study designs and numbers

The main text keeps one or two numbers per item; the rest are here. Sign conventions differ across the screen studies, so all are written here in the direction of "paper better," with the original sign in parentheses.

Karp Gershon, Ruipérez-Valiente, Alexandron (2021), International Journal of Educational Technology in Higher Education 18. 158 runs and 120 courses of MITx on edX, 2013–2018, with 1,586,305 people who viewed course material at least once; the unit of analysis is the course run. English-native countries: US, Australia, Canada, Ireland, New Zealand, UK, Trinidad and Tobago; HDI split at 0.7. RQ1 (within high-HDI countries, native versus non-native): mean odds ratio across runs 1.076, p = .464. RQ2 (within non-native countries, high versus low HDI): mean 1.83, p < .001, 137 runs. RQ3: the mean difference in log odds ratios between runs of the same course was 0.16, and between different courses 0.30; from this the authors say the course explains about 47%, which is a ratio of the two differences, not a decomposition of the odds ratio. RQ4: among completers in six courses, five showed no assessment bias; the only one that did, 3.MatSelx, differed by 0.49 standard deviations (p = .022, 152 people). RQ5: after removing items flagged by two differential item functioning methods, the gap went from 0.49 to 0.50 under one and to 0.45 under the other, still significant. Limitations: IP misclassification, country-level HDI as only a rough estimate of individual socioeconomic status, MITx courses only.

Lee, Rose, Macaro, Lee (2025), System 133, read as the author accepted manuscript in Oxford ORA (manuscript title: Impact of English Medium Instruction in Higher Education: A Multi-Level Meta-Analysis). 28 studies, 41 independent samples, 8,747 people, 51 effect sizes. Content learning, within-group pre-post d = 1.57 (7 samples); content learning, between groups d = 0.13 (95% CI −0.04 to 0.31, 16 samples, 19 effect sizes), of which the 12 samples that confirmed group equivalence came to 0.04 (p = .68) and the 4 that did not to 0.41. English learning within groups 0.81 (14 samples); between groups 0.33 (only 4 samples, which the authors say is not suitable for generalizable conclusions). Moderators could only be analyzed by subgroup. The author manuscript has no appendices, so the details of the publication-bias analysis were not available.

Guo (2018), CHI 2018. See the appendix of Part 4.

Kizilcec and Cohen (2017), PNAS 114(17), read as the published version and supplementary information on PMC. Experiment 1: a 10-week graduate-level business course on Coursera, 9,619 people, 86 countries; Experiment 2: a 6-week introductory computing course on Open edX, 8,344 people, 87 countries, four groups (MCII, mental contrasting only, implementation intentions only, control). Culture was classified into three groups by the Hofstede individualism index of the IP country; India counted as culturally balanced in the experiments. Completion (control → MCII): Experiment 1 overall 5.42% → 6.32% (p = .097), individualist countries 5.52% → 7.28%; Experiment 2 overall 25.4% → 27.5% (p = .122), individualist countries 25.7% → 29.5%, collectivist countries 23.5% → 21.1% (not significant). The English proficiency index (EF EPI) correlated with individualism at r = 0.68; in Experiment 1 none of the four country-level interactions was significant (P > 0.16), and in Experiment 2 English proficiency was not significant (P > 0.09). Topic analysis of obstacles, share of "practical barriers": Experiment 1 individualist 27%, balanced 33%, collectivist 40%. Completion was measured, not learning.

Kizilcec, Reich, Yeomans, et al. (2020), PNAS 117(26). See the appendix of Part 19. The share of the focal sample reporting English fluency was 49.7% to 55.4% across waves. The original subgroup for the long-term planning prompt was learners who intended to complete and were fluent in English, with an original effect of 3.9 percentage points (2,053 people); after scaling, +0.19 in year one (26,586 people) and −0.23 in year two (10,372 people).

Pannatier and Bétrancourt (2024), Learning and Instruction 90, published version from the publisher's web page; tables not checked against the PDF layout. A 3 (English subtitles, French subtitles, none) × 3 (English level) between-subjects design, 131 people (by format 41 / 46 / 44; by level elementary 21, intermediate 55, advanced 55). Video 5 minutes, watched once. Main effect of subtitle format: labeled diagram p = .694, questionnaire p = .344, transfer p = .505; interactions p = .972, .511, .350. English level: questionnaire p = .004 (η²p = .114), transfer p = .003 (η²p = .090), labeled diagram p = .078. Transfer, advanced versus elementary d = 0.75. Subtitle format also had no effect on cognitive load or interest. Sample size was planned for f = 0.35, larger than a conventional medium effect.

Wang and Pellicer-Sánchez (2023), Studies in Second Language Acquisition 45(4), read as the earlier-version author manuscript in UCL Discovery; the published version was not checked. 112 recruited, 106 in the comprehension analysis, 100 in eye tracking. Out of 34: bilingual 27.93, Chinese subtitles 27.38, English captions 23.48, no subtitles 20.48. Bilingual versus Chinese p = .97; bilingual versus English captions d = 0.89; Chinese versus English captions d = 0.56; English captions versus none p = .06. Share of total reading time while subtitles were on screen: bilingual group subtitle area 0.60, image 0.39; within the bilingual group, Chinese line 0.42, English line 0.20; skipping rate for the English line 0.43, Chinese line 0.11. Longer reading of English captions went with lower comprehension, significant only in the English-caption group. Several numbers in the manuscript do not reconcile (one χ² incompatible with its p, one t incompatible with its p, and table d values that do not match the means), which may have been corrected in the published version.

Liao, Kruger, Doherty (2020), The Journal of Specialised Translation 33, published version. Within-subjects design, counterbalanced with a Latin square. 20 recruited; eye tracking 16 per cell, cognitive load 18, recall 15. Free recall (scale 1 to 24.5): Chinese subtitles 9.88, English subtitles 9.82, bilingual 10.83, none 8.45, F(3, 42) = 1.447, p = .243. Share of dwell time on the subtitle area: Chinese 21.55%, English 32.15%, bilingual 33.62%, with only bilingual greater than Chinese significant. Self-reported intrinsic load: none 18.78, higher than English subtitles and bilingual; extraneous load: none 14.56, higher than Chinese subtitles and bilingual; no difference detected among the three subtitle conditions. The original germane-load scale had very low reliability and was used only after dropping one item.

Kurokawa, Hein, Uchihara (2024), Language Learning 75(4), publisher's published PDF. Three-level meta-analysis, 49 experiments, 89 effect sizes; 43 experiments with English as the L2. Overall g = 0.56 (95% CI 0.38–0.73), Q(88) = 552.88. In the 18 experiments and 30 effect sizes where absolute learning could be computed: caption groups learned 13.75% of target words, no-caption groups 10.64%. Educational level significant: primary 1.36 (6 experiments), secondary 0.44, university 0.44. L2 proficiency not significant (p = .070), intermediate 0.81. Target audience of the video significant: made for learners 1.04, made for native speakers 0.46. Video type not significant: animation 0.56, documentary 0.60, drama 0.44, lectures and talks 0.14 (not significant). Pretest: same-format 0.35, different-format 0.29, no pretest 1.01. Only 8 experiments had delayed tests, which were not analyzed. Additional analysis requested by reviewers: culturally specific content 0.44 (18 effect sizes), neutral content 0.55, p = .581. Egger's test p = .316. The online supplementary materials were not read.

Kwak, Dunder, Viberg, Pardos (2025), L@S '25; the PDF's first page carries a CC BY-SA 4.0 license. Translation test set of 200 sentences (100 problems or hints, 100 section titles), with human translations by one native-Swedish-speaking author as reference. Problem sentences BLEU 84.42–94.25; title sentences 17.72 (mBART) to 61.27 (RAG). xCOMET was slightly higher on titles, for example GPT-4o problems 91.88, titles 92.95; RAG problems 92.06, titles 95.15. Title BLEU for GPT-4o without intervention 54.23, with RAG 61.27. Deployed content 2,544 words and 332 sentences, of which the instructor changed 30 words (1.18%) and 5 sentences (1.51%). Curriculum alignment: 92 textbook problems; the two methods together yielded 16 skills, of which the instructor judged 10 relevant and 6 not. Two gap figures in the body do not reconcile with Table 1; the table is taken as authoritative.

Dasgupta and Hill (2017), L@S 2017. See the appendix of Part 4.

Delgado, Vargas, Ackerman, Salmerón (2018), Educational Research Review 25. See the appendix of Part 9. Also used here: handheld devices k = 14, paper better by 0.12 (original −0.12, p = .11); computers k = 42, 0.23 (p < .001); difference between them QB = 1.55, not significant. Scrolling needed k = 37, 0.25; no scrolling k = 12, 0.13 (p = .06); difference QB = 1.99, not significant.

Clinton-Lisell and Litzinger (2026), Education and Information Technologies 31, Springer published version (CC BY). Frequentist network meta-analysis with paper as the reference; positive values mean paper better. 49 reports, 56 studies, 79 comparisons; expository text only, 2010 onward, reading in the first language, and no screen-only interactive features. Overall: computers 0.17 (0.09–0.25), e-readers 0.18 (p = .07), phones 0.26 (0.06–0.46), tablets 0.09 (p = .13); ranking scores paper 0.97, tablets 0.67, e-readers and computers 0.36, phones 0.13. Number of direct effect sizes: computer versus paper 39, tablet versus paper 16, e-reader versus paper 6, phone versus paper 3; for phone versus paper, direct estimate 0.12, indirect 0.41. Scrolling needed (25 studies, 29 comparisons, no e-readers): computers 0.35, tablets 0.38, phones 0.48 (−0.14 to 1.10). No scrolling (31 studies, 39 comparisons): computers 0.07, e-readers 0.12, phones 0.12, tablets 0.03, none significant. Egger's test p = .35. The body says 28 studies did not report reliability; counting the table gives 27.

Salmerón, Altamura, Delgado, Karagiorgi, Vargas (2024), Journal of Educational Psychology 116(2), read as the author manuscript in the University of Valencia repository; the online supplementary tables were not read. Tablets and e-readers only. 49 studies, 63 effect sizes. Between participants: after excluding 2 outliers, 38 effect sizes, paper better by 0.113 (original −0.113, p = .030), I² = 94.39%, prediction interval −0.486 to 0.259; with the outliers back in, 0.069, p = .191. Within participants: 21 effect sizes, 0.103 (p < .001), I² = 0, with no original study significant on its own. Between-participants moderators: university students 0.227, school students −0.057 (the school point estimate slightly favors the screen, not significant), p = .026; individual sessions 0.230, group sessions −0.004, p = .036; narrative 0.206, expository 0.083, both not significant; only 5 effect sizes involved scrolling. The only two large representative samples (US NAEP grades 4 and 8) both favored paper.

Honma et al. (2022), Scientific Reports 12, published version on PMC. 34 Japanese university students (20 women, mean age 20.4), within-subjects crossover, each reading two excerpts with neither medium nor novel repeated. Excerpts of 3,060 and 3,067 Japanese characters. Phone 5.0 inches; the paper matched the phone in type size, weight (148 g), and outer dimensions. Main effect of medium on reading scores F(1, 32) = 26.445, P < 0.0001; means and standard errors appear only in a figure, with no numbers in the text. No difference detected in reading time or viewing distance. More sighs with paper; the main effect of medium on prefrontal activity was not significant on either channel. Path analysis GFI = 0.881. Limitations: subjective cognitive load not measured, page-turning movements not controlled, applicability to other devices unconfirmed.

Russell (2025), The Performance Inequality Gap, 2026, author's blog, November 2025. Network 9 / 3 Mbps, 100 ms; test devices Samsung Galaxy A24 4G and an HP 14 laptop. Critical-path resource budgets (two TLS connections): within 3 seconds, a page that is 15% JavaScript gets 2.0 MiB in total and a page that is 50% JavaScript 1.2 MiB; within 5 seconds, 3.7 and 2.3 MiB. Median mobile page weight 2.6 MiB, with the 75th percentile above two copies of DOOM (2.48 MiB); median mobile JavaScript 680 KiB. Fewer than half of sites pass Core Web Vitals on mobile. 75th percentile by country: UK 21/7/34, US 17/5.5/47, global 9/3/100, India 6.2/2.1/85, Nigeria 3.1/1/190 (down / up Mbps / latency ms). The author says Cloudflare's data skew toward wealthier users.

This site's measurement, 2026-10-14 at 19:22 Taiwan time: Playwright with the Pixel 5 device profile loaded the Chinese version of Part 1 on the production site, unthrottled and with no cache. Total transfer 1,881,703 bytes (28 requests): font 1,404,865, HTML 149,535, JavaScript 289,227 (including the Cloudflare analytics script, 10,438), CSS 36,921. Uncompressed HTML 458,029 bytes, of which glossary data 270,403 bytes. Page height 25,994 px, screen height 727 px. Body text 16 px, line height 28.8 px, paragraph width 325 px. Table width 1,682 px, frame 367 px. The English version of the same post transferred 1,836,635 bytes (font 1,401,566), page height 39,213 px, with the opening table 3,078 px wide.

References

Each source was read in full in the version I obtained, with an extraction record; what I could not read is noted in parentheses.

• Karp Gershon, Ruipérez-Valiente & Alexandron (2021), Defining and measuring completion and assessment biases with respect to English language and development status: not all MOOCs are equal (publisher's published version; figures read from captions only).
• Lee, Rose, Macaro & Lee (2025), Effectiveness of EMI in higher education: A multi-level meta-analysis (read as the author accepted manuscript in Oxford ORA; appendices not available).
• Guo (2018), Non-Native English Speakers Learning Computer Programming: Barriers, Desires, and Design Opportunities (final version on the author's page).
• Kizilcec & Cohen (2017), Eight-minute self-regulation intervention raises educational attainment at scale in individualist but not collectivist cultures (published version and supplementary information on PMC; PDF layout not checked).
• Kizilcec, Reich, Yeomans et al. (2020), Scaling up behavioral science interventions in online education (published version on PMC and SI Appendix).
• Pannatier & Bétrancourt (2024), Learning from academic video with subtitles: When foreign language proficiency matters (open-access published version on the publisher's site; tables not checked against the PDF layout; supplementary materials not read).
• Wang & Pellicer-Sánchez (2023), Examining the effectiveness of bilingual subtitles for comprehension: An eye-tracking study (read as the earlier-version author manuscript in UCL Discovery; published version not read; supplementary materials not read).
• Liao, Kruger & Doherty (2020), The impact of monolingual and bilingual subtitles on visual attention, cognitive load, and comprehension (open-access published version on the journal's site).
• Kurokawa, Hein & Uchihara (2024), Incidental Vocabulary Acquisition Through Captioned Viewing: A Meta-Analysis (open-access published version from the publisher; online supplementary materials and OSF coding sheet not read).
• Kwak, Dunder, Viberg & Pardos (2025), Adaptive Tutoring Goes to Sweden: Machine Translation and Alignment of English OERs to a Swedish Calculus Course (published PDF, CC BY-SA 4.0; KTH DiVA record).
• Dasgupta & Hill (2017), Learning to Code in Localized Programming Languages (published version).
• Delgado, Vargas, Ackerman & Salmerón (2018), Don't throw away your printed books (full text in the University of Valencia repository; appendix table of individual studies not read).
• Clinton-Lisell & Litzinger (2026), Decoding digital reading: a network meta-analysis of comprehension across devices (open-access published version from Springer; figures read from captions only).
• Salmerón, Altamura, Delgado, Karagiorgi & Vargas (2024), Reading comprehension on handheld devices versus on paper (read as the author manuscript in the University of Valencia repository; online supplementary tables not read).
• Honma et al. (2022), Reading on a smartphone affects sigh generation, brain activity, and comprehension (published version on PMC; supplementary materials not read).
• Cloudflare Web Analytics documentation: Dimensions (used to show that this site's aggregate analytics include device type as a dimension).
• Russell (2025), The Performance Inequality Gap, 2026 (author's blog; values in the page's interactive charts not read).
• Posts on this site: Handling Terms, Symbols, and Formulas in a Teaching Article, Pairing a Teaching Article with Video, When a Reader Wants the Video Mid-Article, Writing Questions and Wording in a Teaching Article, Why Most Online Self-Learners Don't Finish, Writing the Examples and Openings of a Teaching Article.