I feel so sorry for Arabs now, just read that paragraph about everyday experience of trying to write English-Arabic text in the mail or any other editor.
> I have watched senior engineers, fluent in both Arabic and English, give up on writing a long email in Outlook on a Wednesday afternoon because the cursor would not behave, and switch to Arabic-only or English-only because the cognitive cost of fighting the editor exceeded the cost of monolingual phrasing. Actually I remember very well suffering this while using Facebook for the first time in my life, and I could not register; I was very slow typer that when I reached the moment the cursor does this weird thing, I would just stare at it and never progress.
> This is the ordinary experience of writing mixed Arabic-English text in 2026, in every major editor, email client, and chat application I know of. The pettier cousins are everywhere too, and I collect them: a range like 10–20 silently reading as twenty-to-ten, because digits are weak and the dash is neutral; a trailing exclamation mark teleporting to the far end of the line; a password, toggled visible, displaying in an order that does not match what was typed. None of these are anyone's bug, exactly.
My own Cyrillic struggles are nothing in comparison.
The complexities of mixed LR and RL text are quite astonishing since it’s not really even a case of just switching modes when switching scripts since double-nested (or more) texts can change the semantics of line breaks. This article provides a good overview: https://tug.org/TUGboat/tb08-1/tb17knutmix.pdf [1]
In college [2], when I wanted to quote some texts from Exodus in Hebrew in a paper that I wrote, I ended up avoiding the issue by hand-reversing the letter order and manually breaking lines. 8 bits is insufficient to cover all the possible combinations of letters and vowel markings so the font didn’t include any vowel markings and only did dageshim for בּ and פּ if I recall correctly.
⸻
1. As an aside, it would have been really nice if Unicode provided a R-L mirrored Latin alphabet to make it easier for monolingual developers to grasp the complexities surrounding mixed directional typesetting. I suppose it could still be added, although Unicode tends towards conservatism on adding additional characters.
2. This was 1990, well before Unicode in the era of a hundred or so 8-bit character encodings, most of which were not implemented widely. I also had to type the text using the arbitrary ASCII-Hebrew mapping of the font I was using which, among other things, led me to discover that letter frequency in Hebrew is much more uniform than it is in English.
RFC 3986 (STD 66) recommends (in appendix C) delimiting URLs in angle brackets to avoid the problem which your link now has. I.e. if you’d written <https://tug.org/TUGboat/tb08-1/tb17knutmix.pdf>¹ there would have been no problem.
I was lucky enough to date someone from the Gulf, so I forced myself to understand and read Arabic scripts (but not understand the text), which is a huge bonus when creating multi-lingual designs.
I already had a good understand of CJK scripts, and you'll come across RtL there, with things like tategaki which is both vertical and RtL at the same time (and can include quotes in other languages such as English and Arabic). Here's some lyrics I made in that format for reference:
I love this question. Basically all scripts have things that make them challenging to render, sometimes little things, sometimes bigger ones.
Cyrillic for Russian is reasonably straightforward, but it's also used for many other languages. The variation in style is particularly notable for Bulgarian[1]. A sophisticated font might have a "loca" table with locale-specific adjustments, but this is not universal yet, for example the issue to add it to Open Sans is still open[2]. To see the differences, try [3] and use the Language dropdown to select Bulgarian.
Wow, thank you for this response. This is actually fascinating and frustrating at the same time. So Bulgarian uses the same Unicode points, but also uses the language/culture metadata to let the font know which set to use.
I'm building out a multi-lingual wiki and just about to start adding Cyrillic support, and this really helps me understand that there is more research I need to do to support other Cyrillic languages.
Any time, friend. Your basic move is to set the lang metadata accurately and choose a font that does have these style differences implemented in the loca table.
Best of luck and feel free to reach out if you have more specific questions.
I'm already strict with marking the html pages with the correct lang and culture tags etc, but I'll be double-checking everything now. Thank you again for educating me :)
One thing I sometimes think about when I think about text layout problems is how the text we use also has a bunch of complexities that we can take for granted.
Think of variable width characters and kerning and ligatures and hyphenation and justification. Imagine computers had been won by a CJK language, which have none of these problems. You could imagine a similar article about how exotic and difficult English layout is.
Both Latin and Chinese have been modified by the technology used to write them.
When carved in stone the lines are much straighter. When written with brush or pen they became semi-cursive. When printing was introduced, they became grid-like and regular.
What westerners who are passingly familiar would think of as the standard Chinese typeface - the strict square grid with straight-line characters - arises in part from printing technology. Easy to carve that into wood blocks, and easy to line up the slots into a grid.
Latin was similarly morphed to fit into the realities of printing in the 1500s. And is still being morphed. Notice how numbers 123... are in-line and at the same height as the letters. That's a very modern convention, typewriter and computer influence on our orthography. Traditionally digits were more likely to appear as subscript, off-centre.
what selective pressures against oldstyle numerals with ascenders/descenders existed that wouldn't have equally applied to letterforms with those same features?
(aha i have found the answer to my own question: miniaturization for fractions in phototypesetting)
Not really. The selective pressure really comes well before that: Tabular presentation of numbers, whether that was log/trig tables or railroad time tables, there was a preference for uniform-width and regular height characters for those contexts (this is also why there is a number-width parameter in TT typography to enable a designer to let digits be variable-width in text but still allow tabular setting if desired).
the other part is that numbers and symbols were very much not the priority. The printing press was for books, magazines etc. math remained hand written until the computer
Nope, not at all. Monotype had a special system for doing math in hot metal typesetting. With handset type it was possible, but very time-consuming. You can find typeset mathematics going back centuries before the computer. There were also (somewhat impractical) systems for setting music with metal type although engraving was more common because of the interactions of lines and symbols.
Conversely, English has a joined form(cursive) that is nearly dead because mechanical text assistance devices (first typewriters, now computers) work much better with the block form. While sad in a cultural loss sort of way the joined form only really makes sense when the text is hand written.
I am not familiar with the history of Arabic typography, but I sort of assume there was an archaic block form and their current joined form is the result of many centuries of encoding hand writing practice. advanced enough that falling back to a block form is impossible with the side effect of making simple mechanical text formatting also impossible.
As for Chinese derived characters. we currently are able to jam them awkwardly into our alphabet optimized structures(one code per character) but I wonder if a Chinese native encoding would look different. Would it make sense to try and represent the sub-characters present in each Chinese character in the encoding? I suspect not, Chinese works, but it also does not appear amiable to simple mechanical assistance.
There's the https://en.wikipedia.org/wiki/Ideographic_Description_Charac... that kind of does that. The problem is that there's character divergence (see all the brouhaha about Unicode Han unification), so there needs to be something else to select variants too.
As a reference, I don't believe any of the pre-Unicode CJK&c encodings attempted that.
Another wrinkle with Arabic is linguistic conservatism. Due to Islamism and the idea that Arabic is the language of of God (the Quran was written in Arabic by the supposedly illiterate prophet), Arabic has lagged behind other languages in terms of innovation.
Hebrew is a closely related semitic language that simply adopted a block and cursive form. It has also been greatly simplified and friendlier towards loanwords, which has made it far easier to learn.
Muslims don’t believe Arabic is the language of God. They believe that the Quran was revealed in Arabic (true). Thinking the creator of the heavens and earth only speaks one language is absurd. It also kind of implies that Muslims believe in a superiority of Arabs which is also not true.
Weird to say Arabic hasn’t innovated or evolved considering the wild variety of dialects spoken in the modern world.
Conflating the language with the script is also bizarre. In terms of adapting Arabic to technology, look into romanized Arabic which was used before Unicode was common.
I didn't write "God only speaks Arabic" in Islam. That's your intepretation of my post. All I meant was that Arabic has special status in Islam.
> Weird to say Arabic hasn’t innovated or evolved considering the wild variety of dialects spoken in the modern world.
I didn't say Arabic has not innovated or evolved; only that it "has lagged behind other languages in terms of innovation". My belief is that that is due to linguistic conservatism, and linked to Islamism (or, at minimum, the centrality of Islam in Arab culture). Also related to this is the existence of Fusha, its place in Arab culture, and its branding as "modern standard Arabic".
I didn't conflate anything. While a script and a language are not the same, it's not a coincidence that Arabic is often written today in a script that is very close to Quranic script. And -- to really kick the hornet's nest -- it's also not a coincidence that there have been so few outstanding Arab writers (in Arabic) in the past 100 years. One novelist and a couple poets.
> And -- to really kick the hornet's nest -- it's also not a coincidence that there have been so few outstanding Arab writers (in Arabic) in the past 100 years. One novelist and a couple poets.
Now, reading that point one might ask the question if writing has been properly funded, or if the priority of cultural funding in the Arab world has been lower than, say, the funding of architecture and other forms of art. And on top of that, I'd also have a serious look at the market size, especially when compared with English-language writing.
With all due respect, your comment comes off as a bit ignorant and rude. A few points:
Firstly, the Qur'an wasn't written by the Prophet, he would dictate it and it would be written by his scribes.
Secondly, it's hard to argue that Islam has had a negative effect on Arabic or caused it to lag behind. In fact, it's easy to argue for the opposite. It's a historical fact that the Arabic language developed and proliferated rapidly due to the rise and spread of Islam. This is when its script and grammar were standardized, and when more and more works started being composed. And shortly thereafter the Islamic Golden Age began.
I don't have any issue with Hebrew, and maybe it is easier to learn. But this is because it was a dead language which was revived, resulting in a simplified language. Almost every other major language on Earth will have the same amount of "innovation" as Arabic. In fact, Arabic has many colloquial dialects which are used in day to day conversations, and these do consist of a simplified version with many loanwords. So I really don't know what you mean by a lack of innovation.
I don’t think anybody said that Arabic has suffered a complete standstill, and it has doubtlessly evolved significantly.
But if you compare it with basically any other major language, it’s clearly much, much more conservative. If you are a native English speaker, understanding English from 1,000 years ago is like learning a completely different language. If you are a native speaker of Italian, you cannot understand a text in Latin without significant training. This is true for all European languages other than Icelandic.
Chinese is pretty similar, even though the written language is slightly more stable.
So in comparison, Arabic is incredibly conservative.
There is no one "Arabic". Yes, formal modern Arabic (fusha) is based on (but not identical to) the classical Arabic of the Quran, but nobody speaks this in real life. The actual Arabics are the 20-odd spoken languages, many of which are effectively different languages at this point:
A rough equivalent in both time and space is how the Vatican continues to use Latin, but the rest of the Roman Empire has splintered into Italian, French, Spanish, Romanian, etc.
They speak it on tv and it's written in newpapers. They learn it in schools. Educated Arabs code switch into Fusha all the time. Islamist leaders (e.g. Nasrallah) speak Fusha in their broadcast speeches.
It's also pretty hard for foreigners to learn an ammiyya (outside of immersion). "Studying Arabic" almost always means Fusha.
I agree with you that "the actual Arabics are the 20-odd spoken languages". In a healhier culture, Fusha wouldn't exist or would have the same cultural place as Latin in the Western world.
There is no unjoined form of Arabic. The Arabic script became Arabic when Nabataean script started developing joined letter forms. Unjoined Nabatean is as foreign to Arabic as Phoenician is to Greek.
Looking at dictionaries and printing presses from China before the invention of computers reveals that they probably would have done something similar to ascii, just with more bits to encompass all the characters.
Chinese apparently was originally always written vertically top-to-bottom. (And then columns would be right-to-left.) Modern Chinese just rotates everything except the characters themselves 90 degrees to the Latin order.
I also read that a few Chinese texts only make sense in vertical order: one had a pun where the characters read one way as separated characters, but as stacked was also a single character pun for something like a "crumbly cookie".
Very interesting. I just implemented a text shaper and renderer from scratch with support for complex scripts like Arabic, Nastaliq and Indic (will soon post about it here on HN). Now that you write about it, the lack of stretching really is a deficiency in the OpenType spec.
If you want a solution for this it has to happen in the rendering step, not the shaping (which is HarfBuzz's main task). The shaper has no information about the available space, but when rendering you could stretch individual glyphs to the desired width, similar to adjusting the width of whitespace in Latin, but more complex, because you actually have to modify the glyphs with a scale transform. I am not an expert on Arabic script by any means, but this should be possible IMO. It would at least be an interesting experiment. Of course the JSTF table would be the right way to do it, but there seems to be a lot of confusion around it. Maybe in the age of LLMs we can give it another shot.
It does seem like a KP-like algorithm ought to be able to optimize the break positions without extreme algorithmic difficulty aside from the inputs being considerably more complex than for Latin block print: the cost function for a proposed line is a straightforward [0] calculable function of the contents of the line, and I think one could make a dynamic programming algorithm that tracks, for each input position, the cost of the optimal layout of all text up to that position with a break at that position. This gives an algorithm that takes cubic time. (For input length n, you need to fill in n values in the table. Each value scans the entire table before that position and does a calculation with complexity linear in the proposed line length.)
As a practical matter, there’s an input length n and there is some upper bound B on a credible line length as measured in code points, so there are only at most n*B credible proposed lines to evaluate, which also limits the useful look back on the table to B positions, so I think the time complexity could be reduced to O(n*B^2) without making the results worse on reasonable inputs, and this is probably quite tolerable.
[0] Straightforward once you’ve implemented the whole Arabic rendering stack, anyway. I am certainly not qualified to calculate this function :)
> but when rendering you could stretch individual glyphs to the desired width
“Individual glyphs” :)
It’s Arabic, so you wouldn’t stretch a single glyph, id would have to e done after shaping so you can work out the next run (either a single Aleph or the joined characters) in order to know what is stretchable (then throw it to your layout step)
Internet Explorer 5.5 implements text-justify: kashida. For one brief, weird browser-quarter Microsoft is the only software vendor on earth that can justify Arabic correctly on a screen.
This explains a lot of why I think Arabic looks so beautiful on the page. I also love the way it can sound when spoken. Shame I don't have reason good enough to take the years I would need to learn it.
Learning language is like learning poetry. Your appreciation of the thing grows proportionally with the practice. Which means you don't need to learn to fluency to appreciate Arabic. Even modest practice, just learning to write the alphabet, will increase your appreciation for the language if that's what you'd like to do.
Have experienced this with a bilingual relative whos language i have learned bits and pieces of both of us enjoy it. They give you little challenges from time to time too which is fun
I actually like that they depend on position, it even carries semantic information when reading because of how conjugation works so it actually helps you read faster
This article is wonderful. It's interesting, it's captivating, full with detail, and to think I never gave much thought about Arabic rendering before.
This part nearly had me chuckle audibly:
He says yes. The result is "Simplified Arabic": initial fused into medial, final into isolated, ligatures dropped. It conquers the Arab newsroom in a generation. Mrowa is assassinated at his desk eight years later, by an unrelated faction, in an unrelated dispute.
Also, it's depressing how hundreds of millions of people couldn't even get their language typeset on a computer, and our industry meanwhile was busy building AI-native AI for your groceries (have we mentioned it has AI btw?) and similar performative bullshit.
AI can probably solve "how do we get good shaper implementations into software", provided a good enough spec and test suite are available, but it won't solve "how do we convince stakeholders to value supporting languages spoken by massive numbers of people", most likely
> Also, it's depressing how hundreds of millions of people couldn't even get their language typeset on a computer, and our industry meanwhile was busy building AI-native AI for your groceries (have we mentioned it has AI btw?) and similar performative bullshit.
AI also brought you this "wonderful" article, I would note.
No, it's not. (As should not be a surprise given how extensively LLMs are used to draft English-language articles these days.) It's actually incredibly blatant. Just look at this second paragraph:
> The same six months I had closed three other tickets against the same product, each of which had presented to its filer as the only bug. A customer's name had appeared with its letters unjoined on a printed agreement, the way a sign-painter would have laid them out in 1962, because the PDF library on the receipt server pre-dated the existence of a shaping engine in its language runtime. A search index had been returning empty for accounts the customer service team could see in the database because a 2017 import had encoded twelve thousand names using fossil Unicode codepoints from 1991 instead of regular ones from 1995, and the index, very reasonably, treated the two encodings as different strings, So, that ragged-left ticket was the smallest of the four, HOWEVER, it sat on top of the same iceberg and pointed at the same thing.
And you can punch the essay into Pangram if you have any doubt (omit Arabic text and formatting if you do this, focus on just plain English to be safe). For example, go to the end* and try "Everything in this story that actually works was paid for by almost nobody...Somebody will close it, probably unpaid, possibly reading this (or writing it? who knows)." '100% AI.'
* I always check sections towards the end instead of the beginning, because a lot of authors will write the introduction by hand and then give up and let the AI write the rest; and also more advanced sloppers will fiddle with the opening until it beats Pangram, which is not hard since Pangram heavily favors false negatives on AI contribution, and skip the rest because they assume readers will be too lazy to check beyond that.
> The relevant rule, W2 of UAX #9, reclassifies a digit as an ARABIC NUMBER if any of the previous strong characters in the paragraph were Arabic letters, and as a EUROPEAN NUMBER otherwise. Both render their internal digits left-to-right, which is correct: numbers everywhere on Earth are read most-significant-first.
Does the author mean most-significant-on-the-left? The statement as written is a statement about the order in which one reads or perhaps thinks the number, whereas I think the author is discussing how numbers, including collections of numbers delimited by hyphens and such, should be laid out on the page.
This is a thing I have wondered about. are arabic numbers little endian when embeded in arabic text? The text is read right to left. But the numbers are the same as we put them in our left to right text where they are big-endian(read big to small). so are numbers in original arabic little-endian? read small to large. with the interesting side effect that numbers are universal.
I think least significant first is the most natural order for written numbers (and it makes addition easier.)
I had always assumed that was what was intended with Arabic numbers, only silly Europeans made a mistake when they borrowed the positional system and forgot Arabic is written the other way. (Or perhaps intentionally avoided mirroring the digits for ease of communication?)
But the author of this article makes it sound like even in Arabic, numbers are read out loud most significant first.
I think he's talking about the rendering algorithm with regards to the stream of text. essentially saying rendering direction should follow reading convention.
on the other hand, in formal arabic, it's not unusual that numers are read in clusters from least significant to most significant (right to left). 1984 would be read : eighty four and nine hundred and a thousand. not sure if the author is aware of this
> rendering direction should follow reading convention.
What does that even mean in this context? In a strictly LTR language, sure, you read left-to-right and the glyphs are rendered left-to-right. But the whole discussion is about bidirectional text, where the text is rendered by a complex algorithm. What is the “rendering direction”?
I know just enough about some RTL languages to know that one can absolutely intersperse RTL text with, say, and English phrase, and you still read the first (leftmost in the group) English sound first and so on :)
if you write (with a pen) text with mixed arabic, numbers, english words. then somebody else gets to read it, he will read arabic from right to left, then when when he encouters a number or an english word, he will naturally jump to the first latin letter of the word and start reading left to right, then jump back to the beginning if the next arabic word and switch back to RTL. the alogorithm should copy that behaviour.
Believe it or not, there once was an 8-bit computer with native Arabic. It was a version of the Z-80 MSX series by Al Alamiah called Sakhr. It seems it could switch between English, Arabic, or mixed. (In fact, this was mentioned in the OP.)
Arabic script is a great test to see if your terminal/renderer/UI can handle anything: contexual shaping, cursive connectivity, bidirectional text layout, diacritics and vertical displacement.
I went down this rabbit-hole awhile back and it made me really appreciate the complexity of the script.
> Editable; go on, click in. Place the caret at the start of the line, hold the Left arrow, and enjoy the moment it reaches the version number, where it will jump, double back, or sail through depending on your browser.
what's missing here is "use shift + arrow keys for selection"
Firefox manages movement just fine. Selection tho? oh boi
To me it sounds like it's high time Arabic got a new script that's compatible with computers. I've seen some Lebanese people use mix of Latin letters and digits to type, so the idea isn't entirely stupid.
Computers are tools that exist to serve humans. Humans should never be asked to become subservient to computers. Your suggestion is ludicrous and risible.
The ability to adapt to changing environment is one of core properties of life. You cannot un-invent the printing press. Imagine making an argument "nooooo you cannot print books, it takes away the human element" and then "nooooo you cannot write down stories, it takes away the human element" and then being obliberated by a civilization that embraced new inventions instead of rejecting them based on "no means no".
I could write all day about how badly RtL languages are treated on electronic devices. Most interfaces just slop RtL text into an interface that is otherwise the totally wrong way around.
One thing that amuses me is that people share these "safe zone" templates for short form video to make sure your content isn't hidden behind the buttons:
That might be because of translations from Arabic. The article was also posted on a different website where the author responded
> the Kashida section was contributed to this post from a talk in Arabic of Nawal Hadeed, which she translated and added to the post herself. Although I'm unsure of LLM usage in the translation process, looking at the original Arabic I felt some change in tone while editing the post. I could have either declined the translation and never have this documented, procrastinate in translating it myself (which has been ongoing for a while), or publish as it is. I found the last least damaging.
Disclaimer: I’m not fluent in Arabic by any means, but the stretched out to both margins style looks very Quranic to me. I don’t think it looks appropriate for say, a message about my DoorDasher.
I feel so sorry for Arabs now, just read that paragraph about everyday experience of trying to write English-Arabic text in the mail or any other editor.
> I have watched senior engineers, fluent in both Arabic and English, give up on writing a long email in Outlook on a Wednesday afternoon because the cursor would not behave, and switch to Arabic-only or English-only because the cognitive cost of fighting the editor exceeded the cost of monolingual phrasing. Actually I remember very well suffering this while using Facebook for the first time in my life, and I could not register; I was very slow typer that when I reached the moment the cursor does this weird thing, I would just stare at it and never progress.
> This is the ordinary experience of writing mixed Arabic-English text in 2026, in every major editor, email client, and chat application I know of. The pettier cousins are everywhere too, and I collect them: a range like 10–20 silently reading as twenty-to-ten, because digits are weak and the dash is neutral; a trailing exclamation mark teleporting to the far end of the line; a password, toggled visible, displaying in an order that does not match what was typed. None of these are anyone's bug, exactly.
My own Cyrillic struggles are nothing in comparison.
The complexities of mixed LR and RL text are quite astonishing since it’s not really even a case of just switching modes when switching scripts since double-nested (or more) texts can change the semantics of line breaks. This article provides a good overview: https://tug.org/TUGboat/tb08-1/tb17knutmix.pdf [1]
In college [2], when I wanted to quote some texts from Exodus in Hebrew in a paper that I wrote, I ended up avoiding the issue by hand-reversing the letter order and manually breaking lines. 8 bits is insufficient to cover all the possible combinations of letters and vowel markings so the font didn’t include any vowel markings and only did dageshim for בּ and פּ if I recall correctly.
⸻
1. As an aside, it would have been really nice if Unicode provided a R-L mirrored Latin alphabet to make it easier for monolingual developers to grasp the complexities surrounding mixed directional typesetting. I suppose it could still be added, although Unicode tends towards conservatism on adding additional characters.
2. This was 1990, well before Unicode in the era of a hundred or so 8-bit character encodings, most of which were not implemented widely. I also had to type the text using the arbitrary ASCII-Hebrew mapping of the font I was using which, among other things, led me to discover that letter frequency in Hebrew is much more uniform than it is in English.
RFC 3986 (STD 66) recommends (in appendix C) delimiting URLs in angle brackets to avoid the problem which your link now has. I.e. if you’d written <https://tug.org/TUGboat/tb08-1/tb17knutmix.pdf>¹ there would have been no problem.
That link’s a 404.
The link says
https://tug.org/TUGboat/tb08-1/tb17knutmix.pdf ¹
There is no space between pdf and ¹, so the HN server assumes incorrectly that the ¹ is part of the link.
Strip the superscript-1 character at the end, I'm surprised HNs link formatter regex detects it as part of the link: https://tug.org/TUGboat/tb08-1/tb17knutmix.pdf
I was lucky enough to date someone from the Gulf, so I forced myself to understand and read Arabic scripts (but not understand the text), which is a huge bonus when creating multi-lingual designs.
I already had a good understand of CJK scripts, and you'll come across RtL there, with things like tategaki which is both vertical and RtL at the same time (and can include quotes in other languages such as English and Arabic). Here's some lyrics I made in that format for reference:
https://codepen.io/kingcharlesone/pen/GgRXLoM
What peculiarities does Cyrillic text have? I've never learned to convert Cyrillic to Latin.
I love this question. Basically all scripts have things that make them challenging to render, sometimes little things, sometimes bigger ones.
Cyrillic for Russian is reasonably straightforward, but it's also used for many other languages. The variation in style is particularly notable for Bulgarian[1]. A sophisticated font might have a "loca" table with locale-specific adjustments, but this is not universal yet, for example the issue to add it to Open Sans is still open[2]. To see the differences, try [3] and use the Language dropdown to select Bulgarian.
[1]: https://en.wikipedia.org/wiki/Bulgarian_alphabet
[2]: https://github.com/googlefonts/opensans/issues/114
[3]: https://localfonts.eu/freefonts/traditional-cyrillic-free-fo...
Wow, thank you for this response. This is actually fascinating and frustrating at the same time. So Bulgarian uses the same Unicode points, but also uses the language/culture metadata to let the font know which set to use.
I'm building out a multi-lingual wiki and just about to start adding Cyrillic support, and this really helps me understand that there is more research I need to do to support other Cyrillic languages.
Any time, friend. Your basic move is to set the lang metadata accurately and choose a font that does have these style differences implemented in the loca table.
Best of luck and feel free to reach out if you have more specific questions.
So I went to check on the font that I'd chosen for my European/Latin/Cyrillic body text, and I guess I got very lucky with my pick:
https://github.com/googlefonts/literata
I'm already strict with marking the html pages with the correct lang and culture tags etc, but I'll be double-checking everything now. Thank you again for educating me :)
I recently learned how messed up URLs are in RtL and I'm also eternally grateful to not have to deal with that. Simulated example:
It seems like there are probably some phishing attacks based on this.Monday's editor does this even for English text!
One thing I sometimes think about when I think about text layout problems is how the text we use also has a bunch of complexities that we can take for granted.
Think of variable width characters and kerning and ligatures and hyphenation and justification. Imagine computers had been won by a CJK language, which have none of these problems. You could imagine a similar article about how exotic and difficult English layout is.
Both Latin and Chinese have been modified by the technology used to write them.
When carved in stone the lines are much straighter. When written with brush or pen they became semi-cursive. When printing was introduced, they became grid-like and regular.
What westerners who are passingly familiar would think of as the standard Chinese typeface - the strict square grid with straight-line characters - arises in part from printing technology. Easy to carve that into wood blocks, and easy to line up the slots into a grid.
Latin was similarly morphed to fit into the realities of printing in the 1500s. And is still being morphed. Notice how numbers 123... are in-line and at the same height as the letters. That's a very modern convention, typewriter and computer influence on our orthography. Traditionally digits were more likely to appear as subscript, off-centre.
what selective pressures against oldstyle numerals with ascenders/descenders existed that wouldn't have equally applied to letterforms with those same features?
(aha i have found the answer to my own question: miniaturization for fractions in phototypesetting)
Not really. The selective pressure really comes well before that: Tabular presentation of numbers, whether that was log/trig tables or railroad time tables, there was a preference for uniform-width and regular height characters for those contexts (this is also why there is a number-width parameter in TT typography to enable a designer to let digits be variable-width in text but still allow tabular setting if desired).
the other part is that numbers and symbols were very much not the priority. The printing press was for books, magazines etc. math remained hand written until the computer
Nope, not at all. Monotype had a special system for doing math in hot metal typesetting. With handset type it was possible, but very time-consuming. You can find typeset mathematics going back centuries before the computer. There were also (somewhat impractical) systems for setting music with metal type although engraving was more common because of the interactions of lines and symbols.
Conversely, English has a joined form(cursive) that is nearly dead because mechanical text assistance devices (first typewriters, now computers) work much better with the block form. While sad in a cultural loss sort of way the joined form only really makes sense when the text is hand written.
I am not familiar with the history of Arabic typography, but I sort of assume there was an archaic block form and their current joined form is the result of many centuries of encoding hand writing practice. advanced enough that falling back to a block form is impossible with the side effect of making simple mechanical text formatting also impossible.
As for Chinese derived characters. we currently are able to jam them awkwardly into our alphabet optimized structures(one code per character) but I wonder if a Chinese native encoding would look different. Would it make sense to try and represent the sub-characters present in each Chinese character in the encoding? I suspect not, Chinese works, but it also does not appear amiable to simple mechanical assistance.
There's the https://en.wikipedia.org/wiki/Ideographic_Description_Charac... that kind of does that. The problem is that there's character divergence (see all the brouhaha about Unicode Han unification), so there needs to be something else to select variants too.
As a reference, I don't believe any of the pre-Unicode CJK&c encodings attempted that.
Another wrinkle with Arabic is linguistic conservatism. Due to Islamism and the idea that Arabic is the language of of God (the Quran was written in Arabic by the supposedly illiterate prophet), Arabic has lagged behind other languages in terms of innovation.
Hebrew is a closely related semitic language that simply adopted a block and cursive form. It has also been greatly simplified and friendlier towards loanwords, which has made it far easier to learn.
Muslims don’t believe Arabic is the language of God. They believe that the Quran was revealed in Arabic (true). Thinking the creator of the heavens and earth only speaks one language is absurd. It also kind of implies that Muslims believe in a superiority of Arabs which is also not true.
Weird to say Arabic hasn’t innovated or evolved considering the wild variety of dialects spoken in the modern world.
Conflating the language with the script is also bizarre. In terms of adapting Arabic to technology, look into romanized Arabic which was used before Unicode was common.
I didn't write "God only speaks Arabic" in Islam. That's your intepretation of my post. All I meant was that Arabic has special status in Islam.
> Weird to say Arabic hasn’t innovated or evolved considering the wild variety of dialects spoken in the modern world.
I didn't say Arabic has not innovated or evolved; only that it "has lagged behind other languages in terms of innovation". My belief is that that is due to linguistic conservatism, and linked to Islamism (or, at minimum, the centrality of Islam in Arab culture). Also related to this is the existence of Fusha, its place in Arab culture, and its branding as "modern standard Arabic".
I didn't conflate anything. While a script and a language are not the same, it's not a coincidence that Arabic is often written today in a script that is very close to Quranic script. And -- to really kick the hornet's nest -- it's also not a coincidence that there have been so few outstanding Arab writers (in Arabic) in the past 100 years. One novelist and a couple poets.
> And -- to really kick the hornet's nest -- it's also not a coincidence that there have been so few outstanding Arab writers (in Arabic) in the past 100 years. One novelist and a couple poets.
Now, reading that point one might ask the question if writing has been properly funded, or if the priority of cultural funding in the Arab world has been lower than, say, the funding of architecture and other forms of art. And on top of that, I'd also have a serious look at the market size, especially when compared with English-language writing.
With all due respect, your comment comes off as a bit ignorant and rude. A few points:
Firstly, the Qur'an wasn't written by the Prophet, he would dictate it and it would be written by his scribes.
Secondly, it's hard to argue that Islam has had a negative effect on Arabic or caused it to lag behind. In fact, it's easy to argue for the opposite. It's a historical fact that the Arabic language developed and proliferated rapidly due to the rise and spread of Islam. This is when its script and grammar were standardized, and when more and more works started being composed. And shortly thereafter the Islamic Golden Age began.
I don't have any issue with Hebrew, and maybe it is easier to learn. But this is because it was a dead language which was revived, resulting in a simplified language. Almost every other major language on Earth will have the same amount of "innovation" as Arabic. In fact, Arabic has many colloquial dialects which are used in day to day conversations, and these do consist of a simplified version with many loanwords. So I really don't know what you mean by a lack of innovation.
I don’t think anybody said that Arabic has suffered a complete standstill, and it has doubtlessly evolved significantly.
But if you compare it with basically any other major language, it’s clearly much, much more conservative. If you are a native English speaker, understanding English from 1,000 years ago is like learning a completely different language. If you are a native speaker of Italian, you cannot understand a text in Latin without significant training. This is true for all European languages other than Icelandic.
Chinese is pretty similar, even though the written language is slightly more stable.
So in comparison, Arabic is incredibly conservative.
There is no one "Arabic". Yes, formal modern Arabic (fusha) is based on (but not identical to) the classical Arabic of the Quran, but nobody speaks this in real life. The actual Arabics are the 20-odd spoken languages, many of which are effectively different languages at this point:
https://en.wikipedia.org/wiki/Varieties_of_Arabic
A rough equivalent in both time and space is how the Vatican continues to use Latin, but the rest of the Roman Empire has splintered into Italian, French, Spanish, Romanian, etc.
> but nobody speaks this in real life
They speak it on tv and it's written in newpapers. They learn it in schools. Educated Arabs code switch into Fusha all the time. Islamist leaders (e.g. Nasrallah) speak Fusha in their broadcast speeches.
It's also pretty hard for foreigners to learn an ammiyya (outside of immersion). "Studying Arabic" almost always means Fusha.
I agree with you that "the actual Arabics are the 20-odd spoken languages". In a healhier culture, Fusha wouldn't exist or would have the same cultural place as Latin in the Western world.
Also worth noting that unlike Arabic and Islam, the Jewish tradition is that Hebrew is in fact the language of God and was the pre-Babel language.
Twitter trolls are on HN now?
This is such a bad take on the issue.
There is no unjoined form of Arabic. The Arabic script became Arabic when Nabataean script started developing joined letter forms. Unjoined Nabatean is as foreign to Arabic as Phoenician is to Greek.
Looking at dictionaries and printing presses from China before the invention of computers reveals that they probably would have done something similar to ascii, just with more bits to encompass all the characters.
CJK languages can include vertical and RtL stretches too, to complicate matters. Here's some lyrics I made as a test:
https://codepen.io/kingcharlesone/pen/GgRXLoM
Japanese magazines usually mix three different script types on a majority of the pages like this:
https://imgur.com/a/x61XbIV
(In another quirk some Japanese mags open right-bound, others open left-bound)
Chinese apparently was originally always written vertically top-to-bottom. (And then columns would be right-to-left.) Modern Chinese just rotates everything except the characters themselves 90 degrees to the Latin order.
I also read that a few Chinese texts only make sense in vertical order: one had a pun where the characters read one way as separated characters, but as stacked was also a single character pun for something like a "crumbly cookie".
A more academic treatment of justifying Arabic-script text can be found in https://quod.lib.umich.edu/j/jep/3336451.0023.104?view=text;...
awesome article! i appreciated the alternatives and misc. workarounds to OT/unicode typesetting toward the end, very helpful :D (go blue!)
Very interesting. I just implemented a text shaper and renderer from scratch with support for complex scripts like Arabic, Nastaliq and Indic (will soon post about it here on HN). Now that you write about it, the lack of stretching really is a deficiency in the OpenType spec.
If you want a solution for this it has to happen in the rendering step, not the shaping (which is HarfBuzz's main task). The shaper has no information about the available space, but when rendering you could stretch individual glyphs to the desired width, similar to adjusting the width of whitespace in Latin, but more complex, because you actually have to modify the glyphs with a scale transform. I am not an expert on Arabic script by any means, but this should be possible IMO. It would at least be an interesting experiment. Of course the JSTF table would be the right way to do it, but there seems to be a lot of confusion around it. Maybe in the age of LLMs we can give it another shot.
It does seem like a KP-like algorithm ought to be able to optimize the break positions without extreme algorithmic difficulty aside from the inputs being considerably more complex than for Latin block print: the cost function for a proposed line is a straightforward [0] calculable function of the contents of the line, and I think one could make a dynamic programming algorithm that tracks, for each input position, the cost of the optimal layout of all text up to that position with a break at that position. This gives an algorithm that takes cubic time. (For input length n, you need to fill in n values in the table. Each value scans the entire table before that position and does a calculation with complexity linear in the proposed line length.)
As a practical matter, there’s an input length n and there is some upper bound B on a credible line length as measured in code points, so there are only at most n*B credible proposed lines to evaluate, which also limits the useful look back on the table to B positions, so I think the time complexity could be reduced to O(n*B^2) without making the results worse on reasonable inputs, and this is probably quite tolerable.
[0] Straightforward once you’ve implemented the whole Arabic rendering stack, anyway. I am certainly not qualified to calculate this function :)
> but when rendering you could stretch individual glyphs to the desired width
“Individual glyphs” :)
It’s Arabic, so you wouldn’t stretch a single glyph, id would have to e done after shaping so you can work out the next run (either a single Aleph or the joined characters) in order to know what is stretchable (then throw it to your layout step)
This explains a lot of why I think Arabic looks so beautiful on the page. I also love the way it can sound when spoken. Shame I don't have reason good enough to take the years I would need to learn it.
Learning language is like learning poetry. Your appreciation of the thing grows proportionally with the practice. Which means you don't need to learn to fluency to appreciate Arabic. Even modest practice, just learning to write the alphabet, will increase your appreciation for the language if that's what you'd like to do.
Have experienced this with a bilingual relative whos language i have learned bits and pieces of both of us enjoy it. They give you little challenges from time to time too which is fun
It's funny -- I hate the way Arabic sounds when spoken, even though I agree it looks pretty. To my ears French and Japanese sound the nicest.
I'd like to see some more mainstream usage of disconnected fonts for Arabic, for example like these: https://www.arabaddigital.com/en/article/2100-Quarantining-o...
This was once attempted, and in a way such that each letter needed only one character no matter it's position in the word
Unfortunately it died
https://worksthatwork.com/6/unified-arabic
I actually like that they depend on position, it even carries semantic information when reading because of how conjugation works so it actually helps you read faster
This article is wonderful. It's interesting, it's captivating, full with detail, and to think I never gave much thought about Arabic rendering before.
This part nearly had me chuckle audibly:
He says yes. The result is "Simplified Arabic": initial fused into medial, final into isolated, ligatures dropped. It conquers the Arab newsroom in a generation. Mrowa is assassinated at his desk eight years later, by an unrelated faction, in an unrelated dispute.
Also, it's depressing how hundreds of millions of people couldn't even get their language typeset on a computer, and our industry meanwhile was busy building AI-native AI for your groceries (have we mentioned it has AI btw?) and similar performative bullshit.
maybe AI will solve this ?
AI can probably solve "how do we get good shaper implementations into software", provided a good enough spec and test suite are available, but it won't solve "how do we convince stakeholders to value supporting languages spoken by massive numbers of people", most likely
> Also, it's depressing how hundreds of millions of people couldn't even get their language typeset on a computer, and our industry meanwhile was busy building AI-native AI for your groceries (have we mentioned it has AI btw?) and similar performative bullshit.
AI also brought you this "wonderful" article, I would note.
What? This is one of the most clearly human written articles I could imagine. Can you explain what you mean?
No, it's not. (As should not be a surprise given how extensively LLMs are used to draft English-language articles these days.) It's actually incredibly blatant. Just look at this second paragraph:
> The same six months I had closed three other tickets against the same product, each of which had presented to its filer as the only bug. A customer's name had appeared with its letters unjoined on a printed agreement, the way a sign-painter would have laid them out in 1962, because the PDF library on the receipt server pre-dated the existence of a shaping engine in its language runtime. A search index had been returning empty for accounts the customer service team could see in the database because a 2017 import had encoded twelve thousand names using fossil Unicode codepoints from 1991 instead of regular ones from 1995, and the index, very reasonably, treated the two encodings as different strings, So, that ragged-left ticket was the smallest of the four, HOWEVER, it sat on top of the same iceberg and pointed at the same thing.
Blatantly Claude (which he recently started using, judging by https://lr0.org/diary/2026-02-26/ and https://lr0.org/diary/#08062026 - note how LLM-written the second one sounds, a little ironically).
And you can punch the essay into Pangram if you have any doubt (omit Arabic text and formatting if you do this, focus on just plain English to be safe). For example, go to the end* and try "Everything in this story that actually works was paid for by almost nobody...Somebody will close it, probably unpaid, possibly reading this (or writing it? who knows)." '100% AI.'
Or just compare it to his older writings. Does this 2023 piece https://lr0.org/blog/p/d/ or this 2024 piece https://lr0.org/blog/p/democracy/ sound like OP?
* I always check sections towards the end instead of the beginning, because a lot of authors will write the introduction by hand and then give up and let the AI write the rest; and also more advanced sloppers will fiddle with the opening until it beats Pangram, which is not hard since Pangram heavily favors false negatives on AI contribution, and skip the rest because they assume readers will be too lazy to check beyond that.
Is this a small typo?
> The relevant rule, W2 of UAX #9, reclassifies a digit as an ARABIC NUMBER if any of the previous strong characters in the paragraph were Arabic letters, and as a EUROPEAN NUMBER otherwise. Both render their internal digits left-to-right, which is correct: numbers everywhere on Earth are read most-significant-first.
Does the author mean most-significant-on-the-left? The statement as written is a statement about the order in which one reads or perhaps thinks the number, whereas I think the author is discussing how numbers, including collections of numbers delimited by hyphens and such, should be laid out on the page.
This is a thing I have wondered about. are arabic numbers little endian when embeded in arabic text? The text is read right to left. But the numbers are the same as we put them in our left to right text where they are big-endian(read big to small). so are numbers in original arabic little-endian? read small to large. with the interesting side effect that numbers are universal.
I think least significant first is the most natural order for written numbers (and it makes addition easier.)
I had always assumed that was what was intended with Arabic numbers, only silly Europeans made a mistake when they borrowed the positional system and forgot Arabic is written the other way. (Or perhaps intentionally avoided mirroring the digits for ease of communication?)
But the author of this article makes it sound like even in Arabic, numbers are read out loud most significant first.
Arabic itself got the number system from India, whose scripts are left-to-right
My understanding is that It Depends, but also feast your eyes on this photo I took, which was very confusing to me:
https://imgur.com/a/jL8lfZ1
That's a great example of the problem :)
I think he's talking about the rendering algorithm with regards to the stream of text. essentially saying rendering direction should follow reading convention.
on the other hand, in formal arabic, it's not unusual that numers are read in clusters from least significant to most significant (right to left). 1984 would be read : eighty four and nine hundred and a thousand. not sure if the author is aware of this
> rendering direction should follow reading convention.
What does that even mean in this context? In a strictly LTR language, sure, you read left-to-right and the glyphs are rendered left-to-right. But the whole discussion is about bidirectional text, where the text is rendered by a complex algorithm. What is the “rendering direction”?
I know just enough about some RTL languages to know that one can absolutely intersperse RTL text with, say, and English phrase, and you still read the first (leftmost in the group) English sound first and so on :)
if you write (with a pen) text with mixed arabic, numbers, english words. then somebody else gets to read it, he will read arabic from right to left, then when when he encouters a number or an english word, he will naturally jump to the first latin letter of the word and start reading left to right, then jump back to the beginning if the next arabic word and switch back to RTL. the alogorithm should copy that behaviour.
actually : four and eighty and nine hundred and a thousand
Numbers below 100 are always read from right to left even in vernacular/informal/dialectical Arabic.
Believe it or not, there once was an 8-bit computer with native Arabic. It was a version of the Z-80 MSX series by Al Alamiah called Sakhr. It seems it could switch between English, Arabic, or mixed. (In fact, this was mentioned in the OP.)
I'm curious how it handled mixed entry.
https://forum.vcfed.org/index.php?threads/bought-a-al-alamia...
https://www.msx.org/wiki/Sakhr_AX-230
Arabic script is a great test to see if your terminal/renderer/UI can handle anything: contexual shaping, cursive connectivity, bidirectional text layout, diacritics and vertical displacement.
I went down this rabbit-hole awhile back and it made me really appreciate the complexity of the script.
> Editable; go on, click in. Place the caret at the start of the line, hold the Left arrow, and enjoy the moment it reaches the version number, where it will jump, double back, or sail through depending on your browser.
what's missing here is "use shift + arrow keys for selection"
Firefox manages movement just fine. Selection tho? oh boi
To me it sounds like it's high time Arabic got a new script that's compatible with computers. I've seen some Lebanese people use mix of Latin letters and digits to type, so the idea isn't entirely stupid.
Computers are tools that exist to serve humans. Humans should never be asked to become subservient to computers. Your suggestion is ludicrous and risible.
Yeah cool then enjoy your text being broken
Enjoy being a slave to a machine, I guess?
The ability to adapt to changing environment is one of core properties of life. You cannot un-invent the printing press. Imagine making an argument "nooooo you cannot print books, it takes away the human element" and then "nooooo you cannot write down stories, it takes away the human element" and then being obliberated by a civilization that embraced new inventions instead of rejecting them based on "no means no".
I could write all day about how badly RtL languages are treated on electronic devices. Most interfaces just slop RtL text into an interface that is otherwise the totally wrong way around.
One thing that amuses me is that people share these "safe zone" templates for short form video to make sure your content isn't hidden behind the buttons:
https://imgur.com/a/MrIQZen
But look over the shoulder of someone using the Arabic version of TikTok and you'll realize how flawed that is:
https://imgur.com/a/5iLXd2o
See also: https://notarabic.com/
very interesting, arabic is a good reminder that text rendering is mostly solved for the scripts that shaped the defaults.
The hard part is that typography, shaping, bidi behavior, font fallback, search, and the editor model all leak into each other.
You cannot fix one layer cleanly when the assumptions are wrong in all of them.
[flagged]
That might be because of translations from Arabic. The article was also posted on a different website where the author responded
> the Kashida section was contributed to this post from a talk in Arabic of Nawal Hadeed, which she translated and added to the post herself. Although I'm unsure of LLM usage in the translation process, looking at the original Arabic I felt some change in tone while editing the post. I could have either declined the translation and never have this documented, procrastinate in translating it myself (which has been ongoing for a while), or publish as it is. I found the last least damaging.
Same, and it is a shame. Reading it is wearisome.
Disclaimer: I’m not fluent in Arabic by any means, but the stretched out to both margins style looks very Quranic to me. I don’t think it looks appropriate for say, a message about my DoorDasher.
But a message about your Door dasher also likely wouldn't be justified.