Mappings for representing English in Cirth (as in LOTR) and Futhorc/Runes (as in the Hobbit)

[davidfraser]

This contains two new mappings following how Tolkien represented English in his major works: One for the Cirth as used in key parts of Lord of the Rings (english-cirth-erebor.glaem) and one for his adaptation of Anglo-Saxon runes / Futhorc as used in the Hobbit (english-futhorc.glaem).

Cirth / LOTR
This mapping closely follows the table at https://ring-lord.tripod.com/cirth/angerthaserebor.htm

It can be used to represent the primary examples of Tolkien writing English in Cirth (see the Mellonath Daeron Index
of Certh Specimina). For each of these, the English text, a phoneticized version that can be inputted into Glaemscribe, and the resulting Cirth characters are given:

• DCS 1/2 - The Lord of the Rings Title-page Cirth:
  • The Lord of the Rings translated from the Red Book
  • The Lord of the Riŋs translat+ʌd from the red book/
  • !/ I ab@9 I b4 I !/ I @lhg I 8@cugac8?9 I 3@b6 I !/ I @z9 I 2ne}
  • 
• DCS 6 - Balin's Tomb-inscription (the English part of this)
  • Balin son of Fundin, Lord of Moria
  • Balin|sʌn|ov|Fundin|Lord|ov|Moria
  • 2calugZub43Sdluab@9b46b@lc
  • 

It currently takes an approach of trying to include all the character combinations presented in the tables, even those that are not useful for English. It uses some IPA symbols where there isn't a standard alphabetic character (notably for the schwa vowel representations) or to differentiate between English sounds (so ŋ when ng is not pronounced followed with a hard g sound, but ng otherwise).

Where there are alternative characters available, these are selected by prefixing them with a +. I'm not sure if there is a better way to do this but it does make it possible to represent everything fairly easily...

Anglo-Saxon Runes / The Hobbit
This mapping closely follows the table at https://ring-lord.tripod.com/runes/dwafrune.htm (itself based on the Hobbit)

It maps to the Unicode Runic range; some of these mappings are a slight compromise as appropriate letters are not available (see the comments in the mapping)

It can be used to represent the primary examples of Tolkien writing English in Futhorc (see the Mellonath Daeron Runes Specimina index, and Some comprehensive analysis of the usage of these runes). For each of these, a mildly phoneticized version of the English text that can be inputted into Glaemscribe, and the resulting Unicode Runic characters are given:

• DRS 3 - Thror's Map Explanation
  • Five Feet high the door and three may walk abreast.
  • ᚠᛁᚢᛖ ᚠᛟᛏ ᚻᛁᚷᚻ ᚦᛖ ᛞᚩᚱ ᚫᚾᛞ ᚦᚱᛟ ᛗᚫᚣ ᚹᚫᛚᛷ ᚫᛒᚱᛠᛲᛏ᛬
• DRS 6 - Thor's Map Moon Runes
  • Stand by the grey stone hwen the thrush knocks, and the setting sun with the last light of Durin's Day will shine upon the keyhole
  • ᛲᛏᚫᚾᛞ ᛒᚣ ᚦᛖ ᚷᚱᛖᚣ ᛲᛏᚩᚾᛖ ᚻᚹᛖᚾ ᚦᛖ ᚦᚱᚢᛲᚻ ᛷᚾᚩᚳᛷᛲ ᚫᚾᛞ ᚦᛖ ᛲᛖᛏᛏᛁᛝ ᛲᚢᚾ ᚹᛁᚦ ᚦᛖ ᛚᚫᛲᛏ ᛚᛁᚷᚻᛏ ᚩᚠ ᛞᚢᚱᛁᚾᛲ ᛞᚫᚣ ᚹᛁᛚᛚ ᛲᚻᛁᚾᛖ ᚢᛈᚩᚾ ᚦᛖ ᛷᛖᚣᚻᚩᛚᛖ᛬ ᚦ᛬

[davidfraser]

Any thoughts or comments on the usefulness of this welcome ... I found it helpful in transcribing things where many of the other tools on the web can't do it accurately but Glǽmscribe can!

[davidfraser]

Comparing the Cirth LOTR mapping to Table 2 (page 4) in The Letters of Middle-Earth, the differences are:

• G is represented by CIRTH_29 (which is the alternate representation encoded by +G), not CIRTH_19
• Z is represented by CIRTH_17, not CIRTH_43. This is reflective of Dan Smith's table of Angerthas Moria, not Angerthas Erebor, but the G-rune as CIRTH_29 would conflict with J if this were Angerthas Moria (also see the values for S and H.

Comparing the Futhorc Hobbit mapping to Table 1 (page 3) in The Letters of Middle-Earth, it maps fairly closely apart from some of the dipthongs EA ST and EO at the end.

[BenTalagan]

Hi David, thanks a lot for your contribution ! Having Cirth modes for English would be a wonderful thing.

Just for info, I've been working for a long while on a way to make English modes possible in glaemscribe, and we (my friend Bertrand Bellet and I, Bertrand helping me a lot on all the theoretical linguistic aspects) have chosen a difficult way, this is why the repository has not evolved during the past few years - I haven't yet committed the work in progress, so things were looking a bit stalled (which was not totally false but not true either).

The engine is now complete, and it relies on our custom version of espeakNG, to produce a non-ambiguous IPA representation of english. The logic behind is phonemic, but the engine can also produce multiple phonetic variations (we have currently 4 variants : traditional RP, modern RP, US, and 'as close to JRRT as possible'). To make yourself an idea of what it could be like, you can have a look at the synthesis chart here (where things are grouped by lexical sets and ambiguous cases). The column mapping being the following : en-tengwar=J.R.R.T.'s pronunciation, en-tengwar-gb=modern RP, en-tengwar-rp=traditional RP, en-tengwar-us=US pronunciation:

https://htmlpreview.github.io/?https://github.com/BenTalagan/espeak-ng/blob/glaemscribe/tests-glaemscribe/build/example_chart.html

You could probably build an awesome mode on the top of the engine, and it's just a pity that it's not yet available. The current problem is documentation, as we've made a lot of options available that need to be explained, see :

The integration in a mode is really easy, as you just need to enable the engine with a flag in the mode's source code. The engine will then be automatically applied to the input text just before the mode processing, so that instead of working on the english input directly, you would work on an already transformed version such as, for example :

ðə lˈɔːɹd ɒvðə ɹˈɪŋz,

ɐ bˈʊk baɪ dʒˈɒn ɹˈɒnəld ɹˈuːəl tolkˈiːn.

θɹˈiː ɹˈɪŋz fɚðɪ ˈɛlvən-kˈɪŋz ˌʌndɚ ðə skˈaɪ,
sˈɛvən fɚðə dwˈɔːɹf-lˈɔːɹdz ɪn ðɛɹ hˈɔːlz ɒv stˈoʊn,
nˈaɪn fɔːɹ mˈɔːɹtəl mˈɛn dˈuːmd tʊ̟ dˈaɪ,
wˈɒn fɚðə dˈɑːɹk lˈɔːɹd ˌɒn hɪz dˈɑːɹk θɹˈoʊn
ɪnðə lˈænd ɒv mˈɔːɹdoːɹ ʍˌɛɹ ðə ʃˈædoʊz lˈaɪ.
wˈɒn ɹˈɪŋ tʊ̟ ɹˈuːl ðˌɛm ˈɔːl, wˈɒn ɹˈɪŋ tʊ̟ fˈaɪnd ðˌɛm ,
wˈɒn ɹˈɪŋ tʊ̟ bɹˈɪŋ ðˌɛm ˈɔːl ænd ɪnðə dˈɑːɹknəs bˈaɪnd ðˌɛm
ɪnðə lˈænd ɒv mˈɔːɹdoːɹ ʍˌɛɹ ðə ʃˈædoʊz lˈaɪ.

We're currently working on the documentation after a very long hiatus, and we're trying to pack everything up for the end of August.

So my first reaction is : what do you feel about the potential use of the english engine for your mode ? Do you see any use for it ? Having a Cirth mode and futhorc mode in addition of the Tengwar mode for the release of the english engine would be totally awesome and extremely timely.

Closing this big parenthesis, another immediate reaction on your remarks and questions would be : do not hesitate to use the option system if you have interrogations on what to use to render an ambiguous case. This is generally my ultimate solution to solve the problem of Tolkien's hesitations : you leave the choice to the end user so everyone's satisfied. You can have very atomic options, or if you're afraid that they'd clash together, you can have more general options that affect multiple behaviors.

[davidfraser]

Wow thanks for the detailed info! I had seen you were working on something with espeak which I think is brilliant...
I can see how this is really essential for the Tengwar modes as they are very phonetically oriented. Tolkien's use of the Anglo-saxon runes in the Hobbit for tends to be much closer to orthographic (although not totally consistently), so would probably not benefit; his use of the Cirth for English varies between phonetic and orthographic, so could definitely benefit. As you say options would probably be really helpful here!

I'd love to give the espeak engine a try; is this working on master?

[BenTalagan]

I can see how this is really essential for the Tengwar modes as they are very phonetically oriented. Tolkien's use of the Anglo-saxon runes in the Hobbit for tends to be much closer to orthographic (although not totally consistently), so would probably not benefit; his use of the Cirth for English varies between phonetic and orthographic, so could definitely benefit. As you say options would probably be really helpful here!

Actually, the same problem lies in the "so-called" XD orthographic modes for tengwar. Tolkien introduces a lot of phonetic tricks inside of them, which make them totally hybrid, and totally unregular. That's a nightmare for a transcriptor like glaemscribe, and is probably more difficult to implement than a phonemic mode. We had a discussion with Bertrand on the matter and decided not to implement any of the orthographic modes described on Amanye Tenceli due to all those exceptions. If we were to implement one, we would probably chose something very regular (which is not really interesting) and maybe, on this basis, add some spices, but in a non-ambiguous way exactly as you did with IPA variants.

The english engine is probably not working on the master (honestly, can't remember) :) So I've just committed my mode/english branch (latest version) so that you can play with it in the editor (you will have to load the editor from glaemscribe's repository, not the online version on the official site, which is not up to date). The "debug" tool should help you see what the engine is doing with english, and you can use the english tengwar mode as an example for the undocumented options/features.

[davidfraser]

OK that's really useful, thank you, will check it out!

[BenTalagan]

His use of the Cirth for English varies between phonetic and orthographic, so could definitely benefit.

Having a look at the Book of Mazarbul's transcription, I could check that an automatic translation from the latin orthography "à la Tolkien" is almost impossible, indeed...

They have taken the bridge
ðeɪ hæv tˈeɪkən ðə bɹˈɪdʒ

Tolkien makes the distinction between e representing /e/ (they) and the schwa /ə/ (have, taken, bridge). However, he does not make any distinction between a representing /æ/ (have) and /e/ (taken). From the IPA we thus cannot deduce the original orthography for /e(ɪ)/.

It makes the mode fall into the category of unregular "orthographic" modes where we need to propose a custom orthography to allow to reflect Tolkien's choices and customisations, as you've done in your proposal.

[BenTalagan]

One remark playing with the Erebor mode : because glaemscribe is an automatic tool/stuff and addressed to a large audience (amateurs up to scholars) we try to make modes "usable" in a generic context, and that means "filling the gaps" when there are some. This is really touchy, because it means adding a bit of invention to Tolkien's work, or at least merging ideas that come from different chronological sources (sometimes clashing with each other) so we try to make this clear in the docs (when they exist 😞 😞), but also, we try to stay as close as possible to Tolkien's choices or to canonical/natural choices (for tengwar e.g., there's often a phonological logic behind a system than you can use to reconstruct what is missing). One nice tool is the option system of glaemscribe, that allows to propose multiple solutions to augment the satisfaction of picky scholars - and leave the responsibility to the end user.

Playing with your Erebor mode, I think that to be closer to what we've done yet, you should probably fill a few gaps, since some common english letters are not represented at all : c, q ,x maybe qu would probably need their own entry and I probably forget some (a quick review of orthographic modes for other writing systems would give us an idea of the missing letter clusters worth of interest for Tolkien in english on an orthographic point of view).

For c, there may be an even better solution. It seems to me that the canonical transcription would be /k/ but it's just an impression from the known transcripts, and the dwarves would probably be puzzled when transcribing this letter in some cases. In the book of Mazarbul, e.g., there are the words watcher and chamber, and ch is handled phonetically by the dwarves meaning they would care here to avoid an unreadable transcription. In fact, the pronunciation of c in english seems a bit predictable (most valuable link here) :

https://en.wikipedia.org/wiki/English_orthography

So we could handle separately, ce, ci, cy for the /s/ sound and also handle _cn, _ct as _c and _t , letting the user correct the exceptions if any (cello, special, coercion, celts, etc) ; that would probably be a better choice than using a pure /k/ transcription for c (but we can still offer an option for that). However playing that game could be dangerous because it can lead you far : what about cc, what about ck, etc, and you can slip progressively towards phonemic considerations. So there's a good balance to find about when to stop, and what to treat and not treat.

Also, there are some cirth that exist in Dan Smith's fonts that I've not included in the ds_cirth charset but could (maybe) :

Maybe they could be used, I.D.K, but the documentation is very sparse, and probably some time should be taken to evaluate each cirth's origin before use.

[BenTalagan]

Just in case, I've just added support for the diacritics of the Cirth Erebor fonts in the cirth_ds charset (in the master branch). Also added a few of the extended chars of the chart above to the cirth_ds, but cannot yet identify their origin.

[BenTalagan]

One more note : changing the outspace directly from the root of the mode with a conditional flow will not work (conditions will only work in the pre-processor, processor, and post-processor).

So I've added an \outspace post-processor operator to be able to perform this.

[davidfraser]

Great, thanks for the updates! Will have a look through them...

[BenTalagan]

One remark : since you've merge the mode/english branch in your own english-cirth-mode branch, I've changed the target branch for the PR to point to the mode/english branch itself, and merged the latest master changes into this mode/english branch so it is up to date and benefits from the latest fixes, but can live separately atm (since I try to not include english stuff in the master atm). :)

[davidfraser]

So far I've implemented options for variant characters for both the Erebor and the Futhorc scripts. Any comments on this welcome...

I have along the way come across a bug that if I produce a sequence CIRTH_52_ALT CIRTH_PUNCT_MID_DOT the postprocessor seems to die... even if it's produced directly from a character, and even with the postprocessor removed ... not sure what's happening there...

[davidfraser]

I've added an approach to the handling of the c, x, q and related cases, as follows:

• There's an option to always use_k_for_c, off by default
• Otherwise, it handles:
  • The standard soft c when followed by a vowel
  • A ks when a double c is followed by a vowel
  • Removes silent c before t at the start of the word (though this is a very rare corner case)
• Allows ç for s as in façade
• There's an option to detect soft g and use j
• qu -> kw but q without u -> k
• x -> ks

I initially included a ck -> k mapping but then decided against it. It actually works nicely having ck map naturally to kk especially if you turn on double consonants.

I've implemented this all in the preprocessor, is that OK?

Any comments welcome

[davidfraser]

I've implemented the numeric sub-dots, the underbar for double vowels and the circumflex for double consonants.

I couldn't get virtual characters to render in the Glaemscribe editor; this seems to be true in the current live online editor as well:

As a result, I've left all the diacritics non-virtual for now as it makes it easier for me to test (I can't run the full website locally) but this should obviously be changed

[davidfraser]

Notes:

• I still have a few alternatives selectable by a + prefix... The short forms of the schwa characters and h seem to be selected manually (including by Tolkien within the same text) rather than being a stylistic variant. How do you think this should best be handled?
• I presume there's a more efficient way to add the diacritics to a whole set? (Obviously this needs virtual ones)
• Not sure on how to name the selection of alternate Cirth when they are Tolkien's own inventions and don't have names, only numbers. (For variant drawings of the same number, I've just called it alt).

[BenTalagan]

I couldn't get virtual characters to render in the Glaemscribe editor; this seems to be true in the current live online editor as well

For the virtual characters to be handled you need not to forget to add a call to the \resolve_virtuals operator in the post-processor (or you'll get the nuclear icon as you did). Should be probably stated more clearly in the doc as it is a common mistake when starting a new mode (even for me) ! :)

[BenTalagan]

I have along the way come across a bug that if I produce a sequence CIRTH_52_ALT CIRTH_PUNCT_MID_DOT the postprocessor seems to die... even if it's produced directly from a character, and even with the postprocessor removed ... not sure what's happening there...

Possible problem is that you tried to use CIRTH_PUNCT_MID_DOT in multiple rule groups (punctuation and litteral) and that is not allowed. Maybe the error is not yielded correctly in the library ? I'd be interested to have a test case.

Edit: wrong guess, see below.

[davidfraser]

Possible problem is that you tried to use CIRTH_PUNCT_MID_DOT in multiple rule groups (punctuation and litteral) and that is not allowed. Maybe the error is not yielded correctly in the library ? I'd be interested to have a test case.

On the standard site's IDE if I just update the main example with the following, it happens...

\beg processor
	\beg rules litteral
		ai --> CIRTH_52_ALT 
	\end
  
  \beg rules punctuation
		! --> CIRTH_PUNCT_MID_DOT
	\end
\end

Also if I say ai --> CIRTH_52_ALT CIRTH_PUNT_MID_DOT...

[davidfraser]

Do you have a place for putting sample texts? Or the docs that appear in your main site when a mode is selected?

[BenTalagan]

Possible problem is that you tried to use CIRTH_PUNCT_MID_DOT in multiple rule groups (punctuation and litteral) and that is not allowed. Maybe the error is not yielded correctly in the library ? I'd be interested to have a test case.

Found the problem ! The fix is in the mode/english branch (you can merge). The output of glaemscribe was not html escaped, and these old non unicode fonts could produce some parasiting html characters like < , & , etc.

I realized the popups in the debugger were not working anymore. Fixed too.

[davidfraser]

Fantastic! Will wait for you to push tomorrow :)

[BenTalagan]

I've added an approach to the handling of the c, x, q and related cases, as follows:

• There's an option to always use_k_for_c, off by default

• Otherwise, it handles:

  • The standard soft c when followed by a vowel
  • A ks when a double c is followed by a vowel
  • Removes silent c before t at the start of the word (though this is a very rare corner case)

• Allows ç for s as in façade

• There's an option to detect soft g and use j

• qu -> kw but q without u -> k

• x -> ks

Seems to me completely ok since the automatic behavior is not enabled by default : after thought, those orthographic modes need quite some attention and niggling and using orthographic variants cannot be avoided anyway. So, having an automatic behavior might be "too much" by default. This being said, as long as there's a way to overload the automatic behavior (exemple : write kelt for celt to correct the failure of the automatic transcription), I feel it's ok. One can always use k for /k/, ch for /tʃ/, _k for /k/ (ex: cthulhu).

So the base principle could be : disable all automatic behaviors by default.

Regarding the idea of ç for c pronounced s, being french I really like the idea, but in the meantime the IPA letter ç represents a totally different sound (https://en.wikipedia.org/wiki/Voiceless_palatal_fricative) ; so, already having IPA symbols in the mode, I have some hesitations about this, but... you're right it can be found in english in exotic words so ...

I initially included a ck -> k mapping but then decided against it. It actually works nicely having ck map naturally to kk especially if you turn on double consonants.

I do agree !

I've implemented this all in the preprocessor, is that OK?

Probably the easiest way to do those kind of disambiguation, indeed.

• I still have a few alternatives selectable by a + prefix... The short forms of the schwa characters and h seem to be selected manually (including by Tolkien within the same text) rather than being a stylistic variant. How do you think this should best be handled?

That's a good question. If multiple choices could be made by Tolkien within a same text, then we should probably stay simple and let everything be done manually by the user, all the more that marking the schwas is a deliberate and advanced move, so ...

• I presume there's a more efficient way to add the diacritics to a whole set? (Obviously this needs virtual ones)

Yes, there are multiple ways to do this, one is to use macros when a lot of options may be mixed together (undocumented), another is to make use of sheaves, e.g. one way of refactoring the handling of consonants that can be doubled with a few lines :

    \if use_g_29
      {CIRTH_G}  === CIRTH_29
    \else
      {CIRTH_G}  === CIRTH_19 
    \endif
    
    \if use_n_53
      {CIRTH_N}  === CIRTH_53
    \else
      {CIRTH_N}  === CIRTH_22
    \endif
    
    {SIMGLE_CONSONANTS} === [p  * b  * f  * v  * m  * t  * d  * r  * j  * k  * g  * n  * l  * s  * z  * w]
    {DOUBLE_CONSONANTS} === [pp * bb * ff * vv * mm * tt * dt * rr * jj * kk * gg * nn * ll * ss * zz * ww]
    {_MATCHING_CIRTH_}  === [CIRTH_1 * CIRTH_2 * CIRTH_3 * CIRTH_4 * CIRTH_6 * CIRTH_8 * CIRTH_9 * CIRTH_12 * CIRTH_14 * CIRTH_18 * {CIRTH_G} * {CIRTH_N} * CIRTH_31 * CIRTH_35 * CIRTH_43 * CIRTH_44]
    
    {SIMGLE_CONSONANTS} --> {_MATCHING_CIRTH_}
    \if use_circumflex_for_double_consonants
      {DOUBLE_CONSONANTS} --> {_MATCHING_CIRTH_} TEHTA_CIRCUM
    \endif

For tengwar modes I usually split those into témar (series) to make the mode file feel like the classical charts. It makes lines shorter and more readable.

I usually like to make some variable definitions at the start of the rule group when options leading to base characters are involved (like the variables {CIRTH_G} and {CIRTH_N} once for all).

• Not sure on how to name the selection of alternate Cirth when they are Tolkien's own inventions and don't have names, only numbers. (For variant drawings of the same number, I've just called it alt).

Seems to me that "alt" is perfect since we cannot do better. It's really painful to work with Cirth since we cannot name them.

[BenTalagan]

Do you have a place for putting sample texts? Or the docs that appear in your main site when a mode is selected?

Yes but it's in a private repository since the source code of the website itself is not public, so one way to do this would be for you to prepare some explanations in a doc file, that I would layout and style (when there are charts and quotes, a bit of html/css can't be really avoided). That's how we worked with Da Def for the Lang Belta mode. If you feel like it ?

[BenTalagan]

Fantastic! Will wait for you to push tomorrow :)

Ahh really sorry. I thought I had already pushed the fix but seeing my console history I jumped to something else without reaching the end of the flow ! Now you can merge the BenTalagan:mode/english branch to retrieve it.

[davidfraser]

Thanks for all the detailed feedback and help, will make some changes!

[BenTalagan]

JFYI : since we've talked about it at the beginning of the conversation and your intervention gave me the mood to work on glaemscribe 😄 , I've just implemented an additional Angerthas Daeron mode for english using the new text-to-speach engine based on espeakNG, in the mode/english branch. It has quite a large number of options ATM.

For the numeral system (to make things fully usable, which is not possible with the few runes given by Tolkien) I had to devise my own system based on the Cirth and took inspiration from the pentimal system.

• The base is the vertical bar Cirth which represents the number five.
• The zero is the vertical bar Cirth lightly stroked
• I use 4 left-oriented Cirth for numbers 1 to 4, the additional stroke count gives the number
• I use 4 right-oriented Cirth for numbers 6 to 9, the additional stroke count gives the (number - 5)

[BenTalagan]

A bit of teasing, heh ...

[davidfraser]

That's a fun number system! And that rendering looks great!
I've been slightly distracted by making a visual layout of the Cirth to help when first learning to read; it can otherwise be a bit tricky looking through the characters to find them, especially for my young son :)

[BenTalagan]

Nice chart ! I appreciate the effort to layout it symmetrically and the style reminds me of the Mendeliev table. Covered with runes, it feels like some kind of alchemic cryptic parchment :) Take your time for making the PR evolve, glaemscribe is a long running project ! ;)