Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UTF-8 census in POPC2 digital scores #262

Open
craigsapp opened this issue Feb 8, 2024 · 2 comments
Open

UTF-8 census in POPC2 digital scores #262

craigsapp opened this issue Feb 8, 2024 · 2 comments
Assignees
Labels
Data Issue related to digital scores enhancement New feature or request

Comments

@craigsapp
Copy link
Owner

craigsapp commented Feb 8, 2024

Related to #261

Here is a PERL script using uniname command-line program to list the number of each non-ASCII character in all POPC2 files (both lyrics and elsewhere):

#!/usr/bin/env perl

use strict;

my $uniname = `which uniname`;
chomp $uniname;

die "Cannot find uniname program.  Install it first\n" if $uniname =~ /^\s*$/;

# To install uniname on MacOS:
#    brew install uniutils
# Probably similar on linux:
#    dnf install uniutils
#    rpm install uniutils
# or
#    yum install uniutils

# uniname -a :: do not process ASCII characters (UTF-7)
# uniname -b :: do not list byte offsets in file.
# uniname -c :: do not list character offsets in file.

my @files = sort glob("*/kern/*.krn");

my %output;
foreach my $file (@files) {
	#print STDERR "Processing $file\n";
	my $results .= `LINES=24 $uniname -abc "$file"`;
	my @lines = split(/\n/, $results);
	foreach my $line (@lines) {
		$output{$line}++;
	}
}

my @keys = sort { $output{$b} <=> $output{$a} } keys %output;

print "count		UTF-32   encoded as     glyph   name\n";
foreach my $key (@keys) {
    print "$output{$key}\t\t$key\n";
}

Installed in https://github.com/pl-wnifc/humdrum-polish-scores/blob/main/bin/listUnicodeCharacters
(could be copied into the gitea repository as well)

Run with the command make unicode

The results:

count		UTF-32   encoded as     glyph   name
79082		00017F   C5 BF          ſ      LATIN SMALL LETTER LONG S
63459		000142   C5 82          ł      LATIN SMALL LETTER L WITH STROKE
43277		000119   C4 99          ę      LATIN SMALL LETTER E WITH OGONEK
32397		0000F3   C3 B3          ó      LATIN SMALL LETTER O WITH ACUTE
29198		000105   C4 85          ą      LATIN SMALL LETTER A WITH OGONEK
25589		00015B   C5 9B          ś      LATIN SMALL LETTER S WITH ACUTE
13498		00017C   C5 BC          ż      LATIN SMALL LETTER Z WITH DOT ABOVE
11373		000107   C4 87          ć      LATIN SMALL LETTER C WITH ACUTE
9711		00017A   C5 BA          ź      LATIN SMALL LETTER Z WITH ACUTE
9296		000144   C5 84          ń      LATIN SMALL LETTER N WITH ACUTE
8206		0000A0   C2 A0                  NO-BREAK SPACE
3373		0000E1   C3 A1          á      LATIN SMALL LETTER A WITH ACUTE
2879		0000E9   C3 A9          é      LATIN SMALL LETTER E WITH ACUTE
1919		0000FF   C3 BF          ÿ      LATIN SMALL LETTER Y WITH DIAERESIS
1823		0000F9   C3 B9          ù      LATIN SMALL LETTER U WITH GRAVE
1732		0000E0   C3 A0          à      LATIN SMALL LETTER A WITH GRAVE
1701		000292   CA 92          ʒ      LATIN SMALL LETTER EZH
1640		0000FA   C3 BA          ú      LATIN SMALL LETTER U WITH ACUTE
1625		00266F   E2 99 AF       ♯      MUSIC SHARP SIGN
1453		0000E8   C3 A8          è      LATIN SMALL LETTER E WITH GRAVE
1424		000283   CA 83          ʃ      LATIN SMALL LETTER ESH
1343		002013   E2 80 93       –      EN DASH
1337		0004E1   D3 A1          ӡ      CYRILLIC SMALL LETTER ABKHASIAN DZE
1224		00266D   E2 99 AD       ♭      MUSIC FLAT SIGN
1137		00015A   C5 9A          Ś      LATIN CAPITAL LETTER S WITH ACUTE
911		002019   E2 80 99       ’      RIGHT SINGLE QUOTATION MARK
840		0000FC   C3 BC          ü      LATIN SMALL LETTER U WITH DIAERESIS
733		00017B   C5 BB          Ż      LATIN CAPITAL LETTER Z WITH DOT ABOVE
667		000165   C5 A5          ť      LATIN SMALL LETTER T WITH CARON
644		0000B0   C2 B0          °      DEGREE SIGN
634		000161   C5 A1          š      LATIN SMALL LETTER S WITH CARON
622		0000E4   C3 A4          ä      LATIN SMALL LETTER A WITH DIAERESIS
553		0000F2   C3 B2          ò      LATIN SMALL LETTER O WITH GRAVE
531		00266E   E2 99 AE       ♮      MUSIC NATURAL SIGN
499		001E8F   E1 BA 8F       ẏ      LATIN SMALL LETTER Y WITH DOT ABOVE
497		0000F6   C3 B6          ö      LATIN SMALL LETTER O WITH DIAERESIS
395		00016D   C5 AD          ŭ      LATIN SMALL LETTER U WITH BREVE
341		000141   C5 81          Ł      LATIN CAPITAL LETTER L WITH STROKE
293		0000E6   C3 A6          æ      LATIN SMALL LETTER AE
291		0000E2   C3 A2          â      LATIN SMALL LETTER A WITH CIRCUMFLEX
291		001D23   E1 B4 A3       ᴣ      LATIN LETTER SMALL CAPITAL EZH
269		0000EA   C3 AA          ê      LATIN SMALL LETTER E WITH CIRCUMFLEX
257		0000DF   C3 9F          ß      LATIN SMALL LETTER SHARP S
247		000307   CC 87          ̇      COMBINING DOT ABOVE
242		0000F4   C3 B4          ô      LATIN SMALL LETTER O WITH CIRCUMFLEX
242		0000BA   C2 BA          º      MASCULINE ORDINAL INDICATOR
235		00201E   E2 80 9E       „      DOUBLE LOW-9 QUOTATION MARK
214		00201D   E2 80 9D       ”      RIGHT DOUBLE QUOTATION MARK
160		0000F5   C3 B5          õ      LATIN SMALL LETTER O WITH TILDE
157		0000E3   C3 A3          ã      LATIN SMALL LETTER A WITH TILDE
153		0004F1   D3 B1          ӱ      CYRILLIC SMALL LETTER U WITH DIAERESIS
140		0000EC   C3 AC          ì      LATIN SMALL LETTER I WITH GRAVE
127		0001D4   C7 94          ǔ      LATIN SMALL LETTER U WITH CARON
127		0000F7   C3 B7          ÷      DIVISION SIGN
120		000301   CC 81          ́      COMBINING ACUTE ACCENT
118		00014F   C5 8F          ŏ      LATIN SMALL LETTER O WITH BREVE
111		000188   C6 88          ƈ      LATIN SMALL LETTER C WITH HOOK
108		00010D   C4 8D          č      LATIN SMALL LETTER C WITH CARON
102		001EBD   E1 BA BD       ẽ      LATIN SMALL LETTER E WITH TILDE
94		000115   C4 95          ĕ      LATIN SMALL LETTER E WITH BREVE
90		0000A7   C2 A7          §      SECTION SIGN
75		000169   C5 A9          ũ      LATIN SMALL LETTER U WITH TILDE
75		0000D7   C3 97          ×      MULTIPLICATION SIGN
75		000233   C8 B3          ȳ      LATIN SMALL LETTER Y WITH MACRON
73		0000D3   C3 93          Ó      LATIN CAPITAL LETTER O WITH ACUTE
68		0001B7   C6 B7          Ʒ      LATIN CAPITAL LETTER EZH
67		00017E   C5 BE          ž      LATIN SMALL LETTER Z WITH CARON
54		0000FB   C3 BB          û      LATIN SMALL LETTER U WITH CIRCUMFLEX
52		0004E0   D3 A0          Ӡ      CYRILLIC CAPITAL LETTER ABKHASIAN DZE
52		0000EB   C3 AB          ë      LATIN SMALL LETTER E WITH DIAERESIS
50		0000AD   C2 AD          ­      SOFT HYPHEN
44		0000D4   C3 94          Ô      LATIN CAPITAL LETTER O WITH CIRCUMFLEX
43		000153   C5 93          œ      LATIN SMALL LIGATURE OE
41		002020   E2 80 A0       †      DAGGER
40		00044A   D1 8A          ъ      CYRILLIC SMALL LETTER HARD SIGN
34		0000FD   C3 BD          ý      LATIN SMALL LETTER Y WITH ACUTE
31		000442   D1 82          т      CYRILLIC SMALL LETTER TE
31		0000A9   C2 A9          ©      COPYRIGHT SIGN
30		000103   C4 83          ă      LATIN SMALL LETTER A WITH BREVE
29		000106   C4 86          Ć      LATIN CAPITAL LETTER C WITH ACUTE
28		0000ED   C3 AD          í      LATIN SMALL LETTER I WITH ACUTE
25		00016B   C5 AB          ū      LATIN SMALL LETTER U WITH MACRON
25		0000C9   C3 89          É      LATIN CAPITAL LETTER E WITH ACUTE
23		002026   E2 80 A6       …      HORIZONTAL ELLIPSIS
23		000187   C6 87          Ƈ      LATIN CAPITAL LETTER C WITH HOOK
23		000117   C4 97          ė      LATIN SMALL LETTER E WITH DOT ABOVE
23		002014   E2 80 94       —      EM DASH
23		00044B   D1 8B          ы      CYRILLIC SMALL LETTER YERU
22		00044E   D1 8E          ю      CYRILLIC SMALL LETTER YU
22		001E83   E1 BA 83       ẃ      LATIN SMALL LETTER W WITH ACUTE
22		000178   C5 B8          Ÿ      LATIN CAPITAL LETTER Y WITH DIAERESIS
22		0000EE   C3 AE          î      LATIN SMALL LETTER I WITH CIRCUMFLEX
21		000101   C4 81          ā      LATIN SMALL LETTER A WITH MACRON
20		0025E0   E2 97 A0       ◠      UPPER HALF CIRCLE
20		000293   CA 93          ʓ      LATIN SMALL LETTER EZH WITH CURL
20		0000D8   C3 98          Ø      LATIN CAPITAL LETTER O WITH STROKE
20		0001EF   C7 AF          ǯ      LATIN SMALL LETTER EZH WITH CARON
20		002217   E2 88 97       ∗      ASTERISK OPERATOR
18		000440   D1 80          р      CYRILLIC SMALL LETTER ER
18		0001B5   C6 B5          Ƶ      LATIN CAPITAL LETTER Z WITH STROKE
17		0000BD   C2 BD          ½      VULGAR FRACTION ONE HALF
17		000151   C5 91          ő      LATIN SMALL LETTER O WITH DOUBLE ACUTE
17		0001D0   C7 90          ǐ      LATIN SMALL LETTER I WITH CARON
16		00201C   E2 80 9C       “      LEFT DOUBLE QUOTATION MARK
16		00A729   EA 9C A9       ꜩ      LATIN SMALL LETTER TZ
15		000431   D0 B1          б      CYRILLIC SMALL LETTER BE
15		01D10C   F0 9D 84 8C    𝄌      MUSICAL SYMBOL CODA
15		0000F1   C3 B1          ñ      LATIN SMALL LETTER N WITH TILDE
15		00043C   D0 BC          м      CYRILLIC SMALL LETTER EM
15		001E79   E1 B9 B9       ṹ      LATIN SMALL LETTER U WITH TILDE AND ACUTE
14		000113   C4 93          ē      LATIN SMALL LETTER E WITH MACRON
14		002193   E2 86 93       ↓      DOWNWARDS ARROW
13		00043B   D0 BB          л      CYRILLIC SMALL LETTER EL
13		0027E9   E2 9F A9       ⟩      MATHEMATICAL RIGHT ANGLE BRACKET
13		0000E7   C3 A7          ç      LATIN SMALL LETTER C WITH CEDILLA
13		000179   C5 B9          Ź      LATIN CAPITAL LETTER Z WITH ACUTE
13		0027E8   E2 9F A8       ⟨      MATHEMATICAL LEFT ANGLE BRACKET
12		000438   D0 B8          и      CYRILLIC SMALL LETTER I
12		000411   D0 91          Б      CYRILLIC CAPITAL LETTER BE
12		000227   C8 A7          ȧ      LATIN SMALL LETTER A WITH DOT ABOVE
12		0025E1   E2 97 A1       ◡      LOWER HALF CIRCLE
12		000304   CC 84          ̄      COMBINING MACRON
12		00014D   C5 8D          ō      LATIN SMALL LETTER O WITH MACRON
11		0001B6   C6 B6          ƶ      LATIN SMALL LETTER Z WITH STROKE
11		00043D   D0 BD          н      CYRILLIC SMALL LETTER EN
11		0000EF   C3 AF          ï      LATIN SMALL LETTER I WITH DIAERESIS
10		0000C4   C3 84          Ä      LATIN CAPITAL LETTER A WITH DIAERESIS
9		001D52   E1 B5 92       ᵒ      MODIFIER LETTER SMALL O
9		0000F8   C3 B8          ø      LATIN SMALL LETTER O WITH STROKE
8		000434   D0 B4          д      CYRILLIC SMALL LETTER DE
8		000143   C5 83          Ń      LATIN CAPITAL LETTER N WITH ACUTE
8		0001CE   C7 8E          ǎ      LATIN SMALL LETTER A WITH CARON
8		0000C8   C3 88          È      LATIN CAPITAL LETTER E WITH GRAVE
7		001EE7   E1 BB A7       ủ      LATIN SMALL LETTER U WITH HOOK ABOVE
7		00010C   C4 8C          Č      LATIN CAPITAL LETTER C WITH CARON
7		000104   C4 84          Ą      LATIN CAPITAL LETTER A WITH OGONEK
7		002028   E2 80 A8       
      LINE SEPARATOR
7		01D110   F0 9D 84 90    𝄐      MUSICAL SYMBOL FERMATA
6		0000B7   C2 B7          ·      MIDDLE DOT
6		000424   D0 A4          Ф      CYRILLIC CAPITAL LETTER EF
6		0000C0   C3 80          À      LATIN CAPITAL LETTER A WITH GRAVE
6		0000D2   C3 92          Ò      LATIN CAPITAL LETTER O WITH GRAVE
6		001D49   E1 B5 89       ᵉ      MODIFIER LETTER SMALL E
6		000118   C4 98          Ę      LATIN CAPITAL LETTER E WITH OGONEK
6		000432   D0 B2          в      CYRILLIC SMALL LETTER VE
5		002E13   E2 B8 93       ⸓      DOTTED OBELOS
5		00020F   C8 8F          ȏ      LATIN SMALL LETTER O WITH INVERTED BREVE
5		00043F   D0 BF          п      CYRILLIC SMALL LETTER PE
5		0000DC   C3 9C          Ü      LATIN CAPITAL LETTER U WITH DIAERESIS
5		0025B3   E2 96 B3       △      WHITE UP-POINTING TRIANGLE
5		00263C   E2 98 BC       ☼      WHITE SUN WITH RAYS
5		00011B   C4 9B          ě      LATIN SMALL LETTER E WITH CARON
5		000437   D0 B7          з      CYRILLIC SMALL LETTER ZE
5		00E500   EE 94 80             Unknown character in range Private Use Area - Conlang Saklor
5		000207   C8 87          ȇ      LATIN SMALL LETTER E WITH INVERTED BREVE
4		002192   E2 86 92       →      RIGHTWARDS ARROW
4		00043A   D0 BA          к      CYRILLIC SMALL LETTER KA
4		0000F0   C3 B0          ð      LATIN SMALL LETTER ETH
4		000267   C9 A7          ɧ      LATIN SMALL LETTER HENG WITH HOOK
4		00AB33   EA AC B3       ꬳ      Character in undefined range
4		00016F   C5 AF          ů      LATIN SMALL LETTER U WITH RING ABOVE
4		000160   C5 A0          Š      LATIN CAPITAL LETTER S WITH CARON
4		00FEFF   EF BB BF               ZERO WIDTH NO-BREAK SPACE
4		0004E7   D3 A7          ӧ      CYRILLIC SMALL LETTER O WITH DIAERESIS
3		001D50   E1 B5 90       ᵐ      MODIFIER LETTER SMALL M
3		00044F   D1 8F          я      CYRILLIC SMALL LETTER YA
3		000421   D0 A1          С      CYRILLIC CAPITAL LETTER ES
3		0003A6   CE A6          Φ      GREEK CAPITAL LETTER PHI
3		000463   D1 A3          ѣ      CYRILLIC SMALL LETTER YAT
3		001EF9   E1 BB B9       ỹ      LATIN SMALL LETTER Y WITH TILDE
3		01D10E   F0 9D 84 8E    𝄎      MUSICAL SYMBOL REPEATED FIGURE-2
3		002600   E2 98 80       ☀      BLACK SUN WITH RAYS
3		0000DA   C3 9A          Ú      LATIN CAPITAL LETTER U WITH ACUTE
3		0000D9   C3 99          Ù      LATIN CAPITAL LETTER U WITH GRAVE
3		0003B8   CE B8          θ      GREEK SMALL LETTER THETA
3		0000AC   C2 AC          ¬      NOT SIGN
3		000203   C8 83          ȃ      LATIN SMALL LETTER A WITH INVERTED BREVE
3		000190   C6 90          Ɛ      LATIN CAPITAL LETTER OPEN E
3		000352   CD 92          ͒      COMBINING FERMATA
3		00ECA5   EE B2 A5             Unknown character in range Private Use Area
3		0003B1   CE B1          α      GREEK SMALL LETTER ALPHA
2		0000DB   C3 9B          Û      LATIN CAPITAL LETTER U WITH CIRCUMFLEX
2		001E3F   E1 B8 BF       ḿ      LATIN SMALL LETTER M WITH ACUTE
2		01D10F   F0 9D 84 8F    𝄏      MUSICAL SYMBOL REPEATED FIGURE-3
2		000441   D1 81          с      CYRILLIC SMALL LETTER ES
2		00E262   EE 89 A2             Unknown character in range Private Use Area - Conlang Xaini
2		002205   E2 88 85       ∅      EMPTY SET
2		000159   C5 99          ř      LATIN SMALL LETTER R WITH CARON
2		000130   C4 B0          İ      LATIN CAPITAL LETTER I WITH DOT ABOVE
2		00023B   C8 BB          Ȼ      LATIN CAPITAL LETTER C WITH STROKE
2		00AB4D   EA AD 8D       ꭍ      Character in undefined range
2		0001F0   C7 B0          ǰ      LATIN SMALL LETTER J WITH CARON
2		000443   D1 83          у      CYRILLIC SMALL LETTER U
2		0003BD   CE BD          ν      GREEK SMALL LETTER NU
2		000433   D0 B3          г      CYRILLIC SMALL LETTER GHE
2		01D111   F0 9D 84 91    𝄑      MUSICAL SYMBOL FERMATA BELOW
2		00035C   CD 9C          ͜      COMBINING DOUBLE BREVE BELOW
2		001F77   E1 BD B7       ί      GREEK SMALL LETTER IOTA WITH OXIA
2		000114   C4 94          Ĕ      LATIN CAPITAL LETTER E WITH BREVE
2		0000FE   C3 BE          þ      LATIN SMALL LETTER THORN
2		002349   E2 8D 89       ⍉      APL FUNCTIONAL SYMBOL CIRCLE BACKSLASH
2		0016ED   E1 9B AD       ᛭      RUNIC CROSS PUNCTUATION
1		001E0E   E1 B8 8E       Ḏ      LATIN CAPITAL LETTER D WITH LINE BELOW
1		01D1AF   F0 9D 86 AF    𝆯      MUSICAL SYMBOL PEDAL UP MARK
1		001EF3   E1 BB B3       ỳ      LATIN SMALL LETTER Y WITH GRAVE
1		0003BF   CE BF          ο      GREEK SMALL LETTER OMICRON
1		00201F   E2 80 9F       ‟      DOUBLE HIGH-REVERSED-9 QUOTATION MARK
1		001E9E   E1 BA 9E       ẞ      LATIN CAPITAL LETTER SHARP S
1		00012D   C4 AD          ĭ      LATIN SMALL LETTER I WITH BREVE
1		000414   D0 94          Д      CYRILLIC CAPITAL LETTER DE
1		0000D6   C3 96          Ö      LATIN CAPITAL LETTER O WITH DIAERESIS
1		000305   CC 85          ̅      COMBINING OVERLINE
1		00019F   C6 9F          Ɵ      LATIN CAPITAL LETTER O WITH MIDDLE TILDE
1		0003C0   CF 80          π      GREEK SMALL LETTER PI
1		00045D   D1 9D          ѝ      CYRILLIC SMALL LETTER I WITH GRAVE
1		0000CD   C3 8D          Í      LATIN CAPITAL LETTER I WITH ACUTE
1		002227   E2 88 A7       ∧      LOGICAL AND
1		000308   CC 88          ̈      COMBINING DIAERESIS
1		001E55   E1 B9 95       ṕ      LATIN SMALL LETTER P WITH ACUTE
1		001D57   E1 B5 97       ᵗ      MODIFIER LETTER SMALL T
1		001E45   E1 B9 85       ṅ      LATIN SMALL LETTER N WITH DOT ABOVE
1		002294   E2 8A 94       ⊔      SQUARE CUP
1		0000CA   C3 8A          Ê      LATIN CAPITAL LETTER E WITH CIRCUMFLEX
1		00221E   E2 88 9E       ∞      INFINITY
1		000173   C5 B3          ų      LATIN SMALL LETTER U WITH OGONEK
1		002191   E2 86 91       ↑      UPWARDS ARROW
1		0025D8   E2 97 98       ◘      INVERSE BULLET
1		0002BC   CA BC          ʼ      MODIFIER LETTER APOSTROPHE
1		01D11E   F0 9D 84 9E    𝄞      MUSICAL SYMBOL G CLEF
1		00E047   EE 81 87             Unknown character in range Private Use Area - Conlang Tengwar
1		002215   E2 88 95       ∕      DIVISION SLASH
1		00017D   C5 BD          Ž      LATIN CAPITAL LETTER Z WITH CARON
1		00022F   C8 AF          ȯ      LATIN SMALL LETTER O WITH DOT ABOVE
1		0000CC   C3 8C          Ì      LATIN CAPITAL LETTER I WITH GRAVE
1		00010B   C4 8B          ċ      LATIN SMALL LETTER C WITH DOT ABOVE
1		00ECA3   EE B2 A3             Unknown character in range Private Use Area
1		000436   D0 B6          ж      CYRILLIC SMALL LETTER ZHE
1		000148   C5 88          ň      LATIN SMALL LETTER N WITH CARON
1		001D98   E1 B6 98       ᶘ      LATIN SMALL LETTER ESH WITH RETROFLEX HOOK
1		0002C5   CB 85          ˅      MODIFIER LETTER DOWN ARROWHEAD
1		000109   C4 89          ĉ      LATIN SMALL LETTER C WITH CIRCUMFLEX
1		0000C2   C3 82          Â      LATIN CAPITAL LETTER A WITH CIRCUMFLEX

The "Unknown character in range Private Use Area" and "character in undefined range" characters should be removed from the files.

craigsapp added a commit to pl-wnifc/humdrum-polish-scores that referenced this issue Feb 8, 2024
@craigsapp
Copy link
Owner Author

craigsapp commented Feb 8, 2024

Searching for the last problem character:

grep -rl "" *-*

Results in identifying this file:

pl-wumfc/kern/pl-wumfc--n2100--001-012_dobrzynski-ignacy-feliks--monbar-czyli-flibustierowie-partytura-fortepianowa-akt-ii-11-coro.krn

Editing the file(s):

vi $(grep -rl "" *-*)

Problem character in score, searching for character :

Screenshot 2024-02-07 at 21 27 35

Looking at online score:

https://polishscores.org/?id=pl-wumfc--n2100--001-012

Screenshot 2024-02-07 at 21 30 26

Loading into VHV:

https://verovio.humdrum.org/?file=https%3A%2F%2Fhumdrum.nifc.pl%2Fpl-wumfc--n2100--001-012.krn

Screenshot 2024-02-07 at 21 33 10

Original score:

https://polish.musicsources.pl/en/lokalizacje/galeria/druki-muzyczne/11863/99#nomenu

Screenshot 2024-02-07 at 21 35 05

This particular character should be encoded as [segno] which will be replaced with the segno sign.

Screenshot 2024-02-07 at 21 37 39

Although text-encoded segnos should be made smaller:

Screenshot 2024-02-07 at 21 38 44

Perhaps the * in this case should have a similar text code invented, such as [starsegno].

(NB: updates should be made to gitea and not this repository, which is downstream of gitea).

@craigsapp craigsapp added enhancement New feature or request Data Issue related to digital scores labels Feb 8, 2024
@craigsapp
Copy link
Owner Author

craigsapp commented Feb 8, 2024

Here are unicode characters in **text spines. The listUnicodeCharacters script is updated to add the -t option to only look at lyrics (**text). Run in the base directory of humdrum-polish-scores:

make unicode-text

Results:

count		UTF-32   encoded as     glyph   name
68543		00017F   C5 BF          ſ      LATIN SMALL LETTER LONG S
20829		000142   C5 82          ł      LATIN SMALL LETTER L WITH STROKE
14866		000119   C4 99          ę      LATIN SMALL LETTER E WITH OGONEK
9956		00015B   C5 9B          ś      LATIN SMALL LETTER S WITH ACUTE
9939		000105   C4 85          ą      LATIN SMALL LETTER A WITH OGONEK
7747		00017C   C5 BC          ż      LATIN SMALL LETTER Z WITH DOT ABOVE
7598		0000A0   C2 A0                  NO-BREAK SPACE
7066		0000F3   C3 B3          ó      LATIN SMALL LETTER O WITH ACUTE
6077		000107   C4 87          ć      LATIN SMALL LETTER C WITH ACUTE
3313		0000E1   C3 A1          á      LATIN SMALL LETTER A WITH ACUTE
2987		000144   C5 84          ń      LATIN SMALL LETTER N WITH ACUTE
2579		0000E9   C3 A9          é      LATIN SMALL LETTER E WITH ACUTE
1796		0000FF   C3 BF          ÿ      LATIN SMALL LETTER Y WITH DIAERESIS
1575		000292   CA 92          ʒ      LATIN SMALL LETTER EZH
1371		000283   CA 83          ʃ      LATIN SMALL LETTER ESH
1337		0004E1   D3 A1          ӡ      CYRILLIC SMALL LETTER ABKHASIAN DZE
1262		0000FA   C3 BA          ú      LATIN SMALL LETTER U WITH ACUTE
1193		00017A   C5 BA          ź      LATIN SMALL LETTER Z WITH ACUTE
1143		0000E0   C3 A0          à      LATIN SMALL LETTER A WITH GRAVE
1126		0000E8   C3 A8          è      LATIN SMALL LETTER E WITH GRAVE
678		0000FC   C3 BC          ü      LATIN SMALL LETTER U WITH DIAERESIS
666		000165   C5 A5          ť      LATIN SMALL LETTER T WITH CARON
635		002019   E2 80 99       ’      RIGHT SINGLE QUOTATION MARK
590		000161   C5 A1          š      LATIN SMALL LETTER S WITH CARON
497		001E8F   E1 BA 8F       ẏ      LATIN SMALL LETTER Y WITH DOT ABOVE
484		0000F2   C3 B2          ò      LATIN SMALL LETTER O WITH GRAVE
410		0000F9   C3 B9          ù      LATIN SMALL LETTER U WITH GRAVE
382		0000E4   C3 A4          ä      LATIN SMALL LETTER A WITH DIAERESIS
379		00016D   C5 AD          ŭ      LATIN SMALL LETTER U WITH BREVE
357		0000F6   C3 B6          ö      LATIN SMALL LETTER O WITH DIAERESIS
292		00017B   C5 BB          Ż      LATIN CAPITAL LETTER Z WITH DOT ABOVE
291		001D23   E1 B4 A3       ᴣ      LATIN LETTER SMALL CAPITAL EZH
277		00015A   C5 9A          Ś      LATIN CAPITAL LETTER S WITH ACUTE
275		0000E2   C3 A2          â      LATIN SMALL LETTER A WITH CIRCUMFLEX
268		0000E6   C3 A6          æ      LATIN SMALL LETTER AE
240		0000DF   C3 9F          ß      LATIN SMALL LETTER SHARP S
239		000307   CC 87          ̇      COMBINING DOT ABOVE
230		0000EA   C3 AA          ê      LATIN SMALL LETTER E WITH CIRCUMFLEX
201		0000F4   C3 B4          ô      LATIN SMALL LETTER O WITH CIRCUMFLEX
156		0000E3   C3 A3          ã      LATIN SMALL LETTER A WITH TILDE
150		0000F5   C3 B5          õ      LATIN SMALL LETTER O WITH TILDE
150		0004F1   D3 B1          ӱ      CYRILLIC SMALL LETTER U WITH DIAERESIS
129		000141   C5 81          Ł      LATIN CAPITAL LETTER L WITH STROKE
127		0001D4   C7 94          ǔ      LATIN SMALL LETTER U WITH CARON
120		000301   CC 81          ́      COMBINING ACUTE ACCENT
105		0000EC   C3 AC          ì      LATIN SMALL LETTER I WITH GRAVE
102		001EBD   E1 BA BD       ẽ      LATIN SMALL LETTER E WITH TILDE
94		000115   C4 95          ĕ      LATIN SMALL LETTER E WITH BREVE
85		000188   C6 88          ƈ      LATIN SMALL LETTER C WITH HOOK
75		000169   C5 A9          ũ      LATIN SMALL LETTER U WITH TILDE
74		00010D   C4 8D          č      LATIN SMALL LETTER C WITH CARON
74		000233   C8 B3          ȳ      LATIN SMALL LETTER Y WITH MACRON
68		0001B7   C6 B7          Ʒ      LATIN CAPITAL LETTER EZH
61		00017E   C5 BE          ž      LATIN SMALL LETTER Z WITH CARON
56		00201E   E2 80 9E       „      DOUBLE LOW-9 QUOTATION MARK
52		0004E0   D3 A0          Ӡ      CYRILLIC CAPITAL LETTER ABKHASIAN DZE
48		0000FB   C3 BB          û      LATIN SMALL LETTER U WITH CIRCUMFLEX
47		0000AD   C2 AD          ­      SOFT HYPHEN
44		0000D4   C3 94          Ô      LATIN CAPITAL LETTER O WITH CIRCUMFLEX
41		0000EB   C3 AB          ë      LATIN SMALL LETTER E WITH DIAERESIS
40		00044A   D1 8A          ъ      CYRILLIC SMALL LETTER HARD SIGN
40		000153   C5 93          œ      LATIN SMALL LIGATURE OE
38		00014F   C5 8F          ŏ      LATIN SMALL LETTER O WITH BREVE
30		00201D   E2 80 9D       ”      RIGHT DOUBLE QUOTATION MARK
28		000442   D1 82          т      CYRILLIC SMALL LETTER TE
22		00044E   D1 8E          ю      CYRILLIC SMALL LETTER YU
22		000117   C4 97          ė      LATIN SMALL LETTER E WITH DOT ABOVE
22		001E83   E1 BA 83       ẃ      LATIN SMALL LETTER W WITH ACUTE
22		0000EE   C3 AE          î      LATIN SMALL LETTER I WITH CIRCUMFLEX
22		00044B   D1 8B          ы      CYRILLIC SMALL LETTER YERU
21		000178   C5 B8          Ÿ      LATIN CAPITAL LETTER Y WITH DIAERESIS
21		002013   E2 80 93       –      EN DASH
20		0001EF   C7 AF          ǯ      LATIN SMALL LETTER EZH WITH CARON
20		000293   CA 93          ʓ      LATIN SMALL LETTER EZH WITH CURL
18		000440   D1 80          р      CYRILLIC SMALL LETTER ER
17		000151   C5 91          ő      LATIN SMALL LETTER O WITH DOUBLE ACUTE
17		0001D0   C7 90          ǐ      LATIN SMALL LETTER I WITH CARON
16		00A729   EA 9C A9       ꜩ      LATIN SMALL LETTER TZ
16		000101   C4 81          ā      LATIN SMALL LETTER A WITH MACRON
15		0001B5   C6 B5          Ƶ      LATIN CAPITAL LETTER Z WITH STROKE
15		001E79   E1 B9 B9       ṹ      LATIN SMALL LETTER U WITH TILDE AND ACUTE
15		000431   D0 B1          б      CYRILLIC SMALL LETTER BE
14		00043C   D0 BC          м      CYRILLIC SMALL LETTER EM
12		00043B   D0 BB          л      CYRILLIC SMALL LETTER EL
12		000304   CC 84          ̄      COMBINING MACRON
12		000106   C4 86          Ć      LATIN CAPITAL LETTER C WITH ACUTE
12		000187   C6 87          Ƈ      LATIN CAPITAL LETTER C WITH HOOK
11		000438   D0 B8          и      CYRILLIC SMALL LETTER I
11		000103   C4 83          ă      LATIN SMALL LETTER A WITH BREVE
11		0000D3   C3 93          Ó      LATIN CAPITAL LETTER O WITH ACUTE
11		000113   C4 93          ē      LATIN SMALL LETTER E WITH MACRON
11		00043D   D0 BD          н      CYRILLIC SMALL LETTER EN
10		0000FD   C3 BD          ý      LATIN SMALL LETTER Y WITH ACUTE
10		002026   E2 80 A6       …      HORIZONTAL ELLIPSIS
9		0000E7   C3 A7          ç      LATIN SMALL LETTER C WITH CEDILLA
9		000227   C8 A7          ȧ      LATIN SMALL LETTER A WITH DOT ABOVE
9		0000C4   C3 84          Ä      LATIN CAPITAL LETTER A WITH DIAERESIS
8		000434   D0 B4          д      CYRILLIC SMALL LETTER DE
8		0001CE   C7 8E          ǎ      LATIN SMALL LETTER A WITH CARON
7		0000C8   C3 88          È      LATIN CAPITAL LETTER E WITH GRAVE
7		00014D   C5 8D          ō      LATIN SMALL LETTER O WITH MACRON
6		000432   D0 B2          в      CYRILLIC SMALL LETTER VE
6		002014   E2 80 94       —      EM DASH
5		000437   D0 B7          з      CYRILLIC SMALL LETTER ZE
5		00043F   D0 BF          п      CYRILLIC SMALL LETTER PE
5		0000ED   C3 AD          í      LATIN SMALL LETTER I WITH ACUTE
5		0000F1   C3 B1          ñ      LATIN SMALL LETTER N WITH TILDE
5		001EE7   E1 BB A7       ủ      LATIN SMALL LETTER U WITH HOOK ABOVE
5		000179   C5 B9          Ź      LATIN CAPITAL LETTER Z WITH ACUTE
4		0000DC   C3 9C          Ü      LATIN CAPITAL LETTER U WITH DIAERESIS
4		00AB33   EA AC B3       ꬳ      Character in undefined range
4		0000EF   C3 AF          ï      LATIN SMALL LETTER I WITH DIAERESIS
4		000267   C9 A7          ɧ      LATIN SMALL LETTER HENG WITH HOOK
4		0001B6   C6 B6          ƶ      LATIN SMALL LETTER Z WITH STROKE
4		0004E7   D3 A7          ӧ      CYRILLIC SMALL LETTER O WITH DIAERESIS
4		00FEFF   EF BB BF               ZERO WIDTH NO-BREAK SPACE
3		0000B7   C2 B7          ·      MIDDLE DOT
3		00044F   D1 8F          я      CYRILLIC SMALL LETTER YA
3		000463   D1 A3          ѣ      CYRILLIC SMALL LETTER YAT
3		000203   C8 83          ȃ      LATIN SMALL LETTER A WITH INVERTED BREVE
3		00016F   C5 AF          ů      LATIN SMALL LETTER U WITH RING ABOVE
3		0003B1   CE B1          α      GREEK SMALL LETTER ALPHA
2		0001F0   C7 B0          ǰ      LATIN SMALL LETTER J WITH CARON
2		000130   C4 B0          İ      LATIN CAPITAL LETTER I WITH DOT ABOVE
2		000114   C4 94          Ĕ      LATIN CAPITAL LETTER E WITH BREVE
2		00AB4D   EA AD 8D       ꭍ      Character in undefined range
2		001E3F   E1 B8 BF       ḿ      LATIN SMALL LETTER M WITH ACUTE
2		00016B   C5 AB          ū      LATIN SMALL LETTER U WITH MACRON
2		000443   D1 83          у      CYRILLIC SMALL LETTER U
2		00043A   D0 BA          к      CYRILLIC SMALL LETTER KA
2		001F77   E1 BD B7       ί      GREEK SMALL LETTER IOTA WITH OXIA
2		0003BD   CE BD          ν      GREEK SMALL LETTER NU
2		000433   D0 B3          г      CYRILLIC SMALL LETTER GHE
1		00045D   D1 9D          ѝ      CYRILLIC SMALL LETTER I WITH GRAVE
1		000436   D0 B6          ж      CYRILLIC SMALL LETTER ZHE
1		0025D8   E2 97 98       ◘      INVERSE BULLET
1		00022F   C8 AF          ȯ      LATIN SMALL LETTER O WITH DOT ABOVE
1		000173   C5 B3          ų      LATIN SMALL LETTER U WITH OGONEK
1		0000C0   C3 80          À      LATIN CAPITAL LETTER A WITH GRAVE
1		000109   C4 89          ĉ      LATIN SMALL LETTER C WITH CIRCUMFLEX
1		000190   C6 90          Ɛ      LATIN CAPITAL LETTER OPEN E
1		0000C9   C3 89          É      LATIN CAPITAL LETTER E WITH ACUTE
1		001EF3   E1 BB B3       ỳ      LATIN SMALL LETTER Y WITH GRAVE
1		0002BC   CA BC          ʼ      MODIFIER LETTER APOSTROPHE
1		0000C2   C3 82          Â      LATIN CAPITAL LETTER A WITH CIRCUMFLEX
1		00010B   C4 8B          ċ      LATIN SMALL LETTER C WITH DOT ABOVE
1		001E9E   E1 BA 9E       ẞ      LATIN CAPITAL LETTER SHARP S

It is better not to use characters like this:

120		000301   CC 81          ́      COMBINING ACUTE ACCENT

Since that might cause problems when searching the text (single character with acute accent included is better).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Data Issue related to digital scores enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants