Text insertion based on font encoding

[meghanaviyyapu]

I have tried extracting the spans of the first line of the PDF and reinserting using a different color via page.insert_text().
When I used page.get_fonts() to extract the fonts, I observed that the font name was Arial Unicode MS and the same font was present in the output with different encodings (WinAnsi and Identity-H).

page.insert_text() was able to insert the complete text when font reference name corresponding to Identity-H was used but when font reference name corresponding to WinAnsi encoding was used, the created PDF had weird characters. Could you let me know the reason for this?
hindi.pdf

[JorjMcKie]

I saw that all text on this page is Hindi, written in some Devanagari font.
You can never write Devanagari text using any of the methods insert_text, insert_textbox or TextWriter!
Devanagari-based languages and many others need so-called "text shaping" to correctly write them.
Please use insert_htmlbox. This is the only way in PyMuPDF to write with text shaping.

[meghanaviyyapu]

insert_text worked fine even in the above case but when C2_0 font reference was used which had Identity-H encoding but when TT0 reference was used it did not work. Can you please let me know how to predict only font name corresponding to a specific encoding should be used?

[JorjMcKie]

The attached PDF must be wrong, it contains no text and thus no font!

[meghanaviyyapu]

Osho_Rajneesh_Sambhog_Se_Samadhi_Ki_Or.pdf
Sorry, Have attached the PDF. Was trying to change the font color of the first line in the first page of PDF. The color got changed when font name corresponding to Identity-H encoding was used but not when font name with WinAnsi was used. Can you please let me know how to predict only font name corresponding to a specific encoding should be used?

[JorjMcKie]

This is not possible given that the two font names are exactly equal - even in terms of their subset prefixes.

[meghanaviyyapu]

Okay thanks, can you let me know why Identity-H encoding works while inserting the text but not WinAnsiEncoding in this case?

[JorjMcKie]

Identity-H is defined as:
The horizontal identity mapping for 2-byte CIDs; may be used with CIDFonts using any Registry, Ordering, and Supplement values. It maps 2-byte character codes ranging from 0 to 65,535 to the same 2-byte CID value, interpreted high-order byte first (see below).

WinANSIEncoding is a predefined mapping for the Latin character code range.

[meghanaviyyapu]

Thanks for clarifying. Is there any way to know which font reference name to be used in this case(same font names in page.get_fonts())?

[meghanaviyyapu]

Yes but when I extract the font name the extracted font name (span["font"]) matches with the font name in page.get_fonts() twice so how to handle the ambiguity?

[JorjMcKie]

There is no way - did I not mention that already? All you can possibly get is the font name (not the reference name "F1", etc.). If the font names are equal down to even the subset prefix: bad luck!

[meghanaviyyapu]

It was mentioned but I wanted to know if the font name corresponding to Identity-H encoding be used as default when font names are equal?

[JorjMcKie]

No - no such mechanism.
Possibly, you could look at the two font binaries. Make a fitz.Font from both of them, then look at your text and check if all of its characters are contained in either of them. There is method Font.has_glyph with the ord() value of the characters as argument.
Not sure how reliable this is - but maybe worth a try.

[meghanaviyyapu]

Ok thanks for the information