Specifying special characters in "insert_htmlbox"

[smithah]

Description of the bug

Trying to write this
text - "अधिक जानकारी के लिए customerservice@axismf.com। निवेशकों को केवल पंजीकृत म्यूचुअल फंड से ही लेनदेन करना चाहिए, जिसका विवरण www.sebi.gov.in पर उपलब्ध है -"
to PDF getting blank in the space where English characters is present.

page.insert_htmlbox( # page is a PDF Page object
(32.649353027344, 688.83270263672, 537.9233528971674, 703.54669189453), # rectangle inside the page
"अधिक जानकारी के लिए customerservice@axismf.com। निवेशकों को केवल पंजीकृत म्यूचुअल फंड से ही लेनदेन करना चाहिए, जिसका विवरण www.sebi.gov.in पर उपलब्ध है -", # text string or a Story object
css="body {font-size:7pt;font-family:Noto Sans Devanagari Regular;color:#373334;text-decoration: underline;}",
scale_low=0, # limit scaling down when fitting content
archive=None, # points to locations of fonts and images
rotate=0, # clockwise rotate content by this angle
oc=0, # assign xref of an OCG (conditional visibility)
opacity=1, # make content transparent (default: 1 = no)
overlay=True
# put in foreground (default) or background
)

The text doesn't getting rendered tired to open the PDF is Acrobat reader and Chrome Browser as well, the text though present is not visible Need help on this. Thanks

How to reproduce the bug

Trying to write this
text - "अधिक जानकारी के लिए customerservice@axismf.com। निवेशकों को केवल पंजीकृत म्यूचुअल फंड से ही लेनदेन करना चाहिए, जिसका विवरण www.sebi.gov.in पर उपलब्ध है -"
to PDF getting blank in the space where English characters is present.

page.insert_htmlbox( # page is a PDF Page object
(32.649353027344, 688.83270263672, 537.9233528971674, 703.54669189453), # rectangle inside the page
"अधिक जानकारी के लिए customerservice@axismf.com। निवेशकों को केवल पंजीकृत म्यूचुअल फंड से ही लेनदेन करना चाहिए, जिसका विवरण www.sebi.gov.in पर उपलब्ध है -", # text string or a Story object
css="body {font-size:7pt;font-family:Noto Sans Devanagari Regular;color:#373334;text-decoration: underline;}",
scale_low=0, # limit scaling down when fitting content
archive=None, # points to locations of fonts and images
rotate=0, # clockwise rotate content by this angle
oc=0, # assign xref of an OCG (conditional visibility)
opacity=1, # make content transparent (default: 1 = no)
overlay=True
# put in foreground (default) or background
)

The text doesn't getting rendered tired to open the PDF is Acrobat reader and Chrome Browser as well, the text though present is not visible Need help on this. Thanks

PyMuPDF version

1.24.5

Operating system

Windows

Python version

3.10

[smithah]

I have also used page.clean_contents(sanitize=True) after each insert_htmlbox, still it does not print this line properly

[smithah]

"अधिक जानकारी के लिए customerservice@axismf.com**।** निवेशकों को केवल पंजीकृत म्यूचुअल फंड से ही लेनदेन करना चाहिए, जिसका विवरण www.sebi.gov.in पर उपलब्ध है -"

Its because of the highlighted in bold character in the text string.

[smithah]

The pipe character if present in the html text to be inserted replaced it with - Alternatively, use one of the HTML entities for the pipe character, e.g. | (or the more meaningful |

Now its printing the characters correctly as given below. Is there any other way to handle special characters in the insert_htmlbox while printing text with special characters. Thanks

[smithah]

| or | using either of the two html entities instead of the pipe symbol solves the issue for now, finding the symbol in the text and doing a text replace with either of the above html entites solves the issue as of now.

[JorjMcKie]

I must confess that I am confused.
Are you reporting an issue or not? In the end you seem to say that you found the correct way to achieve what you want.

In any case I see no problem caused by any of PyMuPDF or MuPDF. At best, this is a Discussions item therefore, I am going to transfer this.

[JorjMcKie]

As a general comment, please try to correctly encapsulate code lines using triple back-ticks. Your code was hardly readable without that.

[smithah]

Thanks still looking for a better way to handle special characters in the text

[JorjMcKie]

The problem is the correct specification in the string variable you are passing in - not anywhere else. Definitely not in page.insert_htmlbox, as you have demonstrated yourself.

Once understandably defined, the character is correctly written.
In your original text string definition, "।" is not the pipe character but glyph "danda" with a Unicode value of 2404. | is just the normal chr(124).

[smithah]

Thanks for the concise explanation. Trying to handle that symbol with a font ttf file

[JorjMcKie]

Here is an easy solution with one of the fonts in package pymupdf-fonts:

import pymupdf
doc=pymupdf.open()
page=doc.new_page()
rect=pymupdf.Rect(100,100,300,300)
text="अधिक जानकारी के लिए customerservice@axismf.com। निवेशकों को केवल पंजीकृत म्यूचुअल फंड से ही लेनदेन करना चाहिए, जिसका विवरण www.sebi.gov.in पर उपलब्ध है -"
arch=pymupdf.Archive()
css=pymupdf.css_for_pymupdf_font("figo", archive=arch, name="sans-serif")
css += "* {font-family: sans-serif;}"
page.insert_htmlbox(rect,text,css=css,archive=arch)
doc.ez_save("x.pdf")

Gives you this:

[smithah]

That's awesone Thanks...i need to use the Noto Sans Devanagari font itself ...as it needs to handle various Indic Languages

[JorjMcKie]

Well, then there is no help for you if that font does not contain danda (which I don't know).
The font I used (Fira GO) supports extended Latin and Indic languages so I guess I am out of good advice if you don't like it.
Here is what the font has to say about its support:

Supported Script(s):
Latin
Cyrillic
Greek
Arabic
Hebrew
Thai
Georgian
Devanagari
68 languages and many more:
Afrikaans, Albanian, Basque, Bodo, Bosnian, Breton, Bulgarian, Corsican,
Croatian, Czech, Danish, Dutch, English, Esperanto, Estonian, Faroese,
Farsi, Finnish, French, Galician, German, Greek, Greenlandic, Hebrew, Hindi,
Hungarian, Icelandic, Indonesian, Irish, Italian, Konkani, Kurdish, Latvian,
Leonese, Lithuanian, Luxembourgish, Malay, Malaysian, Malti, Manx, Marathi,
Nepali, Norwegian, Occitan, Polish, Portuguese, Rhaeto-Romanic, Romanian,
Russian, Sami, Scottish Gaelic, Serbian Latin, Slovak, Slovakian, Slovene,
Slovenian, Sorbian (Lower), Sorbian (Upper), Spanish, Swahili, Swedish,
Thai, Turkish, Turkmen, Ukrainian, Vietnamese, Walloon, Yiddish

[smithah]

Testing it give me sometime..will update here

[smithah]

Thanks its working for the languages that i am handling...the font to be chosen for each text to be written is dynamic and is coming from a source, Noto Sans is one of them. Need to check if i can proceed with the font that you have suggested as its handling the languages being used.

Much Appreciate your help on this.

[smithah]

Using this instead now:

css=pymupdf.css_for_pymupdf_font("notos", archive=arch, name="Noto Sans Regular") css += "* {font-family: Noto Sans Regular;}"

its printing the invisible characters , except that "danda" is coming as a box ,

Thank you much for your inputs and suggestions . Totally appreciate it. Will have to check on this.