segfault when passing the wrong type to translate

[jimy-byerley]

Hello @Zuzu-Typ
I have a systematic segfault when passing some wrong type to translate and rotate

right argument types work well, but any argument that should theoretically raise a TypeError triggers a segfault

translate(vec3())  # ok
translate([1,2,3]) # ok
translate((1,2)) # ok

translate('str')  # segfault
translate(0) # segfault
translate(0.0) # segfault

It seems to come from this macro according to gdb

jimy@machine:~/cad/PyGLM$ gdb python
GNU gdb (Debian 13.1-3) 13.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python...
(No debugging symbols found in python)
(gdb) run
Starting program: /usr/bin/python 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from glm import *
>>> translate('truc')

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7323f12 in translate_ (args=<optimized out>) at PyGLM/functions/stable_extensions/matrix_transform.h:182
182             PyGLM_TYPEERROR_2O("invalid argument type(s) for translate(): ", arg1, arg2);
(gdb) bt
#0  0x00007ffff7323f12 in translate_ (args=<optimized out>) at PyGLM/functions/stable_extensions/matrix_transform.h:182
#1  0x0000000000547ae8 in ?? ()
#2  0x0000000000517eb3 in _PyObject_MakeTpCall ()
#3  0x000000000052b940 in _PyEval_EvalFrameDefault ()
#4  0x000000000052360b in PyEval_EvalCode ()
#5  0x0000000000647497 in ?? ()
#6  0x0000000000644d4f in ?? ()
#7  0x00000000004832e7 in ?? ()
#8  0x0000000000482e19 in _PyRun_InteractiveLoopObject ()
#9  0x0000000000463437 in ?? ()
#10 0x00000000004633ca in PyRun_AnyFileExFlags ()
#11 0x0000000000461ce4 in ?? ()
#12 0x00000000006275c7 in Py_BytesMain ()
#13 0x00007ffff7cc524a in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#14 0x00007ffff7cc5305 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#15 0x0000000000627461 in _start ()

I seems to be due to when arg2 is NULL, this macro still attempts to retreive its type name.
So I fixed it by introducing

#define PyGLM_TYPEERROR_1O(str, obj1) PyErr_Format(PyExc_TypeError, "%s'%s'", str, PyGLM_TYPE_AS_CSTRING(obj1));

and calling PyGLM_TYPEERROR_1O when only arg1 is non-null
and calling PyGLM_TYPEERROR_2O when both arg1 and arg2 are non-null

[Zuzu-Typ]

Hi Jimy,

thank you for reporting this!
I'm sure those two aren't the only instances of this oversight.
I'll do some digging.

If I find any additional ones, I'll fix them in your PR.

Cheers
--Zuzu_Typ--

[Zuzu-Typ]

I've looked through the code and found that the only occurence of this error is calling translate with a single argument of an incorrect type.

[jimy-byerley]

Thanks for answering so fast :)