Should MetPy be helping with ndarrays coming from pandas Series of object dtype

[akrherz]

I figured I would start this as a discussion as I am not so sure this is an issue in MetPy's purview, but the pandas 2.2 fillna downcasting change is kind of beating me over the head at the moment. Consider this code example

import numpy as np

import pandas as pd
from metpy.calc import wind_components
from metpy.units import units

pd.set_option("future.no_silent_downcasting", True)
df = pd.DataFrame(
    {
        "sknt": pd.Series([1, 2, 3, None], dtype=object),
        "drct": pd.Series([4, 6, 7, None], dtype=object),
    }
).fillna(np.nan)


print(
    wind_components(
        units("knot") * df["sknt"].values,
        units("degree") * df["drct"].values,
    )
)

This yields

/home/akrherz/projects/MetPy/src/metpy/calc/basic.py:1304: RuntimeWarning: invalid value encountered in greater
  if np.any(np.greater(np.abs(value), max_radians)):
AttributeError: 'float' object has no attribute 'sin'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/akrherz/DEV/metpy/error.py", line 17, in <module>
    wind_components(
  File "/home/akrherz/projects/MetPy/src/metpy/xarray.py", line 1329, in wrapper
    result = func(*bound_args.args, **bound_args.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/akrherz/projects/MetPy/src/metpy/units.py", line 333, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/akrherz/projects/MetPy/src/metpy/calc/basic.py", line 151, in wind_components
    u = -speed * np.sin(wind_direction)
                 ^^^^^^^^^^^^^^^^^^^^^^
TypeError: loop of ufunc does not support argument 0 of type float which has no callable sin method

Now, you may say "what in the world are you doing creating a dtype=object series?" Well, this is just an illustration of things that happen in the wild when reading from databases and other sources whereby sometimes pandas does not create a numeric datatype and you get the the dreaded object. In the good ole days, it seemed like .fillna(np.nan) would automagically fix this situation and I could blissfully go about my day.

The pandas docs suggest adding .infer_objects() after the .fillna() call, which works

(<Quantity([-0.06975647 -0.20905693 -0.36560803 nan], 'knot')>, <Quantity([-0.99756405 -1.98904379 -2.97763845 nan], 'knot')>)

So I am questioning /auditing everywhere in my expansive and junky code that I have .fillna(np.nan) and wondering to do. I suspect others are in my boat and will see seeing fun MetPy exceptions like the above and wondering what to do other than cry and complain on forums, like I do. Perhaps there is magic MetPy could be doing to save (l)users like me from themselves.

[dopplershift]

Well to start, I'd want to see authentic, real-world examples where data are returned as object arrays (not manually constructed), to understand what the challenges are around that being done automatically. I assume there's a reason that Pandas isn't doing that already.

One avenue would be to then turn those authentic examples into examples, either here or in the forthcoming MetPy cookbook over on Project Pythia.

My gut reaction is I'm really hesitant to add more special input processing on our functions. We already are hiding a lot of complexity around NumPy + Pint vs. xarray.