srml.py limitations

[chzapo]

I was looking over the source code and wanted to check my understanding of the variable map and its usage in srml.py.

The website link provided at
https://solardata.uoregon.edu/DataElementNumbers.html

Provides a more extensive list of column header translation than the variable map in pvlib-python. It appears pvlib-python only translates and ingests the columns that have built-in support within pvlib-python.

The comment in the code:
map_variables: bool, default: True When true, renames columns of the DataFrame to pvlib variable names where applicable. See variable :const: VARIABLE_MAP.

seems to corroborate this.

Is this correct?

The reason I was looking at this piece of code was I was hoping to dump a human readable .csv from the srml coded data file. I could write a piece of code to do this myself, but I was made aware that pvlib-python had a similar function. Is the above module the only place this exists or is there somewhere else in pvlib-python that has such a function (with 1-1 correspondence from number to string for the column headers) exists?

If this does not exist...would there be a desire for such a function (for srml) or functions for the other iotools in order to do this type of work (1-1 translation to human readable .csv files from the coded forms)?

Thanks for any feedback on this topic.

(Feels like there are probably 10 personal variations floating around that do this for srml...but maybe not).

[cwhanse]

Is this correct?

Yes.

would there be a desire for such a function (for srml)

The challenge is keeping the VARIABLE_MAP constant up to date with changes in the data source, which are nearly always a surprise to pvlib maintainers.

There are quantities that are useful for pvlib that aren't in the current map, e.g., barometric pressure, rainfall, precipitable water. PR welcome here.

I also don't see any harm in adding mapping for other variables that pvlib doesn't currently use, e.g., tilted irradiance. If we can agree on how to name those quantities.

[chzapo]

Okay I can work on this.

One thing I was thinking as well was maybe breaking out the variable map dictionary into a separate module and calling the module from the primary library module. This doubles the number of files, but separates the data (variable map) piece from the actual code. The advantage here is that we don't have to update the primary library module every time we need to update the variable map. A second advantage is as the variable map dictionary grows then we don't have to have data taking up more and more of the primary library module real estate (thus making reading it unwieldy). I can see two ways of doing this, one just having a global dictionary in the secondary module and calling it via the a module import in the primary library module or two putting the dictionary in a function in the variable map module and calling the function (with its return being the dictionary) after a module import.

Given my lack of experience with programming guidelines in pvlib-python I am not sure which fits project structure best. Which of these options (or if there is another option) would be the preferable route for doing this? Or do we just want to leave the variable map in the primary library module.

Thanks.

[echedey-ls]

That's amazing @chzapo !!
I recommend keeping it in the current module for the following reasons:

• There are other VARIABLE_MAP's in pvlib, each one for different API services.
• Documentation and references are all kept in the same place, to ease maintenance.

Just a heads up, a lot of things go on in each PR and at first it may feel a little overwhelming. So if you need any help feel free to reach out. A good starting point is this documentation page.

[chzapo]

It looks like pull requests can only be submitted after there is a change in the code base. So I think I will hold off on that till I get this all working (since it is only like 60 entries in a dictionary). What I do want to get a comment on is my naming scheme. I am going to post my proposed structure under issues. I also just found the Nomenclature page in yours scheme which is very helpful. I am going to try follow that where ever possible. Hopefully issues is the right place to put that and get comments.