update gtfs.py script

[sidetrackedmind]

I saw the following error

/home/runner/work/opentransit-metrics-fork/opentransit-metrics-fork/backend/models/gtfs.py:756: ShapelyDeprecationWarning: The array interface is deprecated and will no longer work in Shapely 2.0. Convert the '.coords' to a numpy array instead.

When running:
python save_routes.py --s3 --timetables --scheduled-stats --agency=trimet

[nkanderson]

Hi @sidetrackedmind! I started taking a look at this one, and had a couple of questions for you.

• Looking at the shapely docs, it seems like there are some coordinates where the call to .coords needs to be wrapped in np.array. Specifically, get_direction_data seems to have a handful of these calls. Does that sound correct to you?
• Do you know if there are any existing tests that would be helpful in working through this issue? I can indeed replicate the deprecation warning by running the command you provided above, but it also takes a while to run, and produces a lot of output that's not necessarily helpful in troubleshooting this issue. If not, any chance you have any debug config (i.e. a launch.json file) for vscode? :) I can look into creating something like that, but thought I'd see what you might have first.

[sidetrackedmind]

Hey @nkanderson - first off, thanks for taking a look at this!

I think you're on the right track with that shapely doc you referenced. From the doc, it seemed like the .coords call was "okay". I was looking at this line in the shapely doc in particular:

Starting with Shapely 1.8, converting a geometry object to a NumPy array directly will start raising a warning

I see an instance of that in line 842 of get_direction_data.

shape_lon_lat = np.array(geometry).T

Good question about existing tests.

• All the tests for the backend are in this folder: https://github.com/codeforpdx/opentransit-metrics/tree/master/backend/tests. They are run via a github action configured here: https://github.com/codeforpdx/opentransit-metrics/blob/master/.github/workflows/test.yml.
• I don't think there's a test that covers this issue but I'll be honest that I haven't really jumped in and looked at the existing tests. Regarding a debug config (launch.json file) <- I'm not familiar with this but it sounds like something we should have lol.
• Regarding save_routes.py taking a while to run. If you're comfortable, it could be worth tackling the issue one function at a time. For instance, focusing on get_direction_data first and then looking for other function with the np.array(geometry) call.

Sorry that's a lot of info. Please add more comments as you have questions :)!

[nkanderson]

@sidetrackedmind thank you, this is helpful! I think since I'm not yet very familiar with the data being passed around here, it would help me to create some debug config and just poke around a bit. If I get a launch.json file that seems like it could be helpful to others (I know vs code is generally a pretty popular editor), I'll open a PR with it. That all will hopefully help with adding unit tests as well, if I get to that point, or just finding the spots to update np.array calls.

[sidetrackedmind]

@nkanderson - sounds great! Feel free to put questions you have here or in slack.

[GoodGuyGregory]

Can we close this issue? I ask because it appears to be merged.

[nkanderson]

@GoodGuyGregory I know we were able to complete the addition of debugging config in PR #48 linked above, but at least while I was working on this project, we hadn't merged a fix for this issue related to the deprecation message. But if you've been able to verify it's been resolved, closing sounds great as far as I'm concerned!