Move code for sqlalchemy from scaffold to its own package

[mfrlin]

Coming from other frameworks I was surprised that scaffolding generated sqlalchemy integration code. What if I did not use sqlalchemy scaffold or what if best practice for integration changed?

I think it would be better to create a package, that you just include like pyramid_debugtoolbar and it sets up the session on request and has importable Base for models.

I did some research and there are already pyramid_sqlalchemy and pyramid_basemodel, but from my understanding they use thread locals for accessing the session.

Are there any issues or concerns about moving the code, that is currently in scaffolds to its own package?

[stevepiercy]

@mfrlin I'm not clear what you mean.

Pyramid currently comes with 3 scaffolds.

We have plans to remove the scaffolds from Pyramid, and have taken steps in that direction in #2384

cookiecutters are in separate packages now: https://github.com/Pylons?q=cookiecutter

[mfrlin]

I'm talking about 2 files mainly:
https://github.com/Pylons/pyramid/blob/master/pyramid/scaffolds/alchemy/%2Bpackage%2B/models/__init__.py_tmpl

and

https://github.com/Pylons/pyramid/blob/master/pyramid/scaffolds/alchemy/%2Bpackage%2B/models/meta.py

They basically integrate SQLAlchemy into your Pyramid project. But would not be better if you included the package that would done that for you instead of generating/copy pasting code?

[stevepiercy]

I still don't understand why a separate package would be better, unless it's a cookiecutter like we have already done and are already in the process of moving toward. The scaffold or cookiecutter does the generation of the project for you, copying files from the scaffold, filling in the slugs, and saving them into a destination directory.

[zupo]

@stevepiercy: Consider the following two examples:

1. User uses cookiecutter to start a new project. A bunch of SQLAlchemy integration code is generated for the user. Most of the code looks like magic and somewhat distracts the user from proceeding to do what she came there for: create views, etc. Then 6 months pass. Best practices change. User needs to rewrite integration code.

2. User uses cookiecutter to start a new project. The project has a dependency on pyramid_sqla and zero SQLAlchemy integration code in __init__.py making the project code leaner and more readable. Then 6 months pass. Best practices change. User bumps the version of pyramid_sqla and that is it.

This issue proposes to create a new (officially supported, in Pylons repo, ...) pyramid_sqla package that is required for the 2. example above.

[mmerickel]

This is something I've been thinking about for a very very long time and may be close to a possible design. I appreciate the concern but it's not at all easy to come up with something that works well in a variety of projects. The current scaffold could be considered a test bed for the design and so far people have seemed to be very happy with it. Also thanks to work done by @miohtama in websauna there's even more patterns that could be pulled out.

Are there any issues or concerns about moving the code, that is currently in scaffolds to its own package?

Yes, it's very tricky to do right for a variety of scenarios. The advantage of having this code in the scaffolds is that it's directly editable by people who have more advanced use cases which is basically anything under the sun from multiple databases or multiple bases to simply adding various hooks.

Finally, the integration of SQLAlchemy with Pyramid is basically a single file (model/__init__.py). The rest of the code is mostly just boilerplate for configuring sqlalchemy (model/meta.py) and this code is more general than a pyramid binding. The library would need to take that into account as many people using sqlalchemy are using it not just with pyramid but with scripts, queues, etc. Coupling the code to pyramid by doing something like flask-sqlalchemy makes the model harder to reuse in some pretty fundamental ways.

[digitalresistor]

There are various tickets where we have discussed this almost ad nasium. Changing from threadlocal to non-threadlocal and using request.dbsession in an external package would have been a breaking change.

Pulling this out into a separate package doesn't make much sense

[zupo]

The advantage of having this code in the scaffolds is that it's directly editable by people who have more advanced use cases which is basically anything under the sun from multiple databases or multiple bases to simply adding various hooks.

I have to disagree with this one. Scaffolds, as I see it, are meant for newcomers, not advanced users. If a user has to edit the integration code because of advanced use case, the user is advanced enough to move the integration code into their project and edit it there. But such an advanced user is not the target audience for scaffolds.

Also, I've been using pyramid in production for over three years, serving over 5000 customers. Not once have I had a need to for multiple SQL databases. For users such as me, the integration code just sits there and does not really change. And as such, it makes sense to keep it out of the business logic code and in a supporting package.

[ztane]

I have to disagree with @zupo - I always start a new project with a scaffold. That things are spelled out in the scaffold is a big plus over Django, where one doesn't have any clue about how the magic works.

And Flask-SA is just... well, let's leave it there.