Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-106104: Use public weakref APIs in sqlite3 #106105

Closed

Conversation

erlend-aasland
Copy link
Contributor

@erlend-aasland erlend-aasland commented Jun 26, 2023

The sqlite3 extension module should be built using the public API, so
we're removing the Py_BUILD_CORE_MODULE define that was introduced as
part of the gh-105927 changes.

The sqlite3 extension module should be built using the public API, so
we're removing the Py_BUILD_CORE_MODULE define that was introduced as
part of the pythongh-105927 changes.
Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it an issue to use the internal C API?

@@ -102,12 +97,18 @@ pysqlite_close_all_blobs(pysqlite_Connection *self)
{
for (int i = 0; i < PyList_GET_SIZE(self->blobs); i++) {
PyObject *weakref = PyList_GET_ITEM(self->blobs, i);
PyObject *blob = _PyWeakref_GET_REF(weakref);
if (blob == NULL) {
if (!PyWeakref_Check(weakref)) {
continue;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible that blobs contain something which is not a weakref?

Py_DECREF(blob);
PyObject *blob;
if (PyWeakref_GetRef(weakref, &blob) < 0) {
PyErr_WriteUnraisable((PyObject *)self);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should make _PyErr_WriteUnraisableMsg() public, since this one logs a message which is hard to get, it gives no context.

@erlend-aasland
Copy link
Contributor Author

Is it an issue to use the internal C API?

Not a big issue, but historically, the sqlite3 extension module has been implemented (mostly) using public API. See also:

@erlend-aasland
Copy link
Contributor Author

Related: I see that _ssl also accidentally got Py_BUILD_CORE_MODULE defined in 46a3190 ;)

@vstinner
Copy link
Member

Related: I see that _ssl also accidentally got Py_BUILD_CORE_MODULE defined in 46a3190 ;)

How is it an issue to use the internal C API? The internal C API is written for best performances thanks to inlining when you control the argument types.

Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Well, if you don't want to use the internal C API in sqlite3, yep, the change is good. But warning: I made other sqlite3 changes in the meanwhile to use more internal APIs!?

PyErr_WriteUnraisable((PyObject *)self);
continue;
}
if (item) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest if (!item) continue; to avoid one indentation level.

@erlend-aasland
Copy link
Contributor Author

Related: I see that _ssl also accidentally got Py_BUILD_CORE_MODULE defined in 46a3190 ;)

How is it an issue to use the internal C API? The internal C API is written for best performances thanks to inlining when you control the argument types.

Well, I'll let you reply yourself (in #88140 (comment)) 😄:

I would prefer to limit the usage of the internal C API in extension modules built as dynamic libraries. See bpo-41111: "[C API] Convert a few stdlib extensions to the limited C API (PEP-384)".

@vstinner
Copy link
Member

I had great hope in being able to convert some stdlib extensions to the limited C API, but then i fall into Argument Clinic and I lost faith. I mean that I found more urgent things to do.

If someone is volunteer to attempt that again, I'm very supportive.

Maybe we might try to define some embargo on abusing private and internal APIs in Python stdlib extensions 😁 It should be the exception, not the default. I'm not sure.

@vstinner
Copy link
Member

Hi, what's the status of this PR?

About converting some stdlib C extensions to the limited C API, you can see: #85283

@erlend-aasland
Copy link
Contributor Author

I don't plan to follow this up right now. If we decide to convert _sqlite3 to use the Limited API in the future, we can revisit this.

@erlend-aasland erlend-aasland deleted the sqlite-weakref-public-api branch October 22, 2023 21:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The sqlite3 module should not be built as a core module
3 participants