NumFOCUS Small Development Grant Spring 2023
We propose to create an external code completion plugin for the Spyder editor by reusing the code currently available for the Kite provider, document this implementation process, and improve the maintenance of the project by removing the code related to it, which is obsolete now. In addition, we will research, get community feedback and, if time permits, try to develop a full implementation of another completion provider based on AI or Jupyter kernel-based technologies, which users have been requesting after Kite’s support was disabled last year.
As part of a partnership between Spyder and Kite that took place between 2019 and 2021, a new code completion client that used their service was created to improve completions in Spyder’s editor [1]. Although later Kite stopped that service [2] and its client was disabled in Spyder [3], much of Kite’s client code is still present in Spyder’s repository as dead code.
In addition, part of the idea behind partnering with Kite was to provide better completions for our editor by designing an extensible API that could easily allow to supply other sources for completions. After several iterations, that API is now in good shape, so we are in a position to greatly enhance the code completion experience in Spyder. We consider this is an important need for many users because they have let us know the shortcomings of our current completions several times through Github issues and other communication channels. Some have even inquired about the possibility of creating plugins to improve the code completion experience offered by Spyder.
This grant would help to make Spyder’s API to create external code completion plugins clearer and identify other completion providers that could be of interest to the Spyder community to improve the coding experience in its editor. In particular, we propose to do the following:
- Remove the remaining Kite code completion provider code present in Spyder, which would help the project’s maintenance by eliminating a lot of dead code.
- Create an external plugin using that code, which would serve as an example to other developers of how completion plugins should look and be structured.
- Create documentation on how you can develop a completions plugin provider. This will follow the same spirit as the current documentation for general plugins [4].
- Research and, if time permits, develop a full implementation of another code completion provider by leveraging either on AI-based services offered for that (e.g. OpenAI Codex[5] and Tabnine [6]), or other kinds of backends (e.g. the Jupyter kernel connected to Spyder’s IPython console). We plan to decide which provider to develop by asking the broader Spyder Community about their interest in such an implementation effort.
With this proposal we aim to fund Juan Bautista-Rojas, a junior developer who has done a great job helping the project with two previously awarded NumFOCUS Small Development Grants: Improving the Spyder IDE installation experience [7] and Improving Spyder UX/UI [8]. Juan Sebastian will be supervised and working with one of our core developers (Daniel Althviz-Moré), for 6 months full-time in order to develop the mentioned external completions plugin based on the Kite client code, creating the corresponding documentation page for such plugins, and helping do an initial research and gathering general feedback from the Spyder Community regarding other completions providers that they would like to see implemented.
Notes:
[1] Blog | Spyder 4.0: Kite integration is here
[4] Plugin Development with Spyder
[5] OpenAI Codex
[6] TabNine/HowToWriteAClient.md at master
[7] NumFOCUS Small Development Grant 2022 · spyder-ide/spyder Wiki · GitHub (previously awarded grant)
[8] NumFOCUS Small Development Grant Fall 2022 · spyder-ide/spyder Wiki (github.com) (previously awarded grant)
The implementation of this project would have a positive impact on the Spyder community because third-party plugin developers would have more documentation and examples on how to improve the code completion experience in Spyder. At the same time, this proposal would help us better understand users’ expectations about other possible code completion providers. The gained insight would allow us to start working on the provider the community finds the most useful for their coding needs. Finally, we consider this would be quite beneficial to set up a path for the Spyder community to create their own completions plugins, which would increase Spyder’s usability for current and new users.
A major benefit to the Spyder project itself will be that Juan Bautista-Rojas, currently a junior developer who is working on a NumFOCUS Small Development Grant to improve Spyder’s UX/UI, will be able to continue working on Spyder full-time for another 6 months. This funding would also give him the opportunity to continue building up the necessary experience in open source while allocating time to continue his contributions to the project.
- Spyder IDE on Github: https://github.com/spyder-ide/spyder
- Month 1:
- Analysis of the Kite code completion provider code and removal of it from the Spyder main repository.
- Start moving that code to an external repository and do the necessary changes to make it work for Spyder 6.
- Month 2:
- Continue the implementation of an example code completion plugin and refine the completions plugin API as needed.
- Release an initial version of the new code completions plugin.
- Month 3:
- Draft an initial documentation page related to the creation of external completions plugins for Spyder.
- Do bug fix releases for the example code completion plugin as needed.
- Month 4:
- Release the new documentation page related to the creation of external completions plugins for Spyder.
- Start researching alternative providers for services similar to Kite or based on Jupyter kernels.
- Month 5:
- Continue researching alternative completion providers.
- Gather feedback from the Spyder Community regarding the providers they would like to see implemented.
- Month 6:
- Start an initial implementation of a code completions plugin following the previous research.
- Share gathered feedback and advances towards a new completions plugin implementation and the new resources available (example code, documentation, etc) with the Spyder Community through one or more blog posts.
- Juan Bautista-Rojas (Spyder junior developer) - @jsbautista
- Daniel Althviz-Moré (core developer, main reviewer) - @dalthviz
- Spyder volunteer contributors interested in the project
As part of this proposal we were able to:
-
Do a code refactor for the Kite code present inside Spyder main repository and repurpose it in a external repository as an example of how to structure a external completion provider as well enable it usage as an external provider for Spyder 6+
-
A new wiki page was created as an initial documentation for the current Spyder Completion API and ways to extend Spyder completion functionality with it using as an example the newly defined Kite provider
-
Creation and release of an external completion provider for Spyder using Langchain and OpenAI models
-
General review of the Spyder completion API and related functionality available. Elements that will need to be worked on in the future to support AI code assitants like Copilot or Tabnine were defined. Some of issues with discussions and ideas: