Towards a long term, framework independent, Avalanche

[kwcooper]

Hey all, not sure of the best place to put this, and forgive me if off-base or I missed a more extensive discussion, but I thought here was as good a place as any to keep it and a record in one place, as well as a pointer if anyone else has this question.

The question revolves around the Avalanche backend, namely how integrated the library is with PyTorch. Don't get me wrong, PyTorch is awesome, and certainly the or one of the most popular frameworks to date. But this wasn't always the case, and given the speed of development of novel projects, may likely not be in the future.

I remember early conversations about later adding additional backends to the library, namely Tensorflow/Keras. And I've seen others raise that question. But given the scope of adding it, potential marginal benefits, and developmental complexities, it's understandable why it wasn't a priority in the Alpha and now Beta.

So really I just wanted to gauge thoughts of what we all might think would be the best way forward might be:

• Do we think there's a chance some subset of users wouldn't want to use Avalanche because of the backend, or not wish to port their existing code?
• Aside from the huge developmental effort, what advantages/disadvantages might there be to adding additional backends, or even creating 'Avalanche XXX edition' in the future?
• Might future deep learning frameworks outpace PyTorch, potentially deprecating the codebase in the future?
• Or is sticking with what we have fine for the foreseeable future.

By no means do I think this is a priority in this early stage, though I would wager it's worth thinking about should it potentially influence design decisions based on discussions we might have.

Thanks everyone!

[vlomonaco]

This is an important discussion we should keep alive, thanks @kwcooper! I think that given the current human resources we have, we cannot support the development of an additional backend or a separate codebase based on a different framework. That's said it would be awesome to generalize avalanche and to make it framework independent in the future. When I first envisioned the tool, this was on top of my desiderata list. However it adds a lot of complexity which makes us move slowly in terms of integrating new continual learning features into the tool.

[ggraffieti]

Hi Keiland, these are some important considerations and I'm glad that you point them out.
I agree with @vlomonaco, given the current human resources and the fact that we are all volunteering in the development of Avalanche, now we don't have the strength to support a new version of Avalanche for another deep learning framework.
To integrate what Vincenzo said, Avalanche was first conceived as a multi-framework tool, to work with PyTorch and TensorFlow. However, the difficulties were quite too high, and we decided to only focus on PyTorch since nowadays is the most used framework in research. As far as I know, no project integrates into the same codebase both TensorFlow and PyTorch, let alone all the other frameworks (Jax, mxnet). The idea of providing Avalanche also for other frameworks is one of our "big desiderata" right now, but maintain a parallel version of Avalanche for another framework could be very difficult since all the versions should always be aligned (we should provide the same functionalities in both the framework), so the work to do is at least twice the work required now.
You raised 4 very interesting points, I'll try to give you my view on them:

• Do we think there's a chance some subset of users wouldn't want to use Avalanche because of the backend, or not wish to port their existing code?
  Yes, there is the risk but it's a calculated risk. Every person has his/her favorite framework, and many times people are reluctant to change their habits. We are aware that some people would not use Avalanche because they work with TensorFlow/Keras/Jax... and don't want to switch all their work to another framework, but we can do anything about it. As I said before, a multi-framework tool is too difficult to design and develop, and more versions of the same tool for different frameworks require a lot of work and effort, and we cannot afford it now.

• Aside from the huge developmental effort, what advantages/disadvantages might there be to adding additional backends, or even creating 'Avalanche XXX edition' in the future?
  As I said before, having more than one 'Avalanche xyz edition' requires a lot of additional work, not only to design and develop the new version for the new framework, but also for maintaining all the versions aligned and with the same functionalities. Moreover, every time we find a bug, we'll have to control every other Avalanche edition and fix the bug in every different one of them, slowing down the process especially because different framework may have different ways to do the same things, so a fix for one framework could not be used in another framework. I would personally exclude the possibility of additional backends. As far as I know, no project has both PyTorch and TensorFlow as dependencies, and I think is easier to develop Avalanche-TF than inserting TF in the current version of Avalanche.

• Might future deep learning frameworks outpace PyTorch, potentially deprecating the codebase in the future?
  Yes, but I don't think this will happen soon. Pytorch is pretty solid and has a big community that uses it, Is currently the most used framework in research, and has a quite huge ecosystem of different projects (like Avalanche) for almost everything. Of course, there is a greater than zero risk that PyTorch will be outdated in the future, but we would have the same risk with every other framework.

• Or is sticking with what we have fine for the foreseeable future.
  IMO we should think about how to expand Avalanche to work with other frameworks, but as you said I don't think is a priority now. We are still in the alpha stage, I think that now we should focus all our efforts on the beta and the official release of Avalanche 1.0. We should discuss these issues (thank you again for starting the conversation), but I would not put any physical effort into them yet. I think a good starting point to work on another 'edition' of Avalanche will be the release of the first stable release (Avalanche 1.0).

I agree with you that the design choices that we make now will influence the future of Avalanche, so we definitely should take into account also the possibility to support more than one deep learning framework.