It should be possible to save recipes to remotes

[MarcoRuzzon]

No description provided.

[MarcoRuzzon]

What about:
forest --push-recipes <url> <branch>
forest --set-push-recipes-url <url>

--push-recipes would:

1. check if url repo exists/user has access rights
2. check if branch exists
3. if <branch> doesn't exists:

• clone repo (without history for speed) in tmp folder
• add an orphan branch named <branch>
• copy recipes folder
• push the new branch

4. if <branch> exists

• give feedback ti user and ask if he wants to update the recipes
• if yes ask for commit message, must be not empty
• clone repo (without history for speed) in tmp folder
• substitute the recipes in the repo with the local ones
• commit the change
• push the branch

[alaurenzi]

Maybe we can keep this simpler.. how about using symlinks?

My proposal is:

• upon forest --add-recipes we clone the recipe repository in src/ (or in a different folder maybe, but I like the recipe-is-source-code idea) and symlink all the yaml files to the usual recipes/ folder
• the user edits a recipe, and by doing so he/she actually changes the content of the original file
• the user can just cd src/<recipe_pkg_name>, and use the standard git workflow that he or she is already used to

I see the clear benefit the no additional workflow has to be learned

What do you think? Do you see any shortcoming?

[MarcoRuzzon]

Nice idea, just one thing. What would be the workflow if someone wants to write a new recipe and not simply edit one?

He would need to get inside the git repo, add the recipe and do the symlink himself ?

[alaurenzi]

Maybe through the forest -u option?

[MarcoRuzzon]

The -u option now would be used to symlink the recipes in the local repository/ies. In case of conflicts, recipes with the same name in different repositories, the user would be prompted to choose.

[alaurenzi]

Proposal is the following:

• forest --add-recipes git@github.com:ciao/miao.git [master [foldername [recipe1 recipe2 ...]]] does the following
  • clone repository (with given branch) into recipes/src (todo review name)
  • symlink all .yaml files from recipes/src/miao/foldername to recipes
  • handle conflicts, as follows
    • in default mode, conflict is an error
    • with --allow-recipe-overwrite, last file is used and an informative message is printed
• forest --update recipes is deprecated in favor of the well known git flow
• .recipes is deprecated

[MarcoRuzzon]

How to deal with same repo different branches

Technically it is possible to add recipes for the same repo but different branches, since the recipes are simlinks to their sources it means we must have multiple source folders for the same repo.

How to organize these folders?
Using the name of the branches to create subfolders or modifying the name of the repo folders could lead to confusion. Especially, if it is possible to change branches. One idea could be to prevent switching branches in recipes source folders. We could use git clone url -b branch --single-branch as described here https://stackoverflow.com/questions/1778088/how-do-i-clone-a-single-branch-in-git/9920956#9920956 making it impossible to switch branches.

[MarcoRuzzon]

What if a branch of a recipes' repository has already been cloned and forest --add-recipes url branch folder recipe1 recipe2 is called?

Forest should warn the repo is already present locally, and an eventual pull should be performed by the user if necessary. This command will simply symlink the recipes.

[MarcoRuzzon]

It would be useful to have a forest --status to show:

• recipes' symlinks
• which recipes' sources have uncommitted/unpushed modifications
• which sources have uncommitted/unpushed modifications

[MarcoRuzzon]

@alaurenzi this commit 2ac69bd contains some initial work and also the subparsers I was telling you about. For now, recipes' sources folder management is as suggested here #31 (comment)

[EdoardoRomiti]

Some feedback after using this workflow for a little bit.
Having multiple folders named as the branches which are git repos but don't allow to switch branch (as described in #31 (comment)) create a bit of confusion in my opinion.

From the "branch-specific" folder you can push/pull, but cannot merge any other branch (because you are not allowed to see them). This kind of breaks how git works and makes synchronizing with other branches quite cumbersome.
It forces the user to have yet another clone of the recipes' repo where all branches can actually be accessed and where merge operations can be done.

Is it really necessary to deal with "same repo different branches"? I think it's super useful to have the possibility to access recipes from multiple repos by using the symlinks, but for recipes from the same repo wouldn't it be better to just create a new branch?
If there is no need to handle different branches at the same time, the src folder could just contain a clone of the recipes' repo, This allows to use the standard git workflow on these folders (src/<recipe_pkg_name>).

@MarcoRuzzon @alaurenzi what do you think? Would this work or would it break something else? Am i missing something?

[alaurenzi]

I tend to agree

[MarcoRuzzon]

It makes sense, this workflow was intended to support the use of multiple branches from the same repo. As @EdoardoRomiti suggested, it can be misleading at times and should be avoided. I will fix it.

[MarcoRuzzon]

Add recipes from remote

usage: forest add-recipes [-h] [--tag TAG] [--subdir-path SUBDIR_PATH]
                         [--recipes RECIPES [RECIPES ...]]
                         [--allow-overwrite] [--verbose][
                         url

positional arguments:
 url                   url of the remote (e.g.
                       git@github.com:<username>/<reponame>.git)
                  

optional arguments:
 -h, --help            show this help message and exit
 --tag TAG, -t TAG
 --subdir-path SUBDIR_PATH, -s SUBDIR_PATH
                       relative path to the folder in which recipes are
                       contained
 --recipes RECIPES [RECIPES ...], -r RECIPES [RECIPES ...]
                       specify which recipes to add, otherwise all recipes in
                       subdir-path are added
 --allow-overwrite, -o
                       allow overwritng local recipes with new ones
 --verbose, -v         print additional information

Suggested new workflow:

• check if repo already in recipe/src -> give feedback
  • if not: clone repository into recipes/src -> give feedback
• symlink all .yaml files of specified recipes from recipes/src/repo/sub_dir_path to recipes -> give feedback
• handle conflicts, as follows
  • in default mode, choose interactively which source to keep
  • with --allow-recipe-overwrite, last file is used and an informative message is printed

Update recipes using local repositories

usage: forest update-recipes [-h] [--subdir-path SUBDIR_PATH]
                         [--recipes RECIPES [RECIPES ...]]
                         [--verbose] [--fix-broken]
                         reponame
positional argumetns:
 reponame              name of the repo  


optional arguments:
 -h, --help            show this help message and exit

 --subdir-path SUBDIR_PATH, -s SUBDIR_PATH
                       relative path to the folder in which recipes are
                       contained
 --recipes RECIPES [RECIPES ...], -r RECIPES [RECIPES ...]
                       specify which recipes to update, otherwise all recipes 
                       in <reponame>/subdir-path are updated         
 --fix-broken          remove empty symlinks
 --verbose, -v         print additional information

Suggested workflow:

• check if reponame in recipe/src -> give feedback
  • if not: abort
• symlink all .yaml files of specified recipes from recipes/src/repo/sub_dir_path to recipes -> give feedback

What happens when branch is checked out?

symlinks can break, use forest fix-recipes to remove empty symlinks

[MarcoRuzzon]

To simplify even more I suggest supporting only one recipes' repository. This repo will be cloned directly in the recipes folder. Modifying recipes and pushing/pulling them to/from remote will be done using only the well know git workflow.

[MarcoRuzzon]

FYI @liesrock

[alaurenzi]

This can be ok as temporary solution, but I think we should try to match the behavior of apt in the long run, and figure out something simple to handle collisions