git-subrepo - Git Submodule Alternative
git subrepo -h # Help Overview
git subrepo clone <remote-url> [<subdir>]
git subrepo init <subdir>
git subrepo pull <subdir>
git subrepo push <subdir>
git subrepo fetch <subdir>
git subrepo branch <subdir>
git subrepo commit <subdir>
git subrepo config <subdir>
git subrepo status [<subdir>]
git subrepo clean <subdir>
git subrepo help [<command> | --all]
git subrepo version
git subrepo upgrade
This git command "clones" an external git repo into a subdirectory of your repo. Later on, upstream changes can be pulled in, and local changes can be pushed back. Simple.
This command is an improvement from git-submodule
and git-subtree
; two other git commands with similar goals, but various problems.
It assumes there are 3 main roles of people interacting with a repo, and attempts to serve them all well:
owner - The person who authors/owns/maintains a repo.
users - People who are just using/installing the repo.
collaborators - People who commit code to the repo and subrepos.
The git-subrepo
command benefits these roles in the following ways:
Simple and intuitive commandline usage (with tab completion).
Users get your repo and all your subrepos just by cloning your repo.
Users do not need to install
git-subrepo
, ever.Collaborators do not need to install unless they want to push/pull.
Collaborators know when a subdir is a subrepo (it has a
.gitrepo
file).The
.gitrepo
file never gets pushed back to the subrepo upstream.Well named branches and remotes are generated for manual operations.
Owners do not deal with the complications of keeping submodules in sync.
Subrepo repositories can contain subrepos themselves.
Branching with subrepos JustWorksâ„¢.
Different branches can have different subrepos in different states, etc.
Moving/renaming/deleting a subrepo subdir JustWorksâ„¢.
You can
init
an existing subdirectory into a subrepo.Your git history is kept squeaky clean.
Upstream history (clone/pull) is condensed into a single commit.
Pulls can use a
merge
,rebase
orforce
strategies.You can see the subrepo history with
git log subrepo/<subdir>/fetch
.Commits pushed back upstream are not condensed (by default).
Trivial to try any subrepo operations and then reset back.
No configuration required.
Does not introduce history that messes up other git commands.
Fixes known rebase failures with
git-subtree
.
The best short answer is:
git clone https://github.com/ingydotnet/git-subrepo /path/to/git-subrepo
echo 'source /path/to/git-subrepo/.rc' >> ~/.bashrc
The complete "Installation Instructions" can be found below.
Note: git-subrepo needs a git version (> 2.7) that supports worktree:s.
All the subrepo commands use names of actual Git commands and try to do operations that are similar to their Git counterparts. They also attempt to give similar output in an attempt to make the subrepo usage intuitive to experienced Git users.
Please note that the commands are not exact equivalents, and do not take all the same arguments. Keep reading…
git subrepo clone <repository> [<subdir>] [-b <branch>] [-f] [-m <msg>] [--file=<msg file>] [-e] [--method <merge|rebase>]
-
Add a repository as a subrepo in a subdir of your repository.
This is similar in feel to
git clone
. You just specify the remote repo url, and optionally a sub-directory and/or branch name. The repo will be fetched and merged into the subdir.The subrepo history is squashed into a single commit that contains the reference information. This information is also stored in a special file called
<subdir>/.gitrepo
. The presence of this file indicates that the directory is a subrepo.All subsequent commands refer to the subrepo by the name of the subdir. From the subdir, all the current information about the subrepo can be obtained.
The
--force
option will "reclone" (completely replace) an existing subdir.The
--method
option will decide how the join process between branches are performed. The default option is merge.The
clone
command accepts the--branch=
--edit
,--file
,--force
and--message=
options. git subrepo init <subdir> [-r <remote>] [-b <branch>] [--method <merge|rebase>]
-
Turn an existing subdirectory into a subrepo.
If you want to expose a subdirectory of your project as a published subrepo, this command will do that. It will split out the content of a normal subdirectory into a branch and start tracking it as a subrepo. Afterwards your original repo will look exactly the same except that there will be a
<<subdir
/.gitrepo >> file.If you specify the
--remote
(and optionally the--branch
) option, the values will be added to the<subdir>/.gitrepo
file. The--remote
option is the upstream URL, and the--branch
option is the upstream branch to push to. These values will be needed to do agit subrepo push
command, but they can be provided later on thepush
command (and saved to<<subdir
/.gitrepo >> if you also specify the--update
option).Note: You will need to create the empty upstream repo and push to it on your own, using
git subrepo push <subdir>
.The
--method
option will decide how the join process between branches are performed. The default option is merge.The
init
command accepts the--branch=
and--remote=
options. git subrepo pull <subdir>|--all [-M|-R|-f] [-m <msg>] [--file=<msg file>] [-e] [-b <branch>] [-r <remote>] [-u]
-
Update the subrepo subdir with the latest upstream changes.
The
pull
command fetches the latest content from the remote branch pointed to by the subrepo's.gitrepo
file, and then tries to merge the changes into the corresponding subdir. It does this by making a branch of the local commits to the subdir and then merging or rebasing (see below) it with the fetched upstream content. After the merge, the content of the new branch replaces your subdir, the.gitrepo
file is updated and a single 'pull' commit is added to your mainline history.The
pull
command will attempt to do the following commands in one go:git subrepo fetch <subdir> git subrepo branch <subdir> git merge/rebase subrepo/<subdir>/fetch subrepo/<subdir> git subrepo commit <subdir> # Only needed for a consequential push: git update-ref refs/subrepo/<subdir>/pull subrepo/<subdir>
In other words, you could do all the above commands yourself, for the same effect. If any of the commands fail, subrepo will stop and tell you to finish this by hand. Generally a failure would be in the merge or rebase part, where conflicts can happen. Since Git has lots of ways to resolve conflicts to your personal tastes, the subrepo command defers to letting you do this by hand.
When pulling new data, the method selected in clone/init is used. This has no effect on the final result of the pull, since it becomes a single commit. But it does affect the resulting
subrepo/<subdir>
branch, which is often used for a subrepopush
command. See 'push' below for more information. If you want to change the method you can use theconfig
command for this.When you pull you can assume a fast-forward strategy (default) or you can specify a
--rebase
,--merge
or--force
strategy. The latter is the same as aclone --force
operation, using the current remote and branch.Like the
clone
command,pull
will squash all the changes (since the last pull or clone) into one commit. This keeps your mainline history nice and clean. You can easily see the subrepo's history with thegit log
command:git log refs/subrepo/<subdir>/fetch
The set of commands used above are described in detail below.
The
pull
command accepts the--all
,--branch=
,--edit
,--file
,--force
,--message=
,--remote=
and--update
options. git subrepo push <subdir>|--all [<branch>] [-m msg] [--file=<msg file>] [-r <remote>] [-b <branch>] [-M|-R] [-u] [-f] [-s] [-N]
-
Push a properly merged subrepo branch back upstream.
This command takes the subrepo branch from a successful pull command and pushes the history back to its designated remote and branch. You can also use the
branch
command and merge things yourself before pushing if you want to (although that is probably a rare use case).The
push
command requires a branch that has been properly merged/rebased with the upstream HEAD (unless the upstream HEAD is empty, which is common when doing a firstpush
after aninit
). That means the upstream HEAD is one of the commits in the branch.By default the branch ref
refs/subrepo/<subdir>/pull
will be pushed, but you can specify a (properly merged) branch to push.After that, the
push
command just checks that the branch contains the upstream HEAD and then pushes it upstream.The
--force
option will do a force push. Force pushes are typically discouraged. Only use this option if you fully understand it. (The--force
option will NOT check for a proper merge. ANY branch will be force pushed!)The
push
command accepts the--all
,--branch=
,--dry-run
,--file
,--force
,--merge
,--message
,--rebase
,--remote=
,--squash
and--update
options. git subrepo fetch <subdir>|--force --all [-r <remote>] [-b <branch>]
-
Fetch the remote/upstream content for a subrepo.
It will create a Git reference called
subrepo/<subdir>/fetch
that points at the same commit asFETCH_HEAD
. It will also create a remote calledsubrepo/<subdir>
. These are temporary and you can easily remove them with the subrepoclean
command.The
fetch
command accepts the--force
,--all
,--branch=
and--remote=
options. git subrepo branch <subdir>|--all [-f] [-F]
-
Create a branch with local subrepo commits.
Scan the history of the mainline for all the commits that affect the
subdir
and create a new branch from them calledsubrepo/<subdir>
.This is useful for doing
pull
andpush
commands by hand.Use the
--force
option to write over an existingsubrepo/<subdir>
branch.The
branch
command accepts the--all
,--fetch
and--force
options. git subrepo commit <subdir> [<subrepo-ref>] [-m <msg>] [--file=<msg file>] [-e] [-f] [-F]
-
Add subrepo branch to current history as a single commit.
This command is generally used after a hand-merge. You have done a
subrepobranch
and merged (rebased) it with the upstream. This command takes the HEAD of that branch, puts its content into the subrepo subdir and adds a new commit for it to the top of your mainline history.This command requires that the upstream HEAD be in the
subrepo/<subdir>
branch history. That way the same branch can push upstream. Use the--force
option to commit anyway.The
commit
command accepts the--edit
,--fetch
,--file
,--force
and--message=
options. git subrepo status [<subdir>|--all|--ALL] [-F] [-q|-v]
-
Get the status of a subrepo. Uses the
--all
option by default. If the--quiet
flag is used, just print the subrepo names, one per line.The
--verbose
option will show all the recent local and upstream commits.Use
--ALL
to show the subrepos of the subrepos (ie the "subsubrepos"), if any.The
status
command accepts the--all
,--ALL
,--fetch
,--quiet
and--verbose
options. git subrepo clean <subdir>|--all|--ALL [-f]
-
Remove artifacts created by
fetch
andbranch
commands.The
fetch
andbranch
operations (and other commands that call them) create temporary things like refs, branches and remotes. This command removes all those things.Use
--force
to remove refs. Refs are not removed by default because they are sometimes needed between commands.Use
--all
to clean up after all the current subrepos. Sometimes you might change to a branch where a subrepo doesn't exist, and then--all
won't find it. Use--ALL
to remove any artifacts that were ever created by subrepo.To remove ALL subrepo artifacts:
git subrepo clean --ALL --force
The
clean
command accepts the--all
,--ALL
, and--force
options. git subrepo config <subdir> <option> [<value>] [-f]
-
Read or update configuration values in the subdir/.gitrepo file.
Because most of the values stored in the .gitrepo file are generated you will need to use
--force
if you want to change anything else then themethod
option.Example to update the
method
option for a subrepo:git subrepo config foo method rebase
git subrepo help [<command>|--all]
-
Same as
git help subrepo
. Will launch the manpage. For the shorter usage, usegit subrepo -h
.Use
git subrepo help <command>
to get help for a specific command. Use--all
to get a summary of all commands.The
help
command accepts the--all
option. git subrepo version [-q|-v]
-
This command will display version information about git-subrepo and its environment. For just the version number, use
git subrepo --version
. Use--verbose
for more version info, and--quiet
for less.The
version
command accepts the--quiet
and--verbose
options. git subrepo upgrade
-
Upgrade the
git-subrepo
software itself. This simply does agit pull
on the git repository that the code is running from. It only works if you are on themaster
branch. It won't work if you installedgit-subrepo
usingmake install
; in that case you'll need tomake install
from the latest code.
-h
-
Show a brief view of the commands and options.
--help
-
Gives an overview of the help options available for the subrepo command.
--version
-
Print the git-subrepo version. Just the version number. Try the
version
command for more version info. --all
(-a
)-
If you have multiple subrepos, issue the command to all of them (if applicable).
--ALL
(-A
)-
If you have subrepos that also have subrepos themselves, issue the command to ALL of them. Note that the
--ALL
option only works for a subset of the commands that--all
works for. --branch=<branch-name>
(-b <branch-name>
)-
Use a different upstream branch-name than the remote HEAD or the one saved in
.gitrepo
locally. --dry-run
(-N
)-
For the push command, do everything up until the push and then print out the actual
git push
command needed to finish the operation. --edit
(-e
)-
Edit the commit message before committing.
--fetch
(-F
)-
Use this option to fetch the upstream commits, before running the command.
--file=<commit msg file>
-
Supply your own commit message from a file
--force
(-f
)-
Use this option to force certain commands that fail in the general case.
NOTE: The
--force
option means different things for different commands. Read the command specific doc for the exact meaning. --merge
(-M
)-
Use a
merge
strategy to include upstream subrepo commits on a pull (or setup for push). --message=<message>
(-m <message>
)-
Specify your own commit message on the command line.
--rebase
(-R
)-
Use a
rebase
strategy to include upstream subrepo commits on a pull (or setup for push). --remote=<remote-url>
(-r <remote-url>
)-
Use a different remote-url than the one saved in
.gitrepo
locally. --squash
(-s
)-
Squash all commits on a push into one new commit.
--update
(-u
)-
If
--branch
or--remote
are used, and the command updates the.gitrepo
file, include these values to the update.
--quiet
(-q
)-
Print as little info as possible. Applicable to most commands.
--verbose
(-v
)-
Print more information about the command execution and results. Applicable to most commands.
--debug
(-d
)-
Show the actual git (and other) commands being executed under the hood. Applicable to most commands.
--DEBUG
(-x
)-
Use the Bash
set -x
option which prints every command before it is run. VERY noisy, but extremely useful in deep debugging. Applicable to all commands.
The git-subrepo
command exports and honors some environment variables:
GIT_SUBREPO_ROOT
-
This is set by the
.rc
file, if you use that method to install / enablegit-subrepo
. It contains the path of thegit-subrepo
repository. GIT_SUBREPO_RUNNING
-
This variable is exported when
git-subrepo
is running. It is set to the pid of thegit-subrepo
process that is running. Other processes, like git hooks for instance, can use this information to adjust accordingly. GIT_SUBREPO_COMMAND
-
This variable is exported when
git-subrepo
is running. It is set to the name of thegit-subrepo
subcommand that is running. GIT_SUBREPO_PAGER
-
Use this to specify the pager to use for long output commands. Defaults to
$PAGER
orless
. GIT_SUBREPO_QUIET
-
Set this for quiet (
-q
) output. GIT_SUBREPO_VERBOSE
-
Set this for verbose (
-v
) output. GIT_SUBREPO_DEBUG
-
Set this for debugging (
-d
) output.
There are currently 3 ways to install git-subrepo
. For all of them you need to get the source code from GitHub:
git clone https://github.com/ingydotnet/git-subrepo /path/to/git-subrepo
The first installation method is preferred: source
the .rc
file. Just add a line like this one to your shell startup script:
source /path/to/git-subrepo/.rc
That will modify your PATH
and MANPATH
, and also enable command completion.
The second method is to do these things by hand. This might afford you more control of your shell environment. Simply add the lib
and man
directories to your PATH
and MANPATH
:
export GIT_SUBREPO_ROOT="/path/to/git-subrepo"
export PATH="/path/to/git-subrepo/lib:$PATH"
export MANPATH="/path/to/git-subrepo/man:$MANPATH"
See below for info on how to turn on Command Completion.
The third method is a standard system install, which puts git-subrepo
next to your other git commands:
make install # Possibly with 'sudo'
This method does not account for upgrading and command completion yet.
This command is known to work in these Windows environments:
Git for Windows -- https://git-for-windows.github.io/
Babun -- http://babun.github.io/
Cygwin -- https://www.cygwin.com/
Let us know if there are others that it works (or doesn't work) in.
The git-subrepo
repository comes with a extensive test suite. You can run it with:
make test
or if you don't have make
on your system:
prove -v test
If you used the .rc
or PATH
method of installation, just run this to upgrade git-subrepo
:
git subrepo upgrade
Or (same thing):
cd /path/to/git-subrepo
git pull
If you used make install
method, then run this again (after git pull
):
make install # Possibly with 'sudo'
The git subrepo
command supports <TAB>
-based command completion. If you don't use the .rc
script (see Installation, above), you'll need to enable this manually to use it.
If your Bash setup does not already provide command completion for Git, you'll need to enable that first:
source <Git completion script>
On your system, the Git completion script might be found at any of the following locations (or somewhere else that we don't know about):
/etc/bash_completion.d/git
/usr/share/bash-completion/git
/usr/share/bash-completion/completions/git
/opt/local/share/bash-completion/completions/git
/usr/local/etc/bash_completion.d/git
~/.homebrew/etc/bash_completion.d/git
In case you can't find any of these, this repository contains a copy of the Git completion script:
source /path/to/git-subrepo/share/git-completion.bash
Once Git completion is enabled (whether you needed to do that manually or not), you can turn on git-subrepo
completion with a command like this:
source /path/to/git-subrepo/share/completion.bash
In the Z shell (zsh), you can manually enable git-subrepo
completion by adding the following line to your ~/.zshrc
, before the compinit
function is called:
fpath=('/path/to/git-subrepo/share/zsh-completion' $fpath)
The git-subrepo command has been used in production and seems to get the job done. Development is still ongoing but mostly just for fixing bugs.
Trying subrepo out is simple and painless (this is not git submodule
). Nothing is permanent (if you do not push to shared remotes). ie You can always play around and reset back to the beginning without pain.
This command has a test suite (run make test
), but surely has many bugs. If you have expertise with Git and subcommands, please review the code, and file issues on anything that seems wrong.
If you want to chat about the git-subrepo
command, join #gitcommands
on irc.freenode.net
.
Works on POSIX systems: Linux, BSD, OSX, etc.
Works on various Windows environments. See "Windows" section above.
The
git-subrepo
repo itself has 2 subrepos under theext/
subdirectory.Written in (very modern) Bash, with full test suite. Take a look.
A
.gitrepo
file never is in the top level dir (next to a.git/
dir).
Ingy döt Net <ingy@ingy.net>
Magnus Carlsson <grimmymail@gmail.com>
Austin Morgan <admorgan@morgancomputers.net>
The MIT License (MIT)
Copyright (c) 2013-2020 Ingy döt Net