Merge subdirectory from another git repository

Let’s say I have two repositories: cvhci-standards and labeltracks.

cvhci-standards

matlab/
...
python/
__init__.py
cvhcistandards.py
cpp/
...
README

labeltracks

labeltracks.py
utilities/
...

I’d like to merge the content of subdirectory python from cvhci-standards into labelstracks/utilities/cvhcistandards. To do this, we use git read-tree, with the following steps:

1. Add remote repository and fetch it:

git remote add -f cvhci-standards git://git.kit.edu:cvhci-standards.git

2. Merge, but don’t commit just yet:

git merge -s ours --no-commit cvhci-standards/master

3. Read a specific subdirectory from cvhci-standards. We specify the the target subdirectory as prefix, and the source-subdirectory (python) after the colon in the tree-ish repository description.

git read-tree --prefix=utilities/cvhcistandards/ -u cvhci-standards/master:python

4. Commit

git commit -m 'merging in cvhci-standards python module'

Later:
5. Pull from cvhci-standards to update the subtree.

git pull -s subtree -X subtree=utilities/cvhcistandards cvhci-standards master

We need to specify the subdirectory to merge to, otherwise git can’t match the subtrees correctly.

Advertisements

8 Responses to Merge subdirectory from another git repository

  1. craig says:

    Tried following your recipe but at the later step I’m finding that git does not match the subtrees correctly.
    Instead, by analogy to your example, its creating trackertools/cvhcistandards/python for the changed files instead of merging the remote python directory into trackertools/cvhcistandards.
    Is there anything more i can do to help git know how to do this?

    • baeuml says:

      Hi Craig, thanks for your comment. I just realized that there is a mistake in the last step, it should of course read (trackertools –> utilities, will fix it in the post):

      git pull -s subtree -X subtree=utilities/cvhcistandards cvhcistandards master

      However, that shouldn’t be the source of your problem. It probably went wrong before while reading the tree the first time. You can check whether setting up the subdirectory with read-tree worked by traversing the git internal tree structure with

      git cat-file -p master^{tree}

      and then replacing the “master” subsequently by the object hashes of the relevant subdirectories. At some point, you should find something like

      # git cat-file -p e2a1f63bc0b841deaf11b0dc71a0646d4f20e9be^{tree}
      040000 tree 7a5baa91e4a895135441fd4b64dd43f7d2377738 cvhcistandards

      If you start traversing from the *remote* master instead, you should find something like

      # git cat-file -p remotes/cvhci-standards/masters^{tree}
      040000 tree 7a5baa91e4a895135441fd4b64dd43f7d2377738 python

      Notice, that the SHA-1 hash is the same (7a5baa…)! That means that the directory entry cvhcistandards in our new repository just points to the tree object of the pulled-in subdirectory from the other project. If that is not the case for you, then the initial set-up did not work.

      Of course, after running step 5 (git pull -s subtree …) the hashes will be updated, but still both “directory entries” should always have the same hash.

      Hope that helps…

      • craig says:

        hi beauml,
        just have couple mins now.
        but yes this is very helpful!
        i can see that if the cat-files match up then yes i can subdirectory to subdirectory merge.
        note i also use –squash at the initial and later commits but i verified it makes no difference.
        so yes presumably i am doing something wrong on occasion.
        meantime i have to study more the remote repo (which had to be recreated too) to try to see if the initial read-tree or the recreate led to the mismatch.

        but my sincere thanks on such informative and quick reply
        craig

  2. craig says:

    hi again baeuml,
    ok so, after my last comment which i then had to leave a starbucks in haste before trying some more stuff, i now have experimented with changing the same file on both sides under the respective subdirectories. One remote side I added line at end of tile, while on near side i added a line at top of file.
    After each side commits, the two subdirectory hashes will be different as expected prior to the pull step — intermediate lines stayed the same.
    At this point after the pull step I start seeing interesting conflict whereby git chose not to keep the file in conflicted state but make a new file on the near side not where you’d want it.
    Using your subdirectories it would be like this if say “some_file” was the single differently changed file on both sides.

    CONFLICT (delete/modify): utilities/cvhcistandards/python/ deleted in HEAD and modified in 78ae7b88a80b41d2558ac103721b514b22788026. Version 78ae7b88a80b41d2558ac103721b514b22788026 of utilities/cvhcistandards/python/ left in tree.
    Squash commit — not updating HEAD
    Automatic merge failed; fix conflicts and then commit the result

    Note that the utilities/cvhcistandards/ also remains unchanged.
    So two files result here but gets somehow reported as a CONFLICT — not as one would have hoped.

    Maybe too much to ask but kind of hoping subtree merge of subdirectories rather like normal merge.
    Need to explore this a bit more.

    your take on what to expect as the near side changes and then re-pulls in changes from the remote.

    cheers

    Craig

    • craig says:

      slight correction as CONFLICT was meant to say

      CONFLICT (delete/modify): utilities/cvhcistandards/python/some-file deleted in HEAD and modified in 78ae7b88a80b41d2558ac103721b514b22788026. Version 78ae7b88a80b41d2558ac103721b514b22788026 of utilities/cvhcistandards/python/some-file left in tree —

      craig

      • baeuml says:

        I don’t think it is possible to make changes in a subtree and then merge from the original repository. Once you make a change anywhere in the subtree, you effectively break the link to the original tree. Technically, git should be able to figure out that both came from the same ancestor but I am not sure where git subtree actually implements this. I guess this is something you should rather ask at a git mailing list đŸ™‚

  3. php curl ssl says:

    i got the same conflict… any solution?

  4. For some reason instructions on merging a part of a different projects are very rare to find. I encountered your post and found it very useful to do my own merge. Thank you for sharing this information!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: