SubRepo support?

Gary's Avatar

Gary

24 May, 2017 01:03 PM

Are there any plans to add sub-repo support? I'm moving from Kiln, which handled this nicely.
Most of my projects share smaller sub-repos, and not being able to manage them as such is causing huge headaches - I'll push an update and then have to run through a pull for countless instances of the same repository across all my projects.
Alas, a package manager isn't really an option here.

  1. 1 Posted by stephen.domingu... on 12 Jun, 2017 01:58 PM

    stephen.dominguez's Avatar

    Curious to see / hear / know what the resolution of this question was...

  2. 2 Posted by Gary Stanton on 12 Jun, 2017 02:00 PM

    Gary Stanton's Avatar

    No response from support.

  3. 3 Posted by stephen.domingu... on 12 Jun, 2017 02:01 PM

    stephen.dominguez's Avatar

    Have you wound up settling on a different platform for your Hg hosting needs?

  4. 4 Posted by Gary Stanton on 12 Jun, 2017 02:04 PM

    Gary Stanton's Avatar

    Not yet... hglab is great (when it works - I've had issues) - but if I
    come across a similar offering with subrepo support then I will
    certainly switch.

  5. 5 Posted by stephen.domingu... on 12 Jun, 2017 02:09 PM

    stephen.dominguez's Avatar

    Why did you leave Kiln?

    ______ ______ ______

    Steve Dominguez
    Team Lead – Seismic Interpretation Software
    P +01 832 351 5626
    M +01 303 570 1581

  6. 6 Posted by Gary Stanton on 12 Jun, 2017 02:27 PM

    Gary Stanton's Avatar

    I'd been meaning to move to a self hosted solution for a while. Kiln do
    a self hosted option, but it's pricey.
    Also, Kiln works best when integrated with FogBugz, and I never really
    made much use of that.
    It's a good platform though - Can't fault it... and their support has
    always been excellent.

  7. 7 Posted by stephen.domingu... on 12 Jun, 2017 02:34 PM

    stephen.dominguez's Avatar

    Ha – yes, I can’t even seem to find my way past the FogBugz stuff, and I know from experience we’ll never get a full and proper ROI at that price level with the entire suite of offerings there…

    We’re definitely locked to a self-hosted approach; and are getting by with just using SSH access.

    HgLabs seems about perfect, but the subrepo thing is going to be a problem :(

    ______ ______ ______

    Steve Dominguez
    Team Lead – Seismic Interpretation Software
    P +01 832 351 5626
    M +01 303 570 1581

  8. 8 Posted by Gary Stanton on 12 Jun, 2017 02:39 PM

    Gary Stanton's Avatar

    You can still have a repo within a repo. If you remove the .hgsub files,
    you can just manage each repo separately.

    It's annoying having to manually update each subrepo, but it'll work ok.

  9. 9 Posted by stephen.domingu... on 12 Jun, 2017 02:45 PM

    stephen.dominguez's Avatar

    We have a “buildtools” repository in Subversion that is basically a collection of company-wide CMake utilities and configuration files – a common ‘bootstrapping’ process for all projects to follow.

    It’s in SVN because we still have a handful of development teams running their projects in SVN, *and* because it contains the CMake binaries (mostly the former reason, but the latter was also a factor)…

    It’s certainly possible to just tell everyone to clone their Mercurial repo, then do an svn checkout of the ‘buildtools’ project at the right revision into the proper spot; heck I could even write a little .bat/.sh script to do that and check it in…. but that’s pretty much what .hgsub is doing anyway! Seems odd to have to reinvent the wheel there…

    ______ ______ ______

    Steve Dominguez
    Team Lead – Seismic Interpretation Software
    P +01 832 351 5626
    M +01 303 570 1581

  10. 10 Posted by Gary Stanton on 12 Jun, 2017 02:56 PM

    Gary Stanton's Avatar

    You have an SVN repo as a subrepo to a Mercurial one? Didn't realise you
    could do that... but then, SVN, y'know... ;)

    In my case I'm basically using the subrepo functionality as a package
    manager. So my apps are broken down into smaller modules that are reused
    when needed and each is a subrepo of whatever project needs it. The
    problem is that if I need to update one of those modules, it needs to be
    updated in every project that uses it. With subrepo functionality this
    happens automatically on the next pull request, but now I've had to
    separate them out it's a very manual process.

    I might think about the scripting option though, that's not a bad idea.

  11. 11 Posted by stephen.domingu... on 12 Jun, 2017 03:30 PM

    stephen.dominguez's Avatar

    *start offtopic* (subrepo approach explained below)

    Yeah – I wrote a handful of CMake things based on “cpm” (cmake package management) to handle that sort of automation as part of the build system itself.

    CPM can fairly nicely handle Git/Mercurial project dependencies all by itself, and if memory serves, SVN dependencies as well – sort of a really lightweight Maven or Ivy configuration written entirely in CMake rather than Java.

    I extended the concept a bit into just executing some basic smbget / windows file copies so as to automatically fetch 3rd party library dependencies as well, given a company-wide common naming convention and network storage location for archives like ‘Boost-1.60.1-win64.zip’, etc.

    Entire project really only took a bit of clever cmake scripting, and now an enterprise-wide globally distributed group of developers working on (depends on how you count) well over 30 different code projects can all sit down in front of any corporate dev machine on the planet, check out source code (from one of like 16 different repo servers – hence my interest in HgLab) and just type “cmake” – everything else is automated.

    *end offtopic – sorry*

    I theorized that perhaps I was just pushing too much too fast, trying to cram down an entire subrepo tree all at once. So I started “from scratch” as if I was just now adding my very first subrepo and created the .hgsub file with appropriate repository reference and tried to commit just that. Turns out this file cannot be ‘empty’ since the commit of .hgsub is a special hook within mercurial that in turn attempts to validate the existence of the expected local content and create the .hgsubstate file. Furthermore, the error messages that come out of mistakes in this sort of thing are truly horrible at worst and misleading at best.

    1) Create your .hgsub file with the proper syntax for the “new” subrepo

    a. (in my case, this was “buildtools = [svn]http://<hidingMyDetails>/buildtools<http://%3chidingMyDetails%3e/buildtools>”)

    2) Do NOT create / copy / import your .hgsubstate file

    3) Refer to your hgsubstate file from your original repository structure and manually checkout / clone / update your subrepo to the RIGHT VERSION

    a. So, my buildtools is supposed to be at rev37, so I manually did an svn checkout to rev37 in the proper spot in my mercurial repository

    4) Hg commit

    5) Hg push

    a. I suspect here if one or more of these subrepositories is particularly “heavy” you might run afoul of some of those transactional limits you’ve already found – but it sounded like you found workarounds for those too

    6) All is well.

    Rinse and repeat for your other subrepositories…

    ______ ______ ______

    Steve Dominguez
    Team Lead – Seismic Interpretation Software
    P +01 832 351 5626
    M +01 303 570 1581

  12. 12 Posted by stephen.domingu... on 13 Jun, 2017 01:53 PM

    stephen.dominguez's Avatar

    .... and as a final follow-up here, while this seems to "work" for command-line and even TortoiseHg-driven operations, if i now go to browse the webpage for the repository, it implodes with some bizarre error output on the screen... *sigh* I'll open a separate ticket on that issue however, since while it's related, it's a little more specific.

  13. 13 Posted by Gary Stanton on 13 Jun, 2017 01:56 PM

    Gary Stanton's Avatar

    I think the response you'll get there is that 'subrepos aren't supported'.
    All this time with zero response suggests there is little to no interest in
    adding the functionality for us. :(

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac

Recent Discussions

21 Sep, 2019 09:28 PM
21 Aug, 2019 09:07 AM
30 Jul, 2019 08:59 PM
31 May, 2019 02:29 PM
26 Nov, 2018 01:47 PM

 

16 Nov, 2018 07:51 PM
06 Nov, 2018 04:30 PM
24 Oct, 2018 07:57 AM
20 Jul, 2018 11:07 PM
05 Jul, 2018 10:53 AM
27 May, 2018 01:33 AM