Similar project discussion: git_cdn

[tardyp]

Hello,

I wanted to say hi and discuss a project that my team has been working on at Renault in a similar timeframe

https://gitlab.com/grouperenault/git_cdn

our design looks very similar, and it makes sense to share experience.

git_cdn is written in python asyncio, and we did not experience much performance bottleneck with the python.
Most of our load comes from the git upload-pack process.

In order to mitigate this load from upload-pack, we had to implement caching of the bigger upload-packs.
This is because our CI users are doing a lot of clone from scratch for reasons that are hard to negociate..

our design also implements cache of LFS files, which can be handy.

We do not support protocol v2 yet, and to my understanding goblet only supports v2.

We would be glad to have a discussions with you either in this issue or in a live hangout session.

[tparker]

Hi,

First of all, nice to (virtually) meet you!

We wrote Goblet so that our customers could deploy it to manage scaling issues with their own resources. On the server side, we have been pushing adoption of protocol changes like https://git-scm.com/docs/partial-clone and https://git-scm.com/docs/packfile-uri to continue scaling Git.

To our knowledge none of our clients has deployed Goblet, they have found other ways to reduce their load and/or take advantage of our server improvements. So we don't have insight on problems that people have had deploying it.

That said, it is always fun to get together and compare notes. I've missed this year since GitMerge didn't happen. Hopefully again in 2022! If you have any specific questions, you can email me directly.