We are in the middle of a migration from subversion to git, and we have some very large repositories.
It would greatly decrease time to clone and lower disk usage if we could configure a QB git repo to use the '--depth' option to clone.
Description
We are in the middle of a migration from subversion to git, and we have some very large repositories.
It would greatly decrease time to clone and lower disk usage if we could configure a QB git repo to use the '--depth' option to clone.
We are experiencing some problems with the shallow clone option for git repositories. After setting it to for example 10 we can observe the following git command running:
This is actually unexpected behaviour as it means that the option is not useful. We need to use a shallow clone since we have huge repositories and we don't need the full history of the repository.
Martin[11/Jun/20 08:29 AM]
We are experiencing some problems with the shallow clone option for git repositories. After setting it to for example 10 we can observe the following git command running:
{code}
ubuntu@ip-XX-XXX-XX-XX:~$ ps aux | grep intake
ubuntu 17186 0.0 0.0 13136 1000 pts/0 S+ 06:44 0:00 grep --color=auto intake
svc_qui+ 20851 0.0 0.0 19592 4260 ? S 06:38 0:00 git clone --no-checkout --depth 10 https://****:****@bitbucket.acme.org/scm/test/main.git /opt/buildagent/workspace/root/dev-pipeline/3-intake/parent_code -q
svc_qui+ 22541 0.8 0.0 35468 16504 ? S 06:39 0:02 git --shallow-file /opt/buildagent/workspace/root/dev-pipeline//3-intake/parent_code/.git/shallow.lock index-pack --stdin --fix-thin --keep=fetch-pack 7317 on quickbuild-agent-XX-XXX-XX-XX --pack_header=2,67859
{code}
and after some minutes a full fetch is invoked
{code}
ubuntu@ip-XX-XXX-XX-XX:/$ ps aux | grep git
svc_qui+ 12558 0.0 0.0 19036 3368 ? S 07:24 0:00 git fetch --all --tags -q
svc_qui+ 12559 0.0 0.0 22708 6084 ? S 07:24 0:00 git fetch --append --tags -q origin
svc_qui+ 12560 0.2 0.0 192036 30400 ? S 07:24 0:05 git-remote-https origin https://****:****@bitbucket.acme.org/scm/main.git
svc_qui+ 12831 0.0 0.0 747816 7100 ? Sl 07:24 0:01 git fetch-pack --stateless-rpc --stdin --lock-pack --thin --no-progress https://****:****@bitbucket.acme.org/scm/test/main.git/
svc_qui+ 17963 0.7 0.1 779548 43252 ? S 07:25 0:14 git index-pack --stdin --fix-thin --keep=fetch-pack 8875 on quickbuild-agent-XX-XXX-XX-XX --pack_header=2,1589619
ubuntu 22020 0.0 0.0 13136 968 pts/1 S+ 07:57 0:00 grep --color=auto git
{code}
This is actually unexpected behaviour as it means that the option is not useful. We need to use a shallow clone since we have huge repositories and we don't need the full history of the repository.
{code}
ubuntu@ip-XX-XXX-XX-XX:~$ ps aux | grep intake
ubuntu 17186 0.0 0.0 13136 1000 pts/0 S+ 06:44 0:00 grep --color=auto intake
svc_qui+ 20851 0.0 0.0 19592 4260 ? S 06:38 0:00 git clone --no-checkout --depth 10 https://****:****@bitbucket.acme.org/scm/test/main.git /opt/buildagent/workspace/root/dev-pipeline/3-intake/parent_code -q
svc_qui+ 22541 0.8 0.0 35468 16504 ? S 06:39 0:02 git --shallow-file /opt/buildagent/workspace/root/dev-pipeline//3-intake/parent_code/.git/shallow.lock index-pack --stdin --fix-thin --keep=fetch-pack 7317 on quickbuild-agent-XX-XXX-XX-XX --pack_header=2,67859
{code}
and after some minutes a full fetch is invoked
{code}
ubuntu@ip-XX-XXX-XX-XX:/$ ps aux | grep git
svc_qui+ 12558 0.0 0.0 19036 3368 ? S 07:24 0:00 git fetch --all --tags -q
svc_qui+ 12559 0.0 0.0 22708 6084 ? S 07:24 0:00 git fetch --append --tags -q origin
svc_qui+ 12560 0.2 0.0 192036 30400 ? S 07:24 0:05 git-remote-https origin https://****:****@bitbucket.acme.org/scm/main.git
svc_qui+ 12831 0.0 0.0 747816 7100 ? Sl 07:24 0:01 git fetch-pack --stateless-rpc --stdin --lock-pack --thin --no-progress https://****:****@bitbucket.acme.org/scm/test/main.git/
svc_qui+ 17963 0.7 0.1 779548 43252 ? S 07:25 0:14 git index-pack --stdin --fix-thin --keep=fetch-pack 8875 on quickbuild-agent-XX-XXX-XX-XX --pack_header=2,1589619
ubuntu 22020 0.0 0.0 13136 968 pts/1 S+ 07:57 0:00 grep --color=auto git
{code}
This is actually unexpected behaviour as it means that the option is not useful. We need to use a shallow clone since we have huge repositories and we don't need the full history of the repository.