|
|
|
Our build server has about 100 configurations.
Each configuration managed by separate departments, thus our configuration tree is very dynamic (many changes of configuration names and configuration places in tree). We do not want to reach the situation that every time we have changes in configuration tree we will run such script or move files manually. Any changes in tree should be transparent and reliable to the QB users. QuickBuild can not move storage directories for the two reasons listed above if storage directory is hierarchically organized and named with configuration name. To workaround this problem, you may have different department using different storage directory, and have all child configurations under the same department using the option "use storage directory of parent configuration". That way all configurations under the same department will share the same global storage, and there is no need to relocate storage directories when change configuration name or hierarchy under that department.
We are not looking to move the old folders to the new path but to be able to see old builds data (logs artifacts) in QB after we change the configuration storage path and after configuration name, tree changes. We can effort that old data will be kept on old place on storage since it will be deleted as a result of builds cleanup police.
Does not the approach I recommended above solve this problem? With that approach, there are separate storage for different department, and any tree change inside that department will not affect visibility of build and configuration data.
The approach does not solve this issue for several reasons:
1. We are looking to manage all builds in file system as they look in configuration tree. 2. There are big chance that one department will reach 100000 builds on short run and then it will terminate all configurations of these department. 3. If for some reason we will have to move configuration from one department to another we will have again the initial problem on unreachable data. As explained in the issue comment, implementation of this feature at QB side is not yet scheduled as it is not trivial to handle this case at QuickBuild in a generic and automatic way. Moving build and configuration data every time a configuration name is changed or moved will result in very slow performance as every single change may cause a massive data moving. Storing folder path per build is also not an option as in that case, the parent folder to store the builds will contain a lot of sub folders and may eventually exceed the 100000 limit.
For now, please consider to use the script we provided to move builds and configurations data between different storage areas manually. |
1. it can not know previous storage location after you've changed it
2. even if it tries to move the build, the storage directory changing will be very slow after saving the setting will cause data copying/moving. Assume the case where you tried to set the storage setting several times to find the correct value.
In your case, I suggest to create a maintenance configuration to execute below groovy script (might need to be changed to adapt to your environment)
groovy:
import com.pmease.quickbuild.util.FileUtils;
globalStorageDir = new File("/path/to/global_storage");
for (buildDir in new File(globalStorageDir, "builds").listFiles()) {
logger.info("Copying build directory: " + buildDir.getAbsolutePath());
buildId = Long.parse(buildDir.getName());
build = system.buildManager.get(buildId);
if (build == null)
continue;
newStorageDir = new File(globalStorageDir, build.getConfiguration().getPathName() + "/builds");
newBuildDir = new File(newStorageDir, buildId.toString());
FileUtils.deleteDir(newBuildDir);
FileUtils.copyDir(buildDir, newBuildDir);
}
for (configurationDir in new File(globalStorageDir, "configurations").listFiles()) {
logger.info("Copying configuration directory: " + configurationDir.getAbsolutePath());
configurationId = Long.parse(configurationDir.getName());
configuration = system.configurationManager.get(configurationId);
if (build == null)
continue;
newStorageDir = new File(globalStorageDir, build.getConfiguration().getPathName() + "/configurations");
newConfigurationDir = new File(newStorageDir, configurationId.toString());
FileUtils.deleteDir(newConfigurationDir);
FileUtils.copyDir(configurationDir, newConfigurationDir);
}
Then run this configuration to have it copying build and configuration data. It might take a very long time, and I recommend to run it without any other builds running.
After correctly copied all build and configuration data, you may safely remove the directory "/path/to/global_storage/builds" and "/path/to/global_storage/configurations".