History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: QB-735
Type: Bug Bug
Status: Open Open
Priority: Major Major
Assignee: Robin Shen
Reporter: Maxim Fainshtein
Votes: 1
Watchers: 2
Operations

If you were logged in you would be able to see more operations.
QuickBuild

unable reach logs and artifacts after change of Storage Path

Created: 11/Nov/10 01:20 PM   Updated: 30/Nov/10 01:15 AM
Return to search
Component/s: None
Affects Version/s: None
Fix Version/s: None

Original Estimate: Unknown Remaining Estimate: Unknown Time Spent: Unknown
Environment:
Operating System Linux
QuickBuild Version 2.1.48


 Description  « Hide
Hi,

Today all our builds stored in one directory when each build has his directory named by builds ID (it is default QB configuration)

Issue begin when we reach 100000 of builds directories in one folder, it is an upper limit in the file system. On this stage QB unable create new directory thus all builds unable to run.

We search for solution to store builds in storage as configuration hierarchy. We found that change of Storage Path in configuration from blank to ${current.name} resolving our issue.
After above step we cant reach old builds data such logs and artifacts.

Copy all builds into new path is out of the question because as I mentioned we have 100000 builds.

Also if we keep our Storage Path = ${current.name} then if we change name or branch conf to another place, we also loose our old builds data (logs artifacts).

Thanks,
Maxim

 All   Comments   Work Log   Change History      Sort Order:
Robin Shen [12/Nov/10 01:16 AM]
In case of storage directory changing. Builds need to be moved to new area before you can access them. QuickBuild does not do this automatically for two reasons:
1. it can not know previous storage location after you've changed it
2. even if it tries to move the build, the storage directory changing will be very slow after saving the setting will cause data copying/moving. Assume the case where you tried to set the storage setting several times to find the correct value.

In your case, I suggest to create a maintenance configuration to execute below groovy script (might need to be changed to adapt to your environment)
groovy:

import com.pmease.quickbuild.util.FileUtils;

globalStorageDir = new File("/path/to/global_storage");

for (buildDir in new File(globalStorageDir, "builds").listFiles()) {
  logger.info("Copying build directory: " + buildDir.getAbsolutePath());
  buildId = Long.parse(buildDir.getName());
  build = system.buildManager.get(buildId);
  if (build == null)
    continue;
  newStorageDir = new File(globalStorageDir, build.getConfiguration().getPathName() + "/builds");
  newBuildDir = new File(newStorageDir, buildId.toString());
  FileUtils.deleteDir(newBuildDir);
  FileUtils.copyDir(buildDir, newBuildDir);
}

for (configurationDir in new File(globalStorageDir, "configurations").listFiles()) {
  logger.info("Copying configuration directory: " + configurationDir.getAbsolutePath());
  configurationId = Long.parse(configurationDir.getName());
  configuration = system.configurationManager.get(configurationId);
  if (build == null)
    continue;
  newStorageDir = new File(globalStorageDir, build.getConfiguration().getPathName() + "/configurations");
  newConfigurationDir = new File(newStorageDir, configurationId.toString());
  FileUtils.deleteDir(newConfigurationDir);
  FileUtils.copyDir(configurationDir, newConfigurationDir);
}

Then run this configuration to have it copying build and configuration data. It might take a very long time, and I recommend to run it without any other builds running.

After correctly copied all build and configuration data, you may safely remove the directory "/path/to/global_storage/builds" and "/path/to/global_storage/configurations".

Maxim Fainshtein [15/Nov/10 10:02 AM]
Our build server has about 100 configurations.
Each configuration managed by separate departments, thus our configuration tree is very dynamic (many changes of configuration names and configuration places in tree).
We do not want to reach the situation that every time we have changes in configuration tree we will run such script or move files manually.
Any changes in tree should be transparent and reliable to the QB users.

Robin Shen [15/Nov/10 10:49 AM]
QuickBuild can not move storage directories for the two reasons listed above if storage directory is hierarchically organized and named with configuration name. To workaround this problem, you may have different department using different storage directory, and have all child configurations under the same department using the option "use storage directory of parent configuration". That way all configurations under the same department will share the same global storage, and there is no need to relocate storage directories when change configuration name or hierarchy under that department.

Maxim Fainshtein [15/Nov/10 11:42 AM]
We are not looking to move the old folders to the new path but to be able to see old builds data (logs artifacts) in QB after we change the configuration storage path and after configuration name, tree changes. We can effort that old data will be kept on old place on storage since it will be deleted as a result of builds cleanup police.

Robin Shen [15/Nov/10 11:50 AM]
Does not the approach I recommended above solve this problem? With that approach, there are separate storage for different department, and any tree change inside that department will not affect visibility of build and configuration data.

Maxim Fainshtein [15/Nov/10 12:14 PM]
The approach does not solve this issue for several reasons:
1. We are looking to manage all builds in file system as they look in configuration tree.
2. There are big chance that one department will reach 100000 builds on short run and then it will terminate all configurations of these department.
3. If for some reason we will have to move configuration from one department to another we will have again the initial problem on unreachable data.

Robin Shen [30/Nov/10 01:15 AM]
As explained in the issue comment, implementation of this feature at QB side is not yet scheduled as it is not trivial to handle this case at QuickBuild in a generic and automatic way. Moving build and configuration data every time a configuration name is changed or moved will result in very slow performance as every single change may cause a massive data moving. Storing folder path per build is also not an option as in that case, the parent folder to store the builds will contain a lot of sub folders and may eventually exceed the 100000 limit.

For now, please consider to use the script we provided to move builds and configurations data between different storage areas manually.