| << Back to previous view |
|
[QB-440] The migration of storage directories from QuickBuild 1.x can cause the ext3 filesystem limit of 32000 entries per directory to be reached and then fails
|
|
| Status: | Resolved |
| Project: | QuickBuild |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Minor |
| Reporter: | Petr | Assigned To: | Robin Shen |
| Resolution: | Fixed | Votes: | 0 |
| Remaining Estimate: | Unknown | Time Spent: | Unknown |
| Original Estimate: | Unknown | ||
| Environment: |
CentOS 5 / RedHat Enterprise Linux 4.6 32-bit x86
ext3 filesystem |
||
| Description |
|
Hello,
we're currently evaluating the QuickBuild 2.0 with 30 day trial key recently received to determine whether renewing support subscription and migrating from our current QuickBuild 1.2.13 to new QuickBuild 2.0 would be an useful option for us. So yesterday I started with test migration of data from QuickBuild 1.2 and have noticed few issues where the following one is the most important. The migration procedure seems to take all storage data from QuickBuild 1.2 and copy them under new hierarchy into the specified storage directory. The first ugly thing is that all the contents in the new storage directory is created as directories with just numbers as their names which makes the whole directory structure hard to review. But the biggest problem is that all the migrated content is put just into the <new-storage-directory>/builds/ directory which can result in tens of thousands directories (with just numbers as their names) to be created in that single directory. For example on ext3 filesystem (probably the most common filesystem on Linux OS at this time) there's a limit of 32000 inodes (31998 subdirectories) per directory so it's pretty easy to reach this limit and end up with migration failure. Some other filesystems such as XFS or JFS have virtually no limit here, but tt may be difficult to migrate to them in some cases so taking such filesystem limits into account during the migration procedure would be essential. Check http://www.inthelight.co.nz/techo/pcfilesystem.htm for rpetty good overview of some filesystems limit. It would be also great to have an option to migrate just the configuration stuff excluding the builds, logs, etc from previous version so that these data won't be duplicated. Then we'll be able to leave the older builds in the old storage directories and only new builds will be created in the new one. |
| Comments |
| Comment by Robin Shen [ 25/Sep/09 01:52 PM ] |
|
You may avoid migrating published artifacts of old builds by renaming the global publish directory in 1.x to a different directory temporarily.
To put build artifacts under project names, please just specify the storage path property of the root configuration as /path/to/storage/${configuration.pathName} |
| Comment by Petr [ 25/Sep/09 02:32 PM ] |
|
> You may avoid migrating published artifacts of old builds by renaming the global publish directory in 1.x to a different directory temporarily.
That's almost exactly what I've done. Actually I copied the 1.x installation to another server and performed migration there for performance reasons and I completely skipped the 1.x publish directory when copying. However it still creates these directories in new storage path, just they are empty now. So the problem with filesystem limitation of n subdirectories per directory still persists even in this case. As a workaround I setup a simple check for the amount of subdirectories in the new storage path and if there were more than 5000 entries I just deleted them. This let the migration to finish at least. |
| Comment by Robin Shen [ 26/Sep/09 01:15 AM ] |
| Thanks for the report. We will fix this by not creating any build directories if the original publish directory can not be found. The fix will be contained in 2.0.1 which will be released soon. |