History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: QB-3503
Type: Improvement Improvement
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Robin Shen
Reporter: Benoit Maury-Bouet
Votes: 0
Watchers: 2

If you were logged in you would be able to see more operations.

Expose grape/ ivy settings

Created: 26/Feb/20 09:27 AM   Updated: 28/Oct/20 11:13 PM
Component/s: None
Affects Version/s: None
Fix Version/s: None

Original Estimate: Unknown Remaining Estimate: Unknown Time Spent: Unknown
File Attachments: 1. XML File grapeConfig.xml (1.0 kb)

Image Attachments:

1. loop.png
(17 kb)

2. plugins_ls.png
(31 kb)

3. step_content.png
(35 kb)

4. step_logs.png
(53 kb)

 Description  « Hide
Dear all,

We would love to use IVY and grape to build groovy libs and apis we could easily reuse in scripts.
so we could use @Grab to point to our gitlab instance.

 All   Comments   Work Log   Change History      Sort Order:
Robin Shen [28/Feb/20 12:43 AM]
I am not familiar with Ivy/grape things. How do you anticipate to use that in QB? Via a special step to call Ivy/grape command?

Benoit Maury-Bouet [02/Mar/20 08:03 AM]
Hi Robin,

Was I did a few months back as a proof of concept :
1. Added ivy-2.5.0-rc1.jar to plugins\com.pmease.quickbuild.libs
2. Create a .groovy folder in quickbuild's user home folder with a grapeConfig.xml
2. Build a groovy library
3. Publish the jar file to artifactory
4. Create a script step using groovy:
@GrabResolver(name='artifactory', root='https://artifactory/artifactory/maven-playground-local/', m2Compatible='true')
import org.ubisoft.bge2.JiraTools

def jt = new JiraTools();

The issue being this is only doable for scripts run on server right now because agents lacks the grapeconfig.xml file
We can of course create our own specific agent setup to make sure this work.

QB has it's own defaultGrapeConfig.xml in
com.pmease.quickbuild_XXX.jar => groovy-all-1.8.9.jar
and if i remeber correctly some paths doesn't exist anymore, so we have to wait for the timeout to fire before we can use our own config file

I'm attaching our grapeConfig.xml as a sample

Change by Benoit Maury-Bouet [02/Mar/20 08:04 AM]
Field Original Value New Value
Attachment grapeConfig.xml [ 10830 ]

Robin Shen [02/Mar/20 11:56 PM]
Looks to me that you want to use new groovy libraries. If so, how about putting the library (jira tool lib here I guess) directly into QB's "plugins/com.pmease.quickbuild.libs" directory?

Maciej M [26/Oct/20 09:53 PM]

Is there any update?

Actually I'm trying to implement the same case.


Robin Shen [26/Oct/20 11:38 PM]
I'd suggest to put desired groovy libraries into QB's "plugins/com.pmease.quickbuild.libs" directory as suggested above. You may also need to put the whole groovy (groovy-all.jar) into that directory if these libraries depends on new groovy versions.

Maciej M [27/Oct/20 01:26 AM]
Robin, we're aware of this possibility.

Although dynamic @Grab imports are more flexible, we don't have to manage dependencies manually.

In our scenario @Grab and import statement makes some kind of infinite loop for script step - we're trying to grab artifact from official maven repository.

Robin Shen [27/Oct/20 01:33 AM]
I am not familiar with Ivy/Grab/Grape things. Can you demonstrate how you want to enhance groovy support in QB via an example?

Maciej M [27/Oct/20 01:51 AM]

Grape is groovy built-in package manager simillar to Maven.
@Grape annotation are resolved during compilation
Dependencies are downloaded and cached as normally as Maven would do

Reference: http://docs.groovy-lang.org/latest/html/documentation/grape.html

Here's sample:

[Misc > Execute Script]

@Grab(group='com.squareup.okhttp3', module='okhttp', version='4.9.0')
import com.squareup.okhttp.OkHttpClient

public final OkHttpClient client = new OkHttpClient();

// Do some fancy API calls

## First test case:
1. As Benoit mentioned I've added --> `1. Added ivy-2.5.0-rc1.jar to plugins\com.pmease.quickbuild.libs`
2. Run QB Script step

>> Resulted: Running script above throws: java.lang.NoClassDefFoundError: org/apache/ivy/core/report/ResolveReport

## Second test case:
1. Added manually 'ivy-2.5.0-rc1.jar' to quickbuild-10.0.21.jar/libs/ivy-2.5.0-rc1.jar
2. Edited quickbuild-10.0.21.jar/.classpath adding:
<classpathentry exported="true" kind="lib" path="lib/ivy-2.5.0-rc1.jar"/>
3. Run QB Script step

>> Resulted: Script runs, but falls into infinite loop without any log to the console

## Third test case:
1. Get into QB machine shell
2. Run grape tool to download dependency manually --> `grape install com.squareup.okhttp3 okhttp`

>> Resulted: Success!

Benoit Maury-Bouet [27/Oct/20 07:33 AM]
Hi Maciej,
Glad you tried too :)
I have some success on my own, by modifying just the wrapper.conf for both server and agents using an envritonment variable to change the folder where the grapeConfig.xml file is read.

Unfortunately there's an issue that bothers me, which is not related to QB but to grape/ivy itself.
It works great for me as long as I'm still using java8 as soon as I move to java11 it crashes in grape.
From what I see this as been reported as an issue for IVY but I failed to find any communication from the apache foundation on any plan to support this.

I added this in both conf/wrapper.conf and agent/buildagent/wrapper.conf

then pushed my grapeConfig.xml to plugins/com.pmease.quickbuild.bootstrap

so both server and agent get the configuration.
for ivy lib I believe just having it in : plugins/com.pmease.quickbuild.libs would work (I didnt had to change the quickbuild-10.0.21.jar/.classpath on my end)

Robin Shen [27/Oct/20 08:07 AM]
Thanks for the detailed instructions. I can run your script successfully by copying groovy-3.0.6.jar and ivy-2.5.0-rc1.jar into "<QB install dir>/plugins/com.pmease.quickbuild.libs". You may also need to copy other groovy 3.0.6 utility jars into that folder if necessary.

Maciej M [27/Oct/20 08:42 AM]
So I've reverted to clean QuickBuild 10.0.21, 2020-07-08.

Copied ivy and groovy to libs and still getting `NoClassDefFoundError`

I've tried many combinations with groovy lib tho.

Tried exact 1.8.9 as QB uses, then 2.4.9, then 3.0.6.

Change by Maciej M [27/Oct/20 08:42 AM]
Attachment step_logs.png [ 10865 ]
Attachment step_content.png [ 10866 ]
Attachment plugins_ls.png [ 10867 ]

Maciej M [27/Oct/20 08:42 AM]
^Attached screenshots

Benoit Maury-Bouet [27/Oct/20 08:48 AM]
Sorry I realized I failed to mention I also added this in our wrapper.conf (again both server and buildagent)


(of course .24 is for us , make just make sure no other line is already using that index)

Robin Shen [27/Oct/20 08:52 AM]
Please remove groovy-2.4.9.jar from the folder. Otherwise it will conflict with groovy 3.0.6

Maciej M [27/Oct/20 09:00 AM]
Thanks Benoit!

I'll try this, but currently we've got scheduled maintenance and I can't access the server :(

Maciej M [27/Oct/20 09:01 AM]
Yes Robin, I'm keeping only one groovy lib at the same time when testing

Robin Shen [27/Oct/20 09:16 AM]
It is odd that it does not work at your side. I am testing with 10.0.25 on Linux with JDK8. And the step works on both server and agent. I have to modify your script a bit though (to remove public final) otherwise there is a compilation error:

@Grab(group='com.squareup.okhttp3', module='okhttp', version='4.9.0')
import com.squareup.okhttp.OkHttpClient

OkHttpClient client = new OkHttpClient();


Maciej M [27/Oct/20 09:33 AM]
Tested code without public final keywords resulted the same `NoClassDefFoundError`

Mine setup:
QB server running in Docker container ( base Ubuntu 20 image ) running on VM (Ubuntu 20)

VM --> Docker --> QB 10.0.21

Will try Benoit's approach with editting wrapper

Although I'm a bit confused why Benoit located ivy-2.5.0-rc1.jar in `/com.pmease.quickbuild.bootstrap` location instead of `/com.pmease.quickbuild.libs`

Isn't everything in com.pmease.quickbuild.bootstrap resolved by QuickBuild when server is starting? I'm not aware of differences between both dirs.

Robin Shen [27/Oct/20 09:39 AM]
Is it possible that your docker container lacks some folder/environment required by Ivy? Please test out side of docker to see if it works at your side. Third party libs are suggested to put in com.pmease.quickbuild.libs as this folder will be synced with agent. Bootstrap jar is used to boot QB and it will not be synced with agent.

Maciej M [27/Oct/20 09:25 PM]

Here's quick update:

Tested outside of docker and still getting the same exception.

Also tried Benoit approach with wrapper.conf and ivy jar in bootstrap dir.

Any ideas?

Change by Maciej M [27/Oct/20 11:12 PM]
Attachment loop.png [ 10868 ]

Maciej M [27/Oct/20 11:22 PM]

I've rebuild docker container and copied ivy jar to "<QB install dir>/plugins/com.pmease.quickbuild.libs"

- user has got ~/.groovy/grapes dir
- user has got access to above
- verified proxy settings

Run example script as Execute Script with DEBUG log level and timeout for 60seconds.


>> Step is running, but cannot be stopped. Finally, after 8 minutes we've got runtime exception:

General error during conversion: Error grabbing Grapes -- [unresolved dependency: com.squareup.okhttp3#okhttp;4.9.0: not found]

java.lang.RuntimeException: Error grabbing Grapes -- [unresolved dependency: com.squareup.okhttp3#okhttp;4.9.0: not found]

As far as you know is there any ivy/grape default config shaded in QB? Seems like Grab cannot pass through proxy.

Robin Shen [27/Oct/20 11:25 PM]
There is no default ivy/grape config shipped with QB. To narrow down the issue, how about testing on a machine with direct access to external network?

Maciej M [27/Oct/20 11:28 PM]
Sorry, I've just realized, that Benoit answered that

Q: As far as you know is there any ivy/grape default config shaded in QB? Seems like Grab cannot pass through proxy.
A:QB has it's own defaultGrapeConfig.xml in com.pmease.quickbuild_XXX.jar => groovy-all-1.8.9.jar

In wrapper.conf
  then pushed my grapeConfig.xml to plugins/com.pmease.quickbuild.bootstrap

Robin Shen [27/Oct/20 11:34 PM]
I am not aware of grape config shipped in groovy-all-1.8.9.jar. However if groovy-3.0.6.jar is put in com.pmease.quickbuild.libs folder, QB will not be using groovy-all-1.8.9.jar.

Maciej M [27/Oct/20 11:44 PM]
Actually, how can I ensure, that QB uses groovy 3.0.6?

Done test case:



>> Resulted: 23:43:18,708 INFO - 1.8.9

Robin Shen [27/Oct/20 11:47 PM]
At my side, it is printing 3.0.6. Since QB does not have official docker image. Please test it on bare metal machine first to narrow down issue.

Benoit Maury-Bouet [28/Oct/20 08:26 AM]
Yes in groovy-all there is a default grape config file in :

unfortunately it contains an url to a server no longer running.

Maciej is your docker running a java 8 vm ? as I mentionned earlier there are issues with versions >= java 11

Also are you confident your grapeconfig is ok ? (did you check with a regular script outside of QB)

In my groovy scripts I also add the grabresolver option :
@GrabResolver(name='artifactory', root='https://artifactory/maven-playground', m2Compatible='true')

in case it helps

Maciej M [28/Oct/20 11:56 AM]
I'm pretty sure it's almost done... Probably proxy settings are missing.


Checked grapeConfig, it's fine.
We're running JDK 1.8


Is there any proxy related setting that QB reads?

I've set proxy settings to both env. variables: JAVA_OPTS, ANT_OPTS

To re-produce QB issue I've implemented clean groovy project in my IDEA.

Run 3 tests:

1. IntelliJ - Build while connected with VPN:

 Error grabbing Grapes -- [unresolved dependency: com.google.code.google-collections#google-collect;snapshot-20080530: not found]

2. IntelliJ - Build while disconnected from VPN:


3. Groovy shell - `groovy test.groovy` - while connected to VPN


Maciej M [28/Oct/20 12:43 PM]

I've found working solution :)

I'm so thankful for your help!

Last piece was adding proxy settings to wrapper.conf like so:

#wrapper.java.additional.26=-Dhttp.proxyHost=<the proxy host>
#wrapper.java.additional.27=-Dhttp.proxyPort=<the proxy port>

Robin Shen [28/Oct/20 11:13 PM]
Glad to see it is working now. :)

Change by Robin Shen [28/Oct/20 11:13 PM]
Status Open [ 1 ] Closed [ 6 ]
Resolution Fixed [ 1 ]