|
|
|
[
Permlink
| « Hide
]
Robin Shen [28/Feb/20 12:43 AM]
I am not familiar with Ivy/grape things. How do you anticipate to use that in QB? Via a special step to call Ivy/grape command?
Hi Robin,
Was I did a few months back as a proof of concept : 1. Added ivy-2.5.0-rc1.jar to plugins\com.pmease.quickbuild.libs 2. Create a .groovy folder in quickbuild's user home folder with a grapeConfig.xml 2. Build a groovy library 3. Publish the jar file to artifactory 4. Create a script step using groovy: ---------------------- groovy: @GrabResolver(name='artifactory', root='https://artifactory/artifactory/maven-playground-local/', m2Compatible='true') @Grab("org.ubisoft.bge2:bge2-qb-groovylib:0.4") import org.ubisoft.bge2.JiraTools def jt = new JiraTools(); ---------------------- The issue being this is only doable for scripts run on server right now because agents lacks the grapeconfig.xml file We can of course create our own specific agent setup to make sure this work. QB has it's own defaultGrapeConfig.xml in com.pmease.quickbuild_XXX.jar => groovy-all-1.8.9.jar and if i remeber correctly some paths doesn't exist anymore, so we have to wait for the timeout to fire before we can use our own config file I'm attaching our grapeConfig.xml as a sample Looks to me that you want to use new groovy libraries. If so, how about putting the library (jira tool lib here I guess) directly into QB's "plugins/com.pmease.quickbuild.libs" directory?
I'd suggest to put desired groovy libraries into QB's "plugins/com.pmease.quickbuild.libs" directory as suggested above. You may also need to put the whole groovy (groovy-all.jar) into that directory if these libraries depends on new groovy versions.
Robin, we're aware of this possibility.
Although dynamic @Grab imports are more flexible, we don't have to manage dependencies manually. In our scenario @Grab and import statement makes some kind of infinite loop for script step - we're trying to grab artifact from official maven repository. I am not familiar with Ivy/Grab/Grape things. Can you demonstrate how you want to enhance groovy support in QB via an example?
Sure,
Grape is groovy built-in package manager simillar to Maven. @Grape annotation are resolved during compilation Dependencies are downloaded and cached as normally as Maven would do Reference: http://docs.groovy-lang.org/latest/html/documentation/grape.html Here's sample: [Misc > Execute Script] ---------------------- groovy: @Grab(group='com.squareup.okhttp3', module='okhttp', version='4.9.0') import com.squareup.okhttp.OkHttpClient public final OkHttpClient client = new OkHttpClient(); // Do some fancy API calls ---------------------- ## First test case: 1. As Benoit mentioned I've added --> `1. Added ivy-2.5.0-rc1.jar to plugins\com.pmease.quickbuild.libs` 2. Run QB Script step >> Resulted: Running script above throws: java.lang.NoClassDefFoundError: org/apache/ivy/core/report/ResolveReport ## Second test case: 1. Added manually 'ivy-2.5.0-rc1.jar' to quickbuild-10.0.21.jar/libs/ivy-2.5.0-rc1.jar 2. Edited quickbuild-10.0.21.jar/.classpath adding: <classpathentry exported="true" kind="lib" path="lib/ivy-2.5.0-rc1.jar"/> 3. Run QB Script step >> Resulted: Script runs, but falls into infinite loop without any log to the console ## Third test case: 1. Get into QB machine shell 2. Run grape tool to download dependency manually --> `grape install com.squareup.okhttp3 okhttp` >> Resulted: Success! Hi Maciej,
Glad you tried too :) I have some success on my own, by modifying just the wrapper.conf for both server and agents using an envritonment variable to change the folder where the grapeConfig.xml file is read. Unfortunately there's an issue that bothers me, which is not related to QB but to grape/ivy itself. It works great for me as long as I'm still using java8 as soon as I move to java11 it crashes in grape. From what I see this as been reported as an issue for IVY but I failed to find any communication from the apache foundation on any plan to support this. I added this in both conf/wrapper.conf and agent/buildagent/wrapper.conf set.grape.config=../plugins/com.pmease.quickbuild.bootstrap/grapeConfig.xml then pushed my grapeConfig.xml to plugins/com.pmease.quickbuild.bootstrap so both server and agent get the configuration. for ivy lib I believe just having it in : plugins/com.pmease.quickbuild.libs would work (I didnt had to change the quickbuild-10.0.21.jar/.classpath on my end) Thanks for the detailed instructions. I can run your script successfully by copying groovy-3.0.6.jar and ivy-2.5.0-rc1.jar into "<QB install dir>/plugins/com.pmease.quickbuild.libs". You may also need to copy other groovy 3.0.6 utility jars into that folder if necessary.
Sorry I realized I failed to mention I also added this in our wrapper.conf (again both server and buildagent)
wrapper.java.classpath.24=../plugins/com.pmease.quickbuild.bootstrap/lib/ivy-2.5.0-rc1.jar (of course .24 is for us , make just make sure no other line is already using that index) Please remove groovy-2.4.9.jar from the folder. Otherwise it will conflict with groovy 3.0.6
It is odd that it does not work at your side. I am testing with 10.0.25 on Linux with JDK8. And the step works on both server and agent. I have to modify your script a bit though (to remove public final) otherwise there is a compilation error:
@Grab(group='com.squareup.okhttp3', module='okhttp', version='4.9.0') import com.squareup.okhttp.OkHttpClient OkHttpClient client = new OkHttpClient(); logger.info(client.toString()); Tested code without public final keywords resulted the same `NoClassDefFoundError`
Mine setup: QB server running in Docker container ( base Ubuntu 20 image ) running on VM (Ubuntu 20) VM --> Docker --> QB 10.0.21 Will try Benoit's approach with editting wrapper Although I'm a bit confused why Benoit located ivy-2.5.0-rc1.jar in `/com.pmease.quickbuild.bootstrap` location instead of `/com.pmease.quickbuild.libs` Isn't everything in com.pmease.quickbuild.bootstrap resolved by QuickBuild when server is starting? I'm not aware of differences between both dirs. Is it possible that your docker container lacks some folder/environment required by Ivy? Please test out side of docker to see if it works at your side. Third party libs are suggested to put in com.pmease.quickbuild.libs as this folder will be synced with agent. Bootstrap jar is used to boot QB and it will not be synced with agent.
Update:
I've rebuild docker container and copied ivy jar to "<QB install dir>/plugins/com.pmease.quickbuild.libs" Checked: - user has got ~/.groovy/grapes dir - user has got access to above - verified proxy settings Run example script as Execute Script with DEBUG log level and timeout for 60seconds. Resulted: >> Step is running, but cannot be stopped. Finally, after 8 minutes we've got runtime exception: General error during conversion: Error grabbing Grapes -- [unresolved dependency: com.squareup.okhttp3#okhttp;4.9.0: not found] java.lang.RuntimeException: Error grabbing Grapes -- [unresolved dependency: com.squareup.okhttp3#okhttp;4.9.0: not found] As far as you know is there any ivy/grape default config shaded in QB? Seems like Grab cannot pass through proxy. There is no default ivy/grape config shipped with QB. To narrow down the issue, how about testing on a machine with direct access to external network?
Sorry, I've just realized, that Benoit answered that
Q: As far as you know is there any ivy/grape default config shaded in QB? Seems like Grab cannot pass through proxy. A:QB has it's own defaultGrapeConfig.xml in com.pmease.quickbuild_XXX.jar => groovy-all-1.8.9.jar In wrapper.conf set.grape.config=../plugins/com.pmease.quickbuild.bootstrap/grapeConfig.xml then pushed my grapeConfig.xml to plugins/com.pmease.quickbuild.bootstrap I am not aware of grape config shipped in groovy-all-1.8.9.jar. However if groovy-3.0.6.jar is put in com.pmease.quickbuild.libs folder, QB will not be using groovy-all-1.8.9.jar.
At my side, it is printing 3.0.6. Since QB does not have official docker image. Please test it on bare metal machine first to narrow down issue.
Yes in groovy-all there is a default grape config file in :
groovy\grape\defaultGrapeConfig.xml unfortunately it contains an url to a server no longer running. Maciej is your docker running a java 8 vm ? as I mentionned earlier there are issues with versions >= java 11 Also are you confident your grapeconfig is ok ? (did you check with a regular script outside of QB) In my groovy scripts I also add the grabresolver option : @GrabResolver(name='artifactory', root='https://artifactory/maven-playground', m2Compatible='true') @Grab("org.company.test:JiraTools:0.0.0.2") in case it helps I'm pretty sure it's almost done... Probably proxy settings are missing.
Benoit: Checked grapeConfig, it's fine. We're running JDK 1.8 Robin: Is there any proxy related setting that QB reads? I've set proxy settings to both env. variables: JAVA_OPTS, ANT_OPTS To re-produce QB issue I've implemented clean groovy project in my IDEA. Run 3 tests: 1. IntelliJ - Build while connected with VPN: Error grabbing Grapes -- [unresolved dependency: com.google.code.google-collections#google-collect;snapshot-20080530: not found] 2. IntelliJ - Build while disconnected from VPN: Success! 3. Groovy shell - `groovy test.groovy` - while connected to VPN Success! Hello!
I've found working solution :) I'm so thankful for your help! Last piece was adding proxy settings to wrapper.conf like so: #wrapper.java.additional.25=-Dhttp.proxySet=true #wrapper.java.additional.26=-Dhttp.proxyHost=<the proxy host> #wrapper.java.additional.27=-Dhttp.proxyPort=<the proxy port> |