Documention on Integration Tests is missing and add them to CI

[norpol]

The integration tests aren't run on the Travis CI, @allburov do you run them against your own Artifactory Instance?

I know that the API isn't available without an Artifactory license, I've just sent them a support request asking if they could also add API scope to the free Artifactory version, otherwise I don't think it will be possible to run integration tests against via. Travis.

Seems go-artifactory is getting an Artifactory Cloud instance sponsored for their CI tests.

[allburov]

Yes, I have test instance on my PC in docker with 30days free licence
https://jfrog.com/artifactory/free-trial/

[allburov]

instance sponsored for their CI tests.

It would be great :)

[zhan9san]

Hi @allburov
Is it valuable to introduce the following integration test approach?

• Mock up Session.request
• Generate an expected response from file

Atlassian test
Mock

[allburov]

@zhan9san I think it's not.
I tried this approach once in TFS library https://github.com/devopshq/tfs and it's too heavy for supporting and doesn't catch real system behavior, only a snapshot of one version.

The best way - create a docker container during integration tests or use SaaS solution and run tests there.

If you want to - you can create separate tests with mocks but let's keep the original integration tests for real integration tests.
Also we can modify them this way:

1. When you run pytest -mintegration --use-mock it uses "mock" implementation
2. When you run pytest -mintegration - it uses an Artifactory instance

[zhan9san]

it's too heavy for supporting and doesn't catch real system behavior, only a snapshot of one version.

I agree with you.

Regarding container, I didn't find an out-of-box solution. At least, there are two manual steps.

1. The trial license could be mounted. In which way, we can bypass applying license issue. But the re-generating is also an issue. Maybe it can be resolved by creating a script to register a trial account and retrieve license in mail box(Too tedious)
2. The initial password need to be reset. Maybe it can be reset by mounting one file somewhere.

We cannot do anything during integration tests workflow until there is an out-of-box container.

BTW, have you tried to reach out to Jfrog getting a sponsored SaaS installation?

[allburov]

BTW, have you tried to reach out to Jfrog getting a sponsored SaaS installation?

I haven't. We've talked about this with one of contributors, but didn't make any steps further.

At least, there are two manual steps.

It's probably was the reason why we didn't add integration tests yet

[beliaev-maksim]

@allburov
what is about cloud version?
looks like it is available for free

[allburov]

@beliaev-maksim does it support API calls? "Free" plan didn't support API before
If it does - let's try to add it in CI\CD pipeline!

[briantist]

I imagine that the free cloud version supports the same APIs as the OSS version (which is missing a lot of basic stuff). I am dealing with the same issues trying to run integration tests with my project (which depends on this project!), so I'll be watching this issue as well:

• Run integration tests against Artifactory briantist/galactory#6

My project is a little easier since most of what it actually does, does not require a pro license (there's a piece that does that I'm working to remove), but the pro APIs would be most useful in trying to set up the CI environment and that's where I'm hitting issues myself.

---

If there's any luck getting a sponsored open source license for this purpose I'd love to hear how that conversation went!