Add a pixi toml url to the build parameter arguments (ported from #839)

[knmcguire]

See the description of #839

Ported to be a branch in ROS/ci so that we could more easily deploy it on a test ci job to test it out according to @claraberendsen

[knmcguire]

The idea is that we will try to deploy this particular branch on one of the test_ci jobs just to try it out since there is no CI to this PR. So we can wait with merging this PR after we have looked and settled that

[cottsay]

@knmcguire, I rebased this to make it easier to deploy to the test_* jobs, FYI.

[cottsay]

Deployed here: https://ci.ros2.org/job/test_ci_windows/

[knmcguire]

Thanks! And good we tested it out here because it indeed fails! I will look into it 😄

[knmcguire]

Still testing things out in the test ci! the latest github outages delayed that a bit, so as soon as I confirm this working I'll convert it back to 'ready for review'

[knmcguire]

Test with empty CI_PIXI_TOML_URL:

Test with msvc 2022 and with an alternative pixi.toml file (for the opencv fix) :

I don't think it is setting the pixi_toml_url correctly... so still some work to do here

[claraberendsen]

@knmcguire I think what is missing is to correctly set the new variable as DOCKER_BUILD_ARGS

See the reference

ci/job_templates/ci_job.xml.em

Lines 175 to 177 in db24a7c

	if [ -n "${CI_ROS_DISTRO+x}" ]; then
	export DOCKER_BUILD_ARGS="${DOCKER_BUILD_ARGS} --build-arg ROS_DISTRO=${CI_ROS_DISTRO}"
	fi

[knmcguire]

Ah yes, but that is the part for the Linux docker build. For the windows docker CI_ROS_DISTRO was part of the BUILD_ARGS

ci/job_templates/ci_job.xml.em

Line 250 in 2f86d08

set BUILD_ARGS=--build-arg WINDOWS_RELEASE_VERSION=%RELEASE_VERSION% --build-arg ROS_DISTRO=%CI_ROS_DISTRO%

Which I didn't want to add the pixi_toml_url by standard hence I added it as an conditional here :

if "!CI_PIXI_TOML_URL!" NEQ "" (
  set "BUILD_ARGS=!BUILD_ARGS! --build-arg PIXI_TOML_URL=%CI_PIXI_TOML_URL%"
)

I'll try to dig it in it a bit further

[knmcguire]

found the culprit! I was adding another argument to the build arguments... after the docker image has already been build.

We can redeploy this to test_ci_windows after this job has finished (which is for something else):

[knmcguire]

@cottsay or @claraberendsen could you deploy this job again to test_ci_windows? It should work this time

[cottsay]

could you deploy this job again to test_ci_windows?

Done.

[knmcguire]

Running again! I'm convieniently combining it with testing #845

see

[knmcguire]

Getting there... forgot to change the main ci branch so it didn't take the new dockerfile from this PR

[knmcguire]

And it builds with the new pixi.toml file! Before it would fail on MSVC 2022 because of the opencv version, but now we only see the nice 1800+ warnings, but that only confirms that it works.

Also in the log you see that pixi has updated opencv to 4.10 as it should:

21:46:09 opencv                                4.10.0                 qt5_py312h7cc77ad_503  26.1 KiB    conda  opencv

I'll redo the last commit as that inconveniently indicates a very big change, but otherwise ready for review, merge and deploy!

[knmcguire]

The force push was necessary to sign all my commits and also apply the same fix the packaging job

Just a question though: Do we want to enable this for the ci_launcher as well?

[claraberendsen]

Do we want to enable this for the ci_launcher as well?

I would say that we probably want that since usually ROS devs use that to launch ci.

[claraberendsen]

I think it would be a nice to have to validate the url being passed to be https and a normal url format. This as to prevent possible man-in-the-middle attacks with http and have some sanitation of the user input.

[knmcguire]

I've added an url check in the docker file.

Also I had to introduce the snippet parameters back in, but avoid using default parameters here.

These are again quite some changes... we should double check with a test deploy? perhaps also with a test_ci_launcher as well (but then we might also need all the test_ci_linux, test_ci_linux-aarch64, test_ci_linux-rhel and test_ci_windows as well)

[claraberendsen]

Changes deployed.
Testing a run with a custom pixi.toml from test_ci_launcher:

Windows:
Linux:

I aborted the other two jobs since we want to see that there is no failure if we don't deploy to the linux jobs the changes.