Xsuite converter

[swhite2401]

This PR implements an Xsuite converter.

Xsuite is not required for this to work as everything is handled using json files.

This is still work in progress fro the moment, comments and suggestions welcome!

[lfarv]

Hello @swhite2401 : very interesting !
After a first look (even if it is still evolving), I have a few remarks:

• the BasicElement class should not inherit from Element: apparently none of the Element methods or attributes are used. And it's not meant to be used as an Element. So inheriting from object looks good enough.
• you could make use of the new Element.from_file class method:

    def get_at_element(self) -> Element:
        self._params_to_at()
        self._integrator_to_at()
        cls = self._class_to_at()
        return cls.from_file(self.atparams)
  

  Here, the _class_to_at method returns the AT class rather than storing it in atparams, and the _xsuite2at_class directories store AT classes rather than AT class names. It looks to me simpler, but your way is good as well.

[swhite2401]

@lfarv thanks for taking a look, good point!
We are doing some benchmarking with @kandre2 and colleagues form CERN to understand some discrepancies.
I may need your help if some question arise, I'll let you know.

[lfarv]

@swhite2401 : I am indeed following this: it's very interesting. May be you could have a look at Mad reading/writing thee there are similar things: JSON encoding, defining unique element names for instance…

[swhite2401]

@swhite2401 : I am indeed following this: it's very interesting. May be you could have a look at Mad reading/writing thee there are similar things: JSON encoding, defining unique element names for instance…

Ok great! Don't hesitate to jump in if you have ideas, I consider this PR as a test branch for the moment, I am not particularly attached to this implementation so feel free to make changes if you se places that could be improved!

And if you want to participate in the benchmarking discussion I can also put you in the loop, let me know!

[lfarv]

@swhite: there are indeed many similarities between this implementation and the one for MAD or Elegant. I think it would be interesting to use the same names for similar things. For instance, the MAD parser uses element classes very similar to your BasicElement class (look at the quadrupole and similar classes in madx.py). But

• MADX, MAD8 and Elegant classes have a to_at method to generate the AT element, instead of your get_at_element,
• They have a from_at class method, acting as a factory method, used to create the instance instead of calling the constructor, when exporting from AT. This more or less replaces your at_to_xsuite function

I think it could be easier for maintenance to have the same names.

I could work on that (I just spent a lot of time cleaning the MAD parser), and the best way not to interfere with your tests is to work on another branch. But the difficulty is to keep track of all the corrections you would do simultaneously…

[lfarv]

2nd point: I am also interested in your benchmarking. It could reveal problems in Mad files. For instance I noticed that you recently added a $\pi$ shift on the RF phase lag. This is the same as changing the voltage sign (AT has a minus sign for the momentum kick in RFCavityPass), or changing the charge of the particle. Is there the same problem with MAD ?

[lfarv]

@swhite2401
While I was looking at the code, with the idea of simplifying the structure, I got on question about the way multipole strengths are specified. Xsuite allows specifications both in terms of strengths: k2, k1s…, and in terms of arrays of integrated strengths knl, ksl. Normally, one should use one or the other. But as far as I could see, the documentation says nothing about the case where both are specified (and possibly conflicting). You made the choice of adding both, which I find a bit dangerous. One could be tempted to specify, for a quadrupole for instance, both k1 = a and knl = [0, a*l], which are redundant. In the present implementation, one would get twice the expected strength.

We had the same question in AT, which was answered in #695 by raising an exception if redundant or conflicting information is given.

Here, one should choose between

• give priority to one way (kx or knl)
• raise an exception if both are given (as in AT)
• add both (as it is done now)

[swhite2401]

@swhite: there are indeed many similarities between this implementation and the one for MAD or Elegant. I think it would be interesting to use the same names for similar things. For instance, the MAD parser uses element classes very similar to your BasicElement class (look at the quadrupole and similar classes in madx.py). But

• MADX, MAD8 and Elegant classes have a to_at method to generate the AT element, instead of your get_at_element,
• They have a from_at class method, acting as a factory method, used to create the instance instead of calling the constructor, when exporting from AT. This more or less replaces your at_to_xsuite function

I think it could be easier for maintenance to have the same names.

I could work on that (I just spent a lot of time cleaning the MAD parser), and the best way not to interfere with your tests is to work on another branch. But the difficulty is to keep track of all the corrections you would do simultaneously…

Hello @lfarv I agree that startting a new branch with a clean implementation would be better, for the benchmarking I do not need to convert lattices for the moment, I will report finding in the new PR so you can integrate them

[swhite2401]

2nd point: I am also interested in your benchmarking. It could reveal problems in Mad files. For instance I noticed that you recently added a π shift on the RF phase lag. This is the same as changing the voltage sign (AT has a minus sign for the momentum kick in RFCavityPass), or changing the charge of the particle. Is there the same problem with MAD ?

Yes this is a bit empirical, in fact I suspected a particle charge issue.... I have been mostly running 4D simulations so far so I did not look further into this.
Btw, I can't remember what is the status or the "particle" object in AT, are we allowed to change the charge? What about non relativistic case?

[swhite2401]

@swhite2401 While I was looking at the code, with the idea of simplifying the structure, I got on question about the way multipole strengths are specified. Xsuite allows specifications both in terms of strengths: k2, k1s…, and in terms of arrays of integrated strengths knl, ksl. Normally, one should use one or the other. But as far as I could see, the documentation says nothing about the case where both are specified (and possibly conflicting). You made the choice of adding both, which I find a bit dangerous. One could be tempted to specify, for a quadrupole for instance, both k1 = a and knl = [0, a*l], which are redundant. In the present implementation, one would get twice the expected strength.

We had the same question in AT, which was answered in #695 by raising an exception if redundant or conflicting information is given.

Here, one should choose between

• give priority to one way (kx or knl)
• raise an exception if both are given (as in AT)
• add both (as it is done now)

In Xsuite the knl arrays are mutlipole field errors: they are added the the main strength so there is no conflict.
We may discuss adopting a similar approach in AT, it could be interesting to add a PolynomA/Berror field to distinguish between main field componenent and field errors.

[lfarv]

In Xsuite the knl arrays are mutlipole field errors
Ok, I misunderstood ! And I agree with you suggestion to add a similar approach in AT (but that's another topic)

Next point: is the format of Xsuite json files documented ? Or did you do reverse engineering ?

[swhite2401]

In Xsuite the knl arrays are mutlipole field errors
Ok, I misunderstood ! And I agree with you suggestion to add a similar approach in AT (but that's another topic)

Next point: is the format of Xsuite json files documented ? Or did you do reverse engineering ?

I did reverse engineering and it was very painful! Their file format is a mess they basically save the whole environment.
If you need examples I can provide them.

[lfarv]

@swhite2401 : Yes, I'm interested in example files (just to make sure I do not break things you had so much difficulties to set up)