Add native importer for Encore (.enc) files

[manolo]

Resolves: #24341

What this adds

A native importer for Encore .enc files, covering the formats written by Encore 2.x through 5.x (v0xA6, v0xC2, v0xC4 and the macOS SCO5 variant). Encrypted ZBOT files are detected and reported with a clear error suggesting a re-save from Encore 5. .mus files are a much older variant, absent from the entire test corpus, and are out of scope.

Particular care went into the latest format (Encore 5.0.x), which is the best supported. If a file written by an older version does not import cleanly, opening it in Encore and saving it again as 5.0.x usually resolves it.

Encore is very permissive about what it writes: a measure's contents frequently do not add up to its nominal time signature. To avoid producing a corrupt score, the importer applies a set of safeguards that reconcile these mismatches. Because there is no single correct way to handle an over-filled or under-filled measure, the import preferences expose options the user can adjust to get better results or to suit a particular score when measures come out shorter or longer than expected.

Features

• Notes, rests and durations, including dotted values
• Tuplets, including nested tuplets
• Ties and slurs
• Hairpins and ottava lines
• Grace notes and cue notes
• Multi-verse lyrics
• Chord symbols, with fretboard diagrams where the chord is known
• Repeats, voltas and jumps
• Clefs, including mid-measure clef changes, key signatures and time signatures
• Tempo marks and dynamics
• Articulations, ornaments, tremolos and fingerings
• Staff text and titles
• Instrument recognition, with template and drumset matching, and per-instrument staff size
• Page size and margins
• An import preferences section for the options above (voice merging, overfull-measure handling, and more)

Design

The importer has two layers with a clean boundary: a parser that reads the binary format into plain structs, and an importer that emits the engraving model from them. The binary format is documented in src/importexport/encore/ENCORE_FORMAT.md (building on the prior work of enc2ly and Enc2MusicXML, credited there).

The implementation decisions are written up in a companion document, ENCORE_IMPORTER.md. It is attached to this PR for reviewer reference and, together with the fixture generator mentioned below, is intentionally not committed to the branch: both are long and not needed in the long term.

Testing

570 regression and unit tests, plus validation against a large corpus of real-world files. In a representative 400-file sample, 99% of the importable files import cleanly; encrypted and non-Encore files are correctly detected and declined. The committed .enc fixtures were produced by a Python generator gen_enc_test_files.py, attached to this PR for reference rather than committed.

Builds for testers

Download links for the CI builds of the latest push (a GitHub account is required to download artifacts). This table is updated on every new build; current build: 2cf06282.

Platform	Build
Windows	download
macOS	download
Linux (x86_64)	download

Each link downloads a .zip. These are unsigned development builds, so the operating system will warn on first launch; the steps below explain how to get past that.

Windows

1. Unzip the download. Inside is a portable build packaged as MuseScore-Studio-<version>-x86_64.7z (there is no installer).
2. Extract that .7z with 7-Zip, or any tool that reads 7z, to a folder of your choice.
3. Open the extracted folder and run bin\MuseScoreStudio5.exe. If SmartScreen shows "Windows protected your PC", click "More info" then "Run anyway". Nothing is installed; delete the folder to remove it.

macOS

1. Unzip the download to get MuseScore-Studio-<version>.dmg, open it, and drag the app to Applications. The app is named like MuseScore <version> Development.
2. Because the build is not notarized, the first launch is blocked. Right-click the app in Applications, choose "Open", and confirm in the dialog. If it still refuses, clear the quarantine flag in Terminal:
   xattr -dr com.apple.quarantine "/Applications/MuseScore*Development.app"
3. Launch it normally afterwards.

Linux

1. Unzip the download to get MuseScore-Studio-<version>-x86_64.AppImage.
2. Make it executable chmod +x MuseScore-Studio-*.AppImage
3. Run it ./MuseScore-Studio-*.AppImage

Acknowledgements

Huge thanks to @pacebes, who beta-tested the importer extensively against real scores and reported most of the issues fixed during the beta round, and to the participants of #24341 for samples and feedback.

Checklist

• I signed the CLA as a regular contributor.
• The title of the PR describes the problem it addresses.
• Each commit's message describes its purpose and effects, and references the issue it resolves. If changes are extensive, there is a sequence of easily reviewable commits.
• The code in the PR follows the coding rules.
• I understand all aspects of the code I'm contributing and I'm able to explain it if requested.
• The code compiles and runs on my machine, preferably after each commit individually. I have manually tested and verified that my changes fulfil their intended purpose.
• No prior attempts to resolve this problem exist, or if they do, I listed them in my PR description and described how I avoided repeating past mistakes.
• There are no unnecessary changes.
• I created a unit test or vtest to verify the changes I made (if applicable).

[pacebes]

Great work @manolo !!.

It has been a pleasure working with you.

[scorster]

How can I help test the Encore (.enc) import feature?

Will Encore score import be available in a MuseScore Studio 4.7.4 nightly ... and/or in an official update?

[manolo]

@scorster The validation process for this pull request automatically generates artifacts for different systems. Please follow the instructions in the "Builds for Testers" section of the first comment in this thread.

Side note: this build is based on MuseScore 5.0, the next major version. Files imported and saved with this build cannot be opened with MuseScore 4.x because the file format has evolved.

[scorster]

This is an outstanding accomplishment Manolo!

Every .enc file I've opened looks credible and playback is fine. However, presently I don't have a functioning version of Encore, so I can't do a real comparison, but I can compare the imported score to a PDF of the score, which sometimes I have on hand.

I have thousands of Encore (enc.) scores, so your work will provide me the immense rewards of accessing work locked up in legacy Encore scores. I'll do more testing in coming days!

@scorster Please follow the instructions in the "Builds for Testers" section of the first comment in this thread.

MacOS Quarantine

As expected I encountered the MacOS quarantine startup issue; unsurprisingly Open With didn't work.

And I was able to remove the quarantine attribute with the Terminal instructions you included in your original post.

MacOS steps

For those on MacOS, the following steps alleviate the need to carefully type the directory path and file name:

• Open Terminal.app
• Paste in:
xattr -dr com.apple.quarantine
• Add a space char
• Locate the installed development version of MuseScore 5 in the MacOS Finder
• Drag that version of MuseScore 5 to the Terminal window This appends the path and file name)
• Press return

Result: The xattr command clears the quarantine attribute from the development version of Encore 5, allowing this version of MuseScore 5 to open. (I performed my dequarantine on MacOS 26.5.)

NOTE: You only have to run the xattr command once for this particular build. However you will likely need to do this with each version you download, until there is an official release of MuseScore 5 which will be notarized and open without issue.

Side note: this build is based on MuseScore 5.0, the next major version. Files imported and saved with this build cannot be opened with MuseScore 4.x because the file format has evolved.

Good point. I'll be sure to use MS5 for testing only. I'll also try export to MusicXML and see if a round trip proves reliable.

One last question. Can you envision backporting your Encore importer to MuseScore 3.7 Evolution?

scorster

[manolo]

@scorster Thanks for testing this, and thanks as well for the helpful tips on installing this specific CI build on macOS.

I'll also try exporting to MusicXML and see if a round trip proves reliable.

That's a good test case, although I think it would mainly demonstrate how well MuseScore's MusicXML exporter and importer preserve the information that MusicXML is able to represent.

Backporting your Encore importer to MuseScore 3.7 Evolution

I haven't tried MuseScore 3.x since moving to MuseScore 4, but a backport should be doable if there is enough interest.

[scorster]

@scorster wrote > > I'll also try exporting to MusicXML and see if a round trip proves reliable.

That's a good test case, although I think it would mainly demonstrate how well MuseScore's MusicXML exporter and importer preserve the information that MusicXML is able to represent.

Sorry if I was unclear. That's what I meant, that I'd check to see if musicXML export/import cycles produce scores with very high similarity accuracy. I've not seen that prove out in any notation app, but Leon Vinken has make some extensive improvements to MuseScore's implementation of MusicXML, so I'm hopeful. (And has advanced TablEdit import too!)

MusicXML would be my only path to bringing my revived Encore scores into MuseScore 3.7. Because currently, like MS4.7.4, MS 3.7 can't open the files I've save from MuseScore Studio 5.

Which led to my backporting question:

[Can you envision] backporting your Encore importer to MuseScore 3.7 Evolution

I haven't tried MuseScore 3.x since moving to MuseScore 4, but a backport should be doable if there is enough interest.

Joaquin Schmitz may have interest in helping with:

      a) backporting your Encore import code into MuseScore 3.7
      b) the option of opening MS5 scores in MuseScore 3.7

https://github.com/Jojo-Schmitz

Again, thanks for this wonderful feature!

scorster