Skip to content

Adds support for writing binary Ion 1.1 data. #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tgregg wants to merge 1 commit into
base: 2.18-ion-11
Choose a base branch
from
Open

Adds support for writing binary Ion 1.1 data. #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ion/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ tree model)
<dependency>
<groupId>com.amazon.ion</groupId>
<artifactId>ion-java</artifactId>
<version>1.11.7</version>
<version>1.11.8-SNAPSHOT</version>
</dependency>

<!-- Extends Jackson core, databind -->
Expand Down
18 changes: 14 additions & 4 deletions ion/src/main/java/com/fasterxml/jackson/dataformat/ion/IonFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
import java.io.Reader;
import java.io.Writer;

import com.amazon.ion.IonEncodingVersion;
import com.fasterxml.jackson.core.JsonEncoding;
import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonGenerator;
Expand Down Expand Up @@ -462,7 +463,11 @@ protected IonGenerator _createGenerator(OutputStream out, JsonEncoding enc, bool
// Binary writers are simpler: no alternate encodings
if (createBinaryWriters()) {
ctxt.setEncoding(enc);
ion = _system.newBinaryWriter(out);
if (IonGenerator.Feature.ION_VERSION_1_1.enabledIn(_ionGeneratorFeatures)) {
ion = IonEncodingVersion.ION_1_1.binaryWriterBuilder().build(out);
Copy link
Copy Markdown

@popematt popematt May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing the answer is "no", but does the IonObjectMapper allow you to provide your own IonWriterBuilder either when configuring the mapper or writing a value?

Copy link
Copy Markdown
Owner Author

@tgregg tgregg May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can achieve this in a few ways:

  1. You can create an IonObjectMapper using an IonFactory. You can provide an IonSystem to an IonFactory. You can provide text or binary writer builders to an IonSystem.
  2. You can provide an IonWriter instance to an IonGenerator or to IonObjectMapper.writeValue.

} else {
ion = _system.newBinaryWriter(out);
}
dst = out;
} else {
if (enc != JsonEncoding.UTF8) { // not sure if non-UTF-8 encodings would be legal...
Expand All @@ -472,9 +477,14 @@ protected IonGenerator _createGenerator(OutputStream out, JsonEncoding enc, bool
// In practice we seem to be better off using Jackson's efficient buffering encoder

ctxt.setEncoding(enc);
Writer w = new UTF8Writer(ctxt, out);
ion = _system.newTextWriter(w);
dst = w;
if (IonGenerator.Feature.ION_VERSION_1_1.enabledIn(_ionGeneratorFeatures)) {
// TODO use IonEncodingVersion.ION_1_1 to create a text writer builder
throw new UnsupportedOperationException("Cannot yet create an Ion 1.1 text writer.");
} else {
Writer w = new UTF8Writer(ctxt, out);
ion = _system.newTextWriter(w);
dst = w;
}
}
return _createGenerator(ion, true, ctxt, dst);
}
Expand Down
7 changes: 7 additions & 0 deletions ion/src/main/java/com/fasterxml/jackson/dataformat/ion/IonGenerator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,13 @@ public enum Feature implements FormatFeature // since 2.12
* @since 2.12
*/
USE_NATIVE_TYPE_ID(true),

/**
* Whether to use the Ion 1.1 encoding (true) or the Ion 1.0 encoding (false). Disabled by default.
*
* @since 2.18 // TODO change according to the minor version this lands in.
*/
ION_VERSION_1_1(false),
Comment on lines +69 to +75
Copy link
Copy Markdown

@popematt popematt May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to set something up so that instead of it being true/false, we have a "feature" that can accept an Ion Version enum?

Copy link
Copy Markdown
Owner Author

@tgregg tgregg May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what I'd prefer too (see my comment in the PR description), but I haven't seen an example of that yet within Jackson. These enums implement a Jackson core interface so that they can be used in various places in that code, and so far I've only seen them set by booleans. I'll keep looking though.

;

protected final boolean _defaultState;
Expand Down
11 changes: 11 additions & 0 deletions ion/src/test/java/com/fasterxml/jackson/dataformat/ion/DataBindRoundtripTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,17 @@ Bean roundTrip(IonObjectMapper m, Bean bean) throws IOException {
return m.readValue(data, 0, data.length, Bean.class);
}
},
BINARY_1_1 {
@Override
Bean roundTrip(IonObjectMapper m, Bean bean) throws IOException {
m.setCreateBinaryWriters(true);
m.configure(IonGenerator.Feature.ION_VERSION_1_1, true);
ByteArrayOutputStream out = new ByteArrayOutputStream();
m.writeValue(out, bean);
byte[] data = out.toByteArray();
return m.readValue(data, 0, data.length, Bean.class);
}
},
TEXT {
@Override
Bean roundTrip(IonObjectMapper m, Bean bean) throws IOException {
Expand Down
18 changes: 16 additions & 2 deletions ion/src/test/java/com/fasterxml/jackson/dataformat/ion/SimpleIonWriteTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,13 @@ public void testSimpleStructWriteText() throws Exception
gen.close();
}

@Test
public void testSimpleStructWriteBinary() throws Exception
private void testSimpleStructWriteBinary(boolean enableIon11) throws Exception
{
IonFactory f = new IonFactory();
f.setCreateBinaryWriters(true);
if (enableIon11) {
f.configure(IonGenerator.Feature.ION_VERSION_1_1, true);
}
ByteArrayOutputStream bos = new ByteArrayOutputStream();
JsonGenerator gen = f.createGenerator(bos);
_writeSimple(gen);
Expand All @@ -60,6 +62,18 @@ public void testSimpleStructWriteBinary() throws Exception
gen.close();
}

@Test
public void testSimpleStructWriteBinary_1_0() throws Exception
{
testSimpleStructWriteBinary(false);
}

@Test
public void testSimpleStructWriteBinary_1_1() throws Exception
{
testSimpleStructWriteBinary(true);
}

/**
* There were some problems with flushing (or lack thereof), so let's
* see that outputting using an encoding stream also works ok
Expand Down