feat(java): add float support by mengnankkkk · Pull Request #3254 · apache/fory

mengnankkkk · 2026-02-03T11:17:48Z

Why?

What does this PR do?

Related issues

Close #3205

Does this PR introduce any user-facing change?

Does this PR introduce any public API change?
Does this PR introduce any binary protocol compatibility change?

Benchmark

java/fory-core/src/main/java/org/apache/fory/serializer/ArraySerializers.java

java/fory-core/src/main/java/org/apache/fory/type/Float16.java

Co-authored-by: Shawn Yang <chaokunyang@apache.org>

...re/src/main/resources/META-INF/native-image/org.apache.fory/fory-core/reflection-config.json

...src/main/resources/META-INF/native-image/org.apache.fory/fory-core/serialization-config.json

...re/src/main/resources/META-INF/native-image/org.apache.fory/fory-core/reflection-config.json

java/fory-core/src/main/java/org/apache/fory/type/DispatchId.java

...re/src/main/resources/META-INF/native-image/org.apache.fory/fory-core/reflection-config.json

java/fory-core/src/test/java/org/apache/fory/serializer/Float16SerializerTest.java

java/fory-core/src/main/java/org/apache/fory/type/DispatchId.java

chaokunyang · 2026-02-26T06:59:10Z

Could you also address those issues?

XLANG support for new Float16 types is broken at runtime.
Float16/Float16List serializers are added (PrimitiveSerializers.java, PrimitiveListSerializers.java), but internal xlang type registration is missing in resolver bootstrap (ClassResolver.java). In xlang mode it falls back to unsupported-class path and fails at deserialize time.
- java/fory-core/src/main/java/org/apache/fory/serializer/PrimitiveSerializers.java:381
- java/fory-core/src/main/java/org/apache/fory/serializer/collection/PrimitiveListSerializers.java:595
- java/fory-core/src/main/java/org/apache/fory/resolver/ClassResolver.java:262
- java/fory-core/src/main/java/org/apache/fory/resolver/ClassResolver.java:295
- java/fory-core/src/main/java/org/apache/fory/resolver/ClassResolver.java:1305
Float16 subnormal decode is numerically wrong.
In float16BitsToFloat, subnormal conversion uses an incorrect shift/exponent formula.
- java/fory-core/src/main/java/org/apache/fory/type/Float16.java:169
- java/fory-core/src/main/java/org/apache/fory/type/Float16.java:173
  Repro: Float16.fromBits((short)0x0001).floatValue() returns 1.7881393E-7, expected 5.9604645E-8 (2^-24).
Float16[] serializer crashes on legal arrays containing null.
The array fast path dereferences element directly (value[i].toBits()), so new Float16[]{Float16.ONE, null} throws NPE.
- java/fory-core/src/main/java/org/apache/fory/serializer/ArraySerializers.java:901
Type-id mapping regression for existing float kinds.
Types.getClassForTypeId now maps FLOAT8 to Float16, and BFLOAT16 mapping was removed (falls through to null).
- java/fory-core/src/main/java/org/apache/fory/type/Types.java:450
- java/fory-core/src/main/java/org/apache/fory/type/Types.java:452
Generated Java equals/hashCode contract can break for float16.
Generator uses equalsValue for FLOAT16 equality, but default hash path remains Objects.hash(...). equalsValue treats +0 and -0 as equal, while Float16.hashCode is bit-based.
- compiler/fory_compiler/generators/java.py:1384
- compiler/fory_compiler/generators/java.py:1467
- java/fory-core/src/main/java/org/apache/fory/type/Float16.java:270
- java/fory-core/src/main/java/org/apache/fory/type/Float16.java:334

java/fory-core/src/main/java/org/apache/fory/serializer/Float16Serializer.java

chaokunyang · 2026-03-05T15:44:35Z

java/fory-core/src/main/java/org/apache/fory/serializer/ArraySerializers.java

+    private void writeNullable(MemoryBuffer buffer, Float16[] value, int length) {
+      int bitmapSize = (length + 7) >>> 3;
+      int payloadSize = Integer.BYTES + bitmapSize + length * 2;
+      int encodedSize = (payloadSize << 1) | NULLABLE_ENCODING_FLAG;


This changes the FLOAT16_ARRAY wire layout when nulls are present, which will break cross-language readers that expect the normal length+payload format for type 53.

In xlang mode, other runtimes decode FLOAT16_ARRAY as plain [byte_length][2-byte elements]. Here we switch to an alternate encoded-size scheme for nullable arrays, so non-Java runtimes will misread the payload. Could we keep FLOAT16_ARRAY encoding stable and handle nullable cases another way (for example, fallback to LIST encoding when null is present)?

chaokunyang · 2026-03-05T15:44:37Z

java/fory-core/src/main/java/org/apache/fory/type/DispatchId.java

-  public static final int EXT_UINT64 = 23;
-  public static final int EXT_VAR_UINT64 = 24;
-  public static final int STRING = 25;
+  public static final int FLOAT16 = 17;


Since FLOAT16 is now a basic dispatch id, FieldSkipper also needs a skip branch for it.

Right now FieldSkipper has no DispatchId.FLOAT16 case. In compatibility reads, skipping an unknown Float16 field will throw Unexpected basic dispatchId instead of advancing by 2 bytes. Please add FLOAT16 to the 2-byte skip branch.

chaokunyang · 2026-03-05T15:44:39Z

java/fory-core/src/main/java/org/apache/fory/serializer/Serializers.java

    resolver.registerInternalSerializer(URI.class, new URISerializer(fory));
    resolver.registerInternalSerializer(Pattern.class, new RegexSerializer(fory));
    resolver.registerInternalSerializer(UUID.class, new UUIDSerializer(fory));
+    resolver.registerInternalSerializer(Float16.class, new Float16Serializer(fory));


Float16 gets registered twice (primitive serializer path and general serializers path).

This overrides the earlier registration and makes it harder to reason about which serializer is intended. It would be cleaner to keep a single registration point for Float16.

chaokunyang · 2026-03-05T15:44:42Z

compiler/fory_compiler/generators/java.py

-                    if kind in (PrimitiveKind.FLOAT32,):
+                    if kind in (PrimitiveKind.FLOAT16,):
+                        comparisons.append(
+                            f"({field_name} == null ? that.{field_name} == null : (that.{field_name} != null && {field_name}.equalsValue(that.{field_name})))"


Generated equals/hashCode for float16 now use numeric semantics, but Float16.equals/hashCode are bitwise.

With this change, +0 and -0 compare equal in generated messages, and NaN is not equal to NaN, while Float16 itself uses raw-bit equality. That mismatch is surprising in user code. Can we align generated message behavior with Float16.equals/hashCode (or make the rule explicit and consistent across runtimes)?

chaokunyang · 2026-03-05T15:44:44Z

java/fory-core/src/main/java/org/apache/fory/collection/Float16List.java

+    return true;
+  }
+
+  public boolean add(float value) {


Float16List primitive methods still allocate Float16 objects on the hot path.

add(float), set(float), and getFloat(int) all go through Float16.valueOf/fromBits, so these paths allocate even though this is a primitive-style container. A static bits conversion helper would keep these methods allocation-free.

mengnankkkk added 3 commits February 2, 2026 14:36

fix: init

46e0e29

fix: remove

a610788

fix: add float16

7fc45b8

mengnankkkk requested a review from chaokunyang as a code owner February 3, 2026 11:17

chaokunyang reviewed Feb 3, 2026

View reviewed changes

java/fory-core/src/main/java/org/apache/fory/serializer/ArraySerializers.java Outdated Show resolved Hide resolved

mengnankkkk added 8 commits February 3, 2026 19:39

fix: add float16serializer

65f6ea3

fix: add float16list

6e37abb

fix: add native

f5ba942

Update native-image.properties

ad18586

fix: add

5cfb49f

Update native-image.properties

2772ff2

Update native-image.properties

f6f38d3

fix: ci

578c05e

chaokunyang reviewed Feb 4, 2026

View reviewed changes

java/fory-core/src/main/java/org/apache/fory/type/Float16.java Outdated Show resolved Hide resolved

Update java/fory-core/src/main/java/org/apache/fory/type/Float16.java

0b98295

Co-authored-by: Shawn Yang <chaokunyang@apache.org>

chaokunyang reviewed Feb 4, 2026

View reviewed changes

...re/src/main/resources/META-INF/native-image/org.apache.fory/fory-core/reflection-config.json Outdated Show resolved Hide resolved

chaokunyang reviewed Feb 4, 2026

View reviewed changes

...src/main/resources/META-INF/native-image/org.apache.fory/fory-core/serialization-config.json Outdated Show resolved Hide resolved

chaokunyang reviewed Feb 4, 2026

View reviewed changes

...re/src/main/resources/META-INF/native-image/org.apache.fory/fory-core/reflection-config.json Outdated Show resolved Hide resolved

mengnankkkk added 4 commits February 4, 2026 22:02

fix: remove something

004663d

Merge branch 'main' into hotfix/add_float16

809ced1

fix: remove

65f2667

fix: ci

602aaed

chaokunyang reviewed Feb 13, 2026

View reviewed changes

java/fory-core/src/main/java/org/apache/fory/type/DispatchId.java Outdated Show resolved Hide resolved

fix

18f899c

chaokunyang reviewed Feb 22, 2026

View reviewed changes

...re/src/main/resources/META-INF/native-image/org.apache.fory/fory-core/reflection-config.json Outdated Show resolved Hide resolved

chaokunyang reviewed Feb 22, 2026

View reviewed changes

java/fory-core/src/test/java/org/apache/fory/serializer/Float16SerializerTest.java Show resolved Hide resolved

chaokunyang reviewed Feb 22, 2026

View reviewed changes

java/fory-core/src/main/java/org/apache/fory/type/DispatchId.java Show resolved Hide resolved

fix: fix codereview

77178dd

fix: from cr

343a6ce

mengnankkkk added 3 commits February 28, 2026 23:36

Merge branch 'main' into hotfix/add_float16

f5ee398

Update Float16Serializer.java

b096c9b

fix: ci

a692296

chaokunyang reviewed Mar 1, 2026

View reviewed changes

java/fory-core/src/main/java/org/apache/fory/serializer/Float16Serializer.java Outdated Show resolved Hide resolved

mengnankkkk added 2 commits March 1, 2026 21:54

Update Float16Serializer.java

ec14a50

Merge branch 'main' into hotfix/add_float16

91e5bbd

chaokunyang reviewed Mar 5, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(java): add float support#3254

feat(java): add float support#3254
mengnankkkk wants to merge 24 commits intoapache:mainfrom
mengnankkkk:hotfix/add_float16

mengnankkkk commented Feb 3, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chaokunyang commented Feb 26, 2026

Uh oh!

Uh oh!

chaokunyang Mar 5, 2026

Uh oh!

chaokunyang Mar 5, 2026

Uh oh!

chaokunyang Mar 5, 2026

Uh oh!

chaokunyang Mar 5, 2026

Uh oh!

chaokunyang Mar 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

mengnankkkk commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why?

What does this PR do?

Related issues

Does this PR introduce any user-facing change?

Benchmark

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chaokunyang commented Feb 26, 2026

Uh oh!

Uh oh!

chaokunyang Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

chaokunyang Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

chaokunyang Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

chaokunyang Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

chaokunyang Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mengnankkkk commented Feb 3, 2026 •

edited

Loading