Skip to content

Keep WordUtils.abbreviate from splitting a surrogate pair#761

Draft
vjsai wants to merge 1 commit into
apache:masterfrom
vjsai:fix/wordutils-abbreviate-surrogate-pair
Draft

Keep WordUtils.abbreviate from splitting a surrogate pair#761
vjsai wants to merge 1 commit into
apache:masterfrom
vjsai:fix/wordutils-abbreviate-surrogate-pair

Conversation

@vjsai

@vjsai vjsai commented Jul 11, 2026

Copy link
Copy Markdown

WordUtils.abbreviate() slices the input at raw char offsets, so an upper limit that lands between a high surrogate and its trailing low surrogate leaves an unpaired surrogate at the end of the result.

Reproduction

WordUtils.abbreviate("😀😀😀", 0, 3, "");

The string is 6 chars long and contains no space, so abbreviate appends str[0, 3) — which is one whole emoji plus the high surrogate of the second one.

  • Before: "😀\uD83D" — one emoji followed by a lone high surrogate (renders as 😀?)
  • After: "😀""😀"

Both call sites are affected: the no-space-found branch (cut at upper) and the space-found branch (cut at min(index, upper)).

Fix

When the cut would split a pair, back it off by one char so the whole pair is dropped, and the result stays within the requested upper limit. This is the same rule the surrounding code already applies elsewhere — WordUtils.wrap() keeps a pair whole at a hard break, and StringUtils.abbreviate() in Commons Lang backs its head cut off a pair boundary via a splitsSurrogatePair helper. This PR adds the same private helper to WordUtils, so the class now handles surrogate pairs consistently across wrap, initials, and abbreviate.

This is a follow-on to the same class of fix already applied in this repo to WordUtils.wrap (#755) and TextStringBuilder.reverse (#756); abbreviate was missed by that sweep.

Test

WordUtilsTest.testAbbreviateSurrogatePairs() covers a cut inside a pair (with and without appendToEnd), a cut short of a space, and a limit that already falls between two pairs (which must be left alone).

The test fails before the change and passes after:

[ERROR] WordUtilsTest.testAbbreviateSurrogatePairs:92 expected: <😀> but was: <😀?>
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0

After the fix, the default mvn goal is green on JDK 17 (clean verify apache-rat:check japicmp:cmp checkstyle:check pmd:check spotbugs:check javadoc:javadoc):

[INFO] Tests run: 1889, Failures: 0, Errors: 0, Skipped: 3
[INFO] BUILD SUCCESS

Behavior for strings without surrogate pairs is unchanged, and the change is binary compatible (the new helper is private) — japicmp reports no issues.


  • Read the contribution guidelines for this project.
  • Read the ASF Generative Tooling Guidance if you use Artificial Intelligence (AI).
  • I used AI to create any part of, or all of, this pull request. Which AI tool was used to create this pull request, and to what extent did it contribute?
    Claude Code (Anthropic) was used to help locate the bug, draft the fix and the test, and run the build. I reviewed and verified the change myself: I confirmed the failing assertion before the fix, confirmed the full default-goal build passes after it, and checked the fix against the existing surrogate-pair handling in WordUtils.wrap and StringUtils.abbreviate. I understand and take responsibility for the contents of this PR.
  • Run a successful build using the default Maven goal with mvn; that's mvn on the command line by itself.
  • Write unit tests that match behavioral changes, where the tests fail if the changes to the runtime are not applied.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Each commit in the pull request should have a meaningful subject line and body.

WordUtils.abbreviate() cuts the string at raw char offsets, so an upper
limit that lands between a high surrogate and its trailing low surrogate
leaves an unpaired surrogate at the end of the result.

For example, WordUtils.abbreviate("😀😀😀", 0, 3, "")
returned one emoji followed by a lone high surrogate instead of just the
emoji.

Back the cut off by one when it would split a pair, in both the
no-space-found and the space-found branches, mirroring the existing
handling in WordUtils.wrap() and StringUtils.abbreviate().
@ecki

ecki commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

We should document that behavior: This does count characters not code units, it might return less characters to avoid Splitting surrogates, but it does not measure the length in codepoints (or something like that?).

maybe add examples to the JavaDoc?

Since you mentioned emojis, what about extended grapheme clusters?

having said that, a byte or codepoint length semantic might be interesting as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants