Skip to content

src: fix two encodeInto() bugs that reject input that fits - #65998

Open
agape1225 wants to merge 1 commit into
nodejs:mainfrom
agape1225:encoding-fix-encodeinto-underfill
Open

src: fix two encodeInto() bugs that reject input that fits#65998
agape1225 wants to merge 1 commit into
nodejs:mainfrom
agape1225:encoding-fix-encodeinto-underfill

Conversation

@agape1225

Copy link
Copy Markdown
Contributor

TextEncoder.encodeInto() could report that a code point does not fit in the destination Uint8Array even when it does.

  1. simpleUtfEncodingLength() used 0x400 as the boundary between 2-byte and 3-byte UTF-8 encodings, but the correct boundary is 0x800: code points in [0x80, 0x800) need 2 bytes in UTF-8, and only code points

    = 0x800 need 3.

  2. The same function is called with a raw char from the Latin1 (one-byte string) code path. char is signed on some platforms, so a byte >= 0x80 gets sign-extended to a large uint16_t value instead of the intended code point, which also made encodeInto() behave differently for the same prefix depending on whether the rest of the source string forced V8 to represent it as one-byte (Latin1) or two-byte (UTF-16) internally.

Fixes: #65994

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Sep 12, 2026
TextEncoder.encodeInto() could report that a code point does not fit
in the destination Uint8Array even when it does.

1. simpleUtfEncodingLength() used 0x400 as the boundary between 2-byte
   and 3-byte UTF-8 encodings, but the correct boundary is 0x800: code
   points in [0x80, 0x800) need 2 bytes in UTF-8, and only code points
   >= 0x800 need 3.

2. The same function is called with a raw `char` from the Latin1
   (one-byte string) code path. `char` is signed on some platforms, so
   a byte >= 0x80 gets sign-extended to a large uint16_t value instead
   of the intended code point, which also made encodeInto() behave
   differently for the same prefix depending on whether the rest of
   the source string forced V8 to represent it as one-byte (Latin1) or
   two-byte (UTF-16) internally.

Fixes: nodejs#65994

Signed-off-by: agape1225 <49804691+agape1225@users.noreply.github.com>
@agape1225
agape1225 force-pushed the encoding-fix-encodeinto-underfill branch from afc9088 to 4e69ac9 Compare September 12, 2026 07:00
@agape1225 agape1225 changed the title encoding: fix two encodeInto() bugs that reject input that fits src: fix two encodeInto() bugs that reject input that fits Sep 12, 2026
@codecov

codecov Bot commented Sep 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 89.99%. Comparing base (565f69f) to head (4e69ac9).
⚠️ Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
src/encoding_binding.cc 66.66% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #65998      +/-   ##
==========================================
- Coverage   89.99%   89.99%   -0.01%     
==========================================
  Files         784      784              
  Lines      268410   268411       +1     
  Branches    51124    51131       +7     
==========================================
- Hits       241562   241561       -1     
+ Misses      17385    17366      -19     
- Partials     9463     9484      +21     
Files with missing lines Coverage Δ
src/encoding_binding.cc 76.92% <66.66%> (+15.81%) ⬆️

... and 29 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TextEncoder.encodeInto() underfills the destination for some non-ASCII text

2 participants