Skip to content

fixed asset download oom issue#272

Draft
shafeeqd959 wants to merge 2 commits into
developmentfrom
fix/DX-9447-asset-download-oom
Draft

fixed asset download oom issue#272
shafeeqd959 wants to merge 2 commits into
developmentfrom
fix/DX-9447-asset-download-oom

Conversation

@shafeeqd959

@shafeeqd959 shafeeqd959 commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • downloadAssets() resolved each asset's download the instant response.pipe(writeStream) was called, not when the file
    finished writing to disk. This made the downloadLimit (default 5) concurrency gate ineffective — new batches kicked off as
    fast as HTTP responses arrived, while writes queued up unbounded, exhausting the heap on large asset libraries (FATAL
    ERROR: ... JavaScript heap out of memory).
  • Made onSuccess async and awaited stream.finished(assetWriterStream) (via promisify) before resolving, so each download
    promise only settles once the file write actually completes. This makes downloadLimit throttle real in-flight transfers as
    intended.
  • Consolidated the separate assetWriterStream.on('error', ...) handler into the same try/catch around finished().

Testing

  • Added a regression test (test/unit/export/modules/assets.test.ts) that drives onSuccess with a controllable PassThrough
    stream and asserts the download promise doesn't settle until the write stream ends — fails on pre-fix code, passes with
    the fix.
  • Full contentstack-export unit suite: 483 passing, no regressions.
  • tsc --noEmit clean.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

🔒 Security Scan Results

ℹ️ Note: Only vulnerabilities with available fixes (upgrades or patches) are counted toward thresholds.

Check Type Count (with fixes) Without fixes Threshold Result
🔴 Critical Severity 0 0 10 ✅ Passed
🟠 High Severity 54 70 25 ❌ Failed
🟡 Medium Severity 0 5 500 ✅ Passed
🔵 Low Severity 0 0 1000 ✅ Passed

⏱️ SLA Breach Summary

⚠️ Warning: The following vulnerabilities have exceeded their SLA thresholds (days since publication).

Severity Breaches (with fixes) Breaches (no fixes) SLA Threshold (with/no fixes) Status
🔴 Critical 0 0 15 / 30 days ✅ Passed
🟠 High 0 0 30 / 120 days ✅ Passed
🟡 Medium 0 1 90 / 365 days ⚠️ Warning
🔵 Low 0 0 180 / 365 days ✅ Passed

ℹ️ Vulnerabilities Without Available Fixes (Informational Only)

The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:

  • Critical without fixes: 0
  • High without fixes: 70
  • Medium without fixes: 5
  • Low without fixes: 0

❌ BUILD FAILED - Security checks failed

Please review and fix the security vulnerabilities before merging.

1 similar comment
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

🔒 Security Scan Results

ℹ️ Note: Only vulnerabilities with available fixes (upgrades or patches) are counted toward thresholds.

Check Type Count (with fixes) Without fixes Threshold Result
🔴 Critical Severity 0 0 10 ✅ Passed
🟠 High Severity 54 70 25 ❌ Failed
🟡 Medium Severity 0 5 500 ✅ Passed
🔵 Low Severity 0 0 1000 ✅ Passed

⏱️ SLA Breach Summary

⚠️ Warning: The following vulnerabilities have exceeded their SLA thresholds (days since publication).

Severity Breaches (with fixes) Breaches (no fixes) SLA Threshold (with/no fixes) Status
🔴 Critical 0 0 15 / 30 days ✅ Passed
🟠 High 0 0 30 / 120 days ✅ Passed
🟡 Medium 0 1 90 / 365 days ⚠️ Warning
🔵 Low 0 0 180 / 365 days ✅ Passed

ℹ️ Vulnerabilities Without Available Fixes (Informational Only)

The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:

  • Critical without fixes: 0
  • High without fixes: 70
  • Medium without fixes: 5
  • Low without fixes: 0

❌ BUILD FAILED - Security checks failed

Please review and fix the security vulnerabilities before merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant