Skip to content

perf(block-impact): byte-exact block slice enables subtractive LOO#62

Merged
aspala merged 1 commit into
mainfrom
perf/loo-original-slice
Jun 7, 2026
Merged

perf(block-impact): byte-exact block slice enables subtractive LOO#62
aspala merged 1 commit into
mainfrom
perf/loo-original-slice

Conversation

@aspala

@aspala aspala commented Jun 7, 2026

Copy link
Copy Markdown
Member

Was

Behebt die Wurzel des gescheiterten subtraktiven LOO-Pfads und macht separator_counts + vowel_density wieder subtraktiv (korrekt).

Wurzelproblem

Der LOO-Pfad baute file-minus-block aus normalisierten structural tokens — die kollabieren inter-token-Whitespace, runden Indent auf 2-Space-Units und mappen Non-ASCII auf Spaces. Der reconstructed string wich also vom Source ab, und jede subtraktive Metrik (baseline ⊖ block) gegen eine Original-File-Baseline driftete mit. Das war der latente separator_counts-Bug (0/50 matches), der durch Entfernen von analyze_loo/2 umgangen worden war.

Fix

slice_without_original/2 schneidet den Block aus den Original-Bytes via line+col des ersten/letzten Tokens. block_content ist damit der verbatim Source-Span, reconstructed das Original minus diesem Span. Schnitte fallen auf Token-Grenzen → counts subtrahieren exakt.

  • Vertrag: analyze_loo/2 nimmt einen geteilten Block-FileContext (einmal pro Node gebaut) statt eines rohen Strings → subtraktive Metriken extrahieren identifiers/tokens über dieselbe Pipeline wie die Baseline.
  • separator_counts + vowel_density wieder subtraktiv, jetzt korrekt: ~23ms / ~7ms pro Node → ~0,1ms. Verifiziert vom subtractive_loo-Goldfile-Guard gegen reale verschachtelte Sample-Blocks.
  • Bewusst NICHT subtraktiv (re-analyze-Fallback): punctuation-densities, halstead n1/n2, ngram, line-basierte — set-basiert oder blockgrenz-kontextabhängig, subtrahieren nicht exakt.

Messung (1/10-Subset, --telemetry)

  • analyze_file pro Node: 196ms → 180ms (~8% head-LOO)
  • Report-Output byte-identisch außer Timestamp (kein Drift)
  • 921 Tests grün, Credo strict clean, Dialyzer pass

Ehrlicher Scope

Der subtraktive Hebel ist physisch auf ~8% begrenzt — die großen Posten (ctx-build, punctuation, halstead, ngram) sind mathematisch nicht subtraktiv. Dieser PR ist der saubere, korrekte Anteil + das Fundament (slice_without_original, geteilter block_ctx) für spätere Metriken (z.B. entropy via baseline-count-Cache). Der echte Laufzeit-Win kam aus #61 (base-snapshot).

The leave-one-out path rebuilt the file-minus-block from normalized structural
tokens, which collapse inter-token whitespace, round indentation to 2-space
units, and map non-ASCII to spaces. So the reconstructed string diverged from
the source, and any subtractive metric (baseline minus block) computed against
an original-file baseline diverged too — the latent separator_counts bug
(0/50 matches), which had been worked around by removing its analyze_loo/2.

Fix the root cause: slice_without_original/2 cuts the block out of the original
bytes using the first/last token's line+col, so block_content is the verbatim
source span and reconstructed is the original file minus that span. Cuts fall on
token boundaries, so counts subtract exactly.

- analyze_loo/2 contract takes a shared block FileContext (built once per node)
  instead of a raw string, so subtractive metrics extract identifiers/tokens via
  the same pipeline as the baseline.
- separator_counts and vowel_density go subtractive again, now correct: ~23ms
  and ~7ms per node drop to ~0.1ms. Verified by the subtractive_loo goldfile
  guard against real nested sample blocks.
- The big LOO costs (punctuation densities, halstead n1/n2, ngram, line-based)
  stay on the re-analyze fallback: they are set-based or context-dependent at
  block boundaries and cannot subtract exactly.

Measured on a 1/10 subset of a large repo via --telemetry: analyze_file per
node 196ms -> 180ms; report output byte-identical except timestamp.
@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Score: C+ → C+ | Δ -1 pts | 0 blocks flagged across 8 files | 7 modified, 1 added

🟠 Code Health: C+ (63/100)

192 files · codeqa-action · 2026-06-07

Combined metric scores use cosine similarity: +1 = metric profile perfectly matches healthy pattern for this behavior, 0 = no signal, −1 = anti-pattern detected. Mapped to 0–100 using breakpoints (approx: ≥0.5→A, ≥0.2→B, ≥0.0→C, ≥−0.3→D, <−0.3→F); actual letter grades use the full 15-step scale.

Metric Changes

Category Base Head Δ
Readability 88.26 97.81 +9.54
Complexity 30.54 41.25 +10.71
Duplication 0.57 0.59 +0.02
Structure 6.17 9.02 +2.85
%%{init: {'theme': 'neutral'}}%%
xychart-beta
    title "Code Health Scores"
    x-axis ["Readability", "Complexity", "Structure", "Duplication", "Naming", "Magic Numbers", "Combined Metrics"]
    y-axis "Score" 0 --> 100
    bar [94, 31, 88, 48, 96, 100, 65]
Loading
Readability       ███████████████████░   94  🟢 A
Complexity        ██████░░░░░░░░░░░░░░   31  🔴 D-
Structure         ██████████████████░░   88  🟢 A-
Duplication       ██████████░░░░░░░░░░   48  🟠 C-
Naming            ███████████████████░   96  🟢 A
Magic Numbers     ████████████████████  100  🟢 A
Combined Metrics  █████████████░░░░░░░   65  🔴 D

@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor
🔍 Top Likely Issues (cosine similarity)

Most negative cosine = file's metric profile best matches this anti-pattern.

Behavior Cosine Score
dependencies.low_coupling -0.56 -12.51
file_structure.single_responsibility -0.52 -12.31
file_structure.line_count_under_300 -0.44 -9.39
code_smells.no_dead_code_after_return -0.43 -33.98
scope_and_assignment.used_only_once -0.42 -14.59
scope_and_assignment.shadowed_by_inner_scope -0.34 -4.90
file_structure.line_length_under_120 -0.30 -8.31
scope_and_assignment.declared_close_to_use -0.26 -1.51
code_smells.no_debug_print_statements -0.24 -2.22
variable_naming.loop_var_is_single_letter -0.23 3.55
🟢 Readability — A (94/100)

Codebase averages: flesch_adapted=97.81, fog_adapted=4.82, avg_tokens_per_line=9.52, avg_line_length=35.74

Metric Value Score
readability.flesch_adapted 97.81 100
readability.fog_adapted 4.82 100
readability.avg_tokens_per_line 9.52 72
readability.avg_line_length 35.74 100
🔴 Complexity — D- (31/100)

Codebase averages: difficulty=41.25, effort=234200.38, volume=4045.56, estimated_bugs=1.35

Metric Value Score
halstead.difficulty 41.25 42
halstead.effort 234200.38 0
halstead.volume 4045.56 46
halstead.estimated_bugs 1.35 47
🟢 Structure — A- (88/100)

Codebase averages: branching_density=0.14, mean_depth=3.86, avg_function_lines=8.30, max_depth=9.21, max_function_lines=19.95, variance=6.85, avg_param_count=1.16, max_param_count=2.05

Metric Value Score
branching.branching_density 0.14 76
indentation.mean_depth 3.86 88
function_metrics.avg_function_lines 8.30 89
indentation.max_depth 9.21 87
function_metrics.max_function_lines 19.95 100
indentation.variance 6.85 100
function_metrics.avg_param_count 1.16 100
function_metrics.max_param_count 2.05 100
🟠 Duplication — C- (48/100)

Codebase averages: redundancy=0.59, bigram_repetition_rate=0.54, trigram_repetition_rate=0.37

Metric Value Score
compression.redundancy 0.59 58
ngram.bigram_repetition_rate 0.54 38
ngram.trigram_repetition_rate 0.37 40
🟢 Naming — A (96/100)

Codebase averages: entropy=0.89, mean=6.65, variance=18.82, avg_sub_words_per_id=1.17

Metric Value Score
casing_entropy.entropy 0.89 100
identifier_length_variance.mean 6.65 100
identifier_length_variance.variance 18.82 85
readability.avg_sub_words_per_id 1.17 100
🟢 Magic Numbers — A (100/100)

Codebase averages: density=0.00

Metric Value Score
magic_number_density.density 0.00 100
🔴 Combined Metrics — D (65/100)
Category Score Grade
Code Smells 25 🔴 D-
Consistency 81 🟡 B+
Dependencies 19 🔴 E+
Documentation 83 🟡 B+
Error Handling 92 🟢 A-
File Structure 48 🟠 C-
Function Design 81 🟡 B+
Naming Conventions 90 🟢 A-
Scope And Assignment 28 🔴 D-
Testing 83 🟡 B+
Type And Value 89 🟢 A-
Variable Naming 74 🟡 B
🔴 Code Smells — D- (25/100)

Cosine similarity scores for 1 behaviors.

Behavior Cosine Score Grade
no_dead_code_after_return -0.43 25 D-
🟡 Consistency — B+ (81/100)

Cosine similarity scores for 1 behaviors.

Behavior Cosine Score Grade
consistent_function_style 0.37 81 B+
🔴 Dependencies — E+ (19/100)

Cosine similarity scores for 1 behaviors.

Behavior Cosine Score Grade
low_coupling -0.56 19 E+
🟡 Documentation — B+ (83/100)

Cosine similarity scores for 3 behaviors.

Behavior Cosine Score Grade
file_has_module_docstring 0.30 77 B
function_has_docstring 0.45 86 A-
docstring_is_nonempty 0.45 87 A-
🟢 Error Handling — A- (92/100)

Cosine similarity scores for 3 behaviors.

Behavior Cosine Score Grade
error_message_is_descriptive 0.52 90 A-
does_not_swallow_errors 0.60 92 A-
returns_typed_error 0.69 94 A
🟠 File Structure — C- (48/100)

Cosine similarity scores for 5 behaviors.

Behavior Cosine Score Grade
single_responsibility -0.52 21 E+
line_count_under_300 -0.44 24 E+
line_length_under_120 -0.30 30 D-
has_consistent_indentation 0.26 74 B
no_magic_numbers 0.57 91 A-
🟡 Function Design — B+ (81/100)

Cosine similarity scores for 3 behaviors.

Behavior Cosine Score Grade
is_less_than_20_lines 0.33 79 B+
no_magic_numbers 0.38 82 B+
has_verb_in_name 0.40 83 B+
🟢 Naming Conventions — A- (90/100)

Cosine similarity scores for 1 behaviors.

Behavior Cosine Score Grade
function_name_is_not_single_word 0.50 90 A-
🔴 Scope And Assignment — D- (28/100)

Cosine similarity scores for 3 behaviors.

Behavior Cosine Score Grade
used_only_once -0.42 25 D-
shadowed_by_inner_scope -0.34 28 D-
declared_close_to_use -0.26 32 D-
🟡 Testing — B+ (83/100)

Cosine similarity scores for 2 behaviors.

Behavior Cosine Score Grade
test_single_concept 0.27 74 B
test_name_describes_behavior 0.53 91 A-
🟢 Type And Value — A- (89/100)

Cosine similarity scores for 1 behaviors.

Behavior Cosine Score Grade
hardcoded_url_or_path 0.49 89 A-
🟡 Variable Naming — B (74/100)

Cosine similarity scores for 1 behaviors.

Behavior Cosine Score Grade
name_is_generic 0.26 74 B

@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

kind: refactoring-tasks
path: /home/runner/work/codeqa-action/codeqa-action
timestamp: 2026-06-07T11:46:02.458052Z
overall_grade: C+
overall_score: 63
task_count: 0
critical: 0
high: 0
instructions: >-
Address the tasks below in order of severity (critical first).
After each fix, run the project's test suite and confirm it passes
before moving on.

No critical or high-severity blocks need attention. ✅

@aspala aspala merged commit 821a884 into main Jun 7, 2026
8 checks passed
@aspala aspala deleted the perf/loo-original-slice branch June 7, 2026 11:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant