Skip to content

perf(health-report): skip leave-one-out in base snapshot#61

Merged
aspala merged 1 commit into
mainfrom
perf/base-snapshot-no-loo
Jun 7, 2026
Merged

perf(health-report): skip leave-one-out in base snapshot#61
aspala merged 1 commit into
mainfrom
perf/base-snapshot-no-loo

Conversation

@aspala

@aspala aspala commented Jun 7, 2026

Copy link
Copy Markdown
Member

Was

Der base-snapshot fährt keine Block-Impact-LOO mehr (compute_nodes: false).

Warum

Der base-snapshot speist nur den Metric-Changes-Delta. Delta.compute liest ausschließlich ["codebase"]["aggregate"]nie die per-node block-impact-Nodes. Trotzdem lief die volle LOO über den kompletten base-Tree und die Nodes wurden weggeworfen. Auf großen Repos war das der dominante Posten (auf product-db ~6,5min von ~10min).

Korrektheit

analyzer.ex baut in beiden compute_nodes-Branches eine identische ["codebase"]["aggregate"] — der einzige Unterschied ist der block_impact-Stage. Der Delta kriegt also dieselben Aggregate. Verifiziert:

  • Gemessen gegen ein 1/10-Subset eines großen Repos via --telemetry
  • Nodes processed: 744 → 372 (base-Tree nicht mehr für LOO re-analysiert)
  • Report-Output byte-identisch außer Timestamp

Scope

Eine Stelle: base_snapshot_for_view/4 in cli/health_report.ex. Head-Analyse unverändert. Die head-LOO selbst (Token-subtraktiv) ist ein separater, größerer Hebel — folgt als eigener PR.

The base snapshot feeds only the metric-changes delta, which reads codebase
aggregates (Delta.compute) and never per-node block impact. It was running the
full LOO over the entire base tree and throwing the nodes away — the dominant
cost on large-repo PR runs.

Build the snapshot with compute_nodes: false (and drop node_paths). Aggregates
are byte-identical between the compute_nodes branches, so the report output is
unchanged; only the wasted work is gone.

Verified against a 1/10 subset of a large repo via --telemetry:
nodes processed 744 -> 372 (base tree no longer re-analyzed for LOO),
report output byte-identical except timestamp.
@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Score: C+ → C+ | Δ -1 pts | 0 blocks flagged across 1 files | 1 modified, 0 added

🟠 Code Health: C+ (63/100)

192 files · codeqa-action · 2026-06-07

Combined metric scores use cosine similarity: +1 = metric profile perfectly matches healthy pattern for this behavior, 0 = no signal, −1 = anti-pattern detected. Mapped to 0–100 using breakpoints (approx: ≥0.5→A, ≥0.2→B, ≥0.0→C, ≥−0.3→D, <−0.3→F); actual letter grades use the full 15-step scale.

Metric Changes

Category Base Head Δ
Readability 88.26 97.86 +9.59
Complexity 30.53 41.04 +10.50
Duplication 0.57 0.59 +0.02
Structure 6.17 8.95 +2.78
%%{init: {'theme': 'neutral'}}%%
xychart-beta
    title "Code Health Scores"
    x-axis ["Readability", "Complexity", "Structure", "Duplication", "Naming", "Magic Numbers", "Combined Metrics"]
    y-axis "Score" 0 --> 100
    bar [94, 31, 88, 48, 96, 100, 65]
Loading
Readability       ███████████████████░   94  🟢 A
Complexity        ██████░░░░░░░░░░░░░░   31  🔴 D-
Structure         ██████████████████░░   88  🟢 A-
Duplication       ██████████░░░░░░░░░░   48  🟠 C-
Naming            ███████████████████░   96  🟢 A
Magic Numbers     ████████████████████  100  🟢 A
Combined Metrics  █████████████░░░░░░░   65  🔴 D

@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor
🔍 Top Likely Issues (cosine similarity)

Most negative cosine = file's metric profile best matches this anti-pattern.

Behavior Cosine Score
dependencies.low_coupling -0.56 -12.42
file_structure.single_responsibility -0.52 -12.26
file_structure.line_count_under_300 -0.44 -9.37
code_smells.no_dead_code_after_return -0.42 -33.78
scope_and_assignment.used_only_once -0.42 -14.59
scope_and_assignment.shadowed_by_inner_scope -0.34 -4.85
file_structure.line_length_under_120 -0.30 -8.31
scope_and_assignment.declared_close_to_use -0.26 -1.42
code_smells.no_debug_print_statements -0.24 -2.20
variable_naming.loop_var_is_single_letter -0.23 3.58
🟢 Readability — A (94/100)

Codebase averages: flesch_adapted=97.86, fog_adapted=4.81, avg_tokens_per_line=9.51, avg_line_length=35.68

Metric Value Score
readability.flesch_adapted 97.86 100
readability.fog_adapted 4.81 100
readability.avg_tokens_per_line 9.51 72
readability.avg_line_length 35.68 100
🔴 Complexity — D- (31/100)

Codebase averages: difficulty=41.04, effort=232179.42, volume=4018.73, estimated_bugs=1.34

Metric Value Score
halstead.difficulty 41.04 42
halstead.effort 232179.42 0
halstead.volume 4018.73 46
halstead.estimated_bugs 1.34 47
🟢 Structure — A- (88/100)

Codebase averages: branching_density=0.14, mean_depth=3.86, avg_function_lines=8.27, max_depth=9.20, max_function_lines=19.85, variance=6.86, avg_param_count=1.15, max_param_count=2.04

Metric Value Score
branching.branching_density 0.14 76
indentation.mean_depth 3.86 88
function_metrics.avg_function_lines 8.27 89
indentation.max_depth 9.20 87
function_metrics.max_function_lines 19.85 100
indentation.variance 6.86 100
function_metrics.avg_param_count 1.15 100
function_metrics.max_param_count 2.04 100
🟠 Duplication — C- (48/100)

Codebase averages: redundancy=0.59, bigram_repetition_rate=0.54, trigram_repetition_rate=0.37

Metric Value Score
compression.redundancy 0.59 58
ngram.bigram_repetition_rate 0.54 38
ngram.trigram_repetition_rate 0.37 40
🟢 Naming — A (96/100)

Codebase averages: entropy=0.89, mean=6.65, variance=18.81, avg_sub_words_per_id=1.17

Metric Value Score
casing_entropy.entropy 0.89 100
identifier_length_variance.mean 6.65 100
identifier_length_variance.variance 18.81 85
readability.avg_sub_words_per_id 1.17 100
🟢 Magic Numbers — A (100/100)

Codebase averages: density=0.00

Metric Value Score
magic_number_density.density 0.00 100
🔴 Combined Metrics — D (65/100)
Category Score Grade
Code Smells 25 🔴 D-
Consistency 81 🟡 B+
Dependencies 19 🔴 E+
Documentation 83 🟡 B+
Error Handling 92 🟢 A-
File Structure 48 🟠 C-
Function Design 81 🟡 B+
Naming Conventions 90 🟢 A-
Scope And Assignment 28 🔴 D-
Testing 83 🟡 B+
Type And Value 89 🟢 A-
Variable Naming 74 🟡 B
🔴 Code Smells — D- (25/100)

Cosine similarity scores for 1 behaviors.

Behavior Cosine Score Grade
no_dead_code_after_return -0.42 25 D-
🟡 Consistency — B+ (81/100)

Cosine similarity scores for 1 behaviors.

Behavior Cosine Score Grade
consistent_function_style 0.37 81 B+
🔴 Dependencies — E+ (19/100)

Cosine similarity scores for 1 behaviors.

Behavior Cosine Score Grade
low_coupling -0.56 19 E+
🟡 Documentation — B+ (83/100)

Cosine similarity scores for 3 behaviors.

Behavior Cosine Score Grade
file_has_module_docstring 0.30 77 B
function_has_docstring 0.45 86 A-
docstring_is_nonempty 0.45 87 A-
🟢 Error Handling — A- (92/100)

Cosine similarity scores for 3 behaviors.

Behavior Cosine Score Grade
error_message_is_descriptive 0.52 90 A-
does_not_swallow_errors 0.60 92 A-
returns_typed_error 0.69 94 A
🟠 File Structure — C- (48/100)

Cosine similarity scores for 5 behaviors.

Behavior Cosine Score Grade
single_responsibility -0.52 21 E+
line_count_under_300 -0.44 24 E+
line_length_under_120 -0.30 30 D-
has_consistent_indentation 0.26 74 B
no_magic_numbers 0.57 91 A-
🟡 Function Design — B+ (81/100)

Cosine similarity scores for 3 behaviors.

Behavior Cosine Score Grade
is_less_than_20_lines 0.33 79 B+
no_magic_numbers 0.38 82 B+
has_verb_in_name 0.40 83 B+
🟢 Naming Conventions — A- (90/100)

Cosine similarity scores for 1 behaviors.

Behavior Cosine Score Grade
function_name_is_not_single_word 0.50 90 A-
🔴 Scope And Assignment — D- (28/100)

Cosine similarity scores for 3 behaviors.

Behavior Cosine Score Grade
used_only_once -0.42 25 D-
shadowed_by_inner_scope -0.34 28 D-
declared_close_to_use -0.26 32 D-
🟡 Testing — B+ (83/100)

Cosine similarity scores for 2 behaviors.

Behavior Cosine Score Grade
test_single_concept 0.27 74 B
test_name_describes_behavior 0.53 91 A-
🟢 Type And Value — A- (89/100)

Cosine similarity scores for 1 behaviors.

Behavior Cosine Score Grade
hardcoded_url_or_path 0.49 89 A-
🟡 Variable Naming — B (74/100)

Cosine similarity scores for 1 behaviors.

Behavior Cosine Score Grade
name_is_generic 0.26 74 B

@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

kind: refactoring-tasks
path: /home/runner/work/codeqa-action/codeqa-action
timestamp: 2026-06-07T11:14:16.309172Z
overall_grade: C+
overall_score: 63
task_count: 0
critical: 0
high: 0
instructions: >-
Address the tasks below in order of severity (critical first).
After each fix, run the project's test suite and confirm it passes
before moving on.

No critical or high-severity blocks need attention. ✅

@aspala aspala merged commit 02fe10a into main Jun 7, 2026
8 checks passed
@aspala aspala deleted the perf/base-snapshot-no-loo branch June 7, 2026 11:15
aspala added a commit that referenced this pull request Jun 7, 2026
After #61 removed leave-one-out from the base snapshot, an observed v1.8 run on
a 714-file repo still spent ~6min there. Telemetry pinned it: the base snapshot
was running run_codebase_metrics — dominated by near_duplicate_blocks_codebase,
which is O(files^2)-ish (17.9s for 373 files alone). Like the LOO nodes, that
output is never read: codebase metrics sit beside the aggregate, and the base
snapshot feeds only Delta.compute, which reads ['codebase']['aggregate'].

Add a skip_codebase_metrics opt and set it for the base snapshot. The aggregate
(all the delta needs) is still built from every file.

Proven identical: analyze_codebase(compute_nodes: false) with and without the
skip yields a byte-equal ['codebase']['aggregate'] on position-db/lib; only the
unread near_duplicate/similarity keys drop.

Measured locally against the real position-db (714 base files): the base
snapshot's analyze phase dropped from ~6min to ~24s. 921 tests green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant