Skip to content

Remove outdated queries and update missing query references#3451

Open
mabijkerk wants to merge 13 commits into
masterfrom
query-validation
Open

Remove outdated queries and update missing query references#3451
mabijkerk wants to merge 13 commits into
masterfrom
query-validation

Conversation

@mabijkerk

@mabijkerk mabijkerk commented Jun 22, 2026

Copy link
Copy Markdown
Member

Context

Queries can reference other queries: Q(query_key). Sometimes queries are removed or renamed. In this case, you have to do a manual check that they are no longer referenced by another query. There was no automatic check for this.

Implemented changes

I had AI write a spec to check for missing queries referenced in queries. Some commments:

  • There is a spec called renamed_functions_spec, which seems somewhat similar but I don't really understand its purpose. Should we retire that spec @aaccensi?
  • This new spec does not check if a query is still referenced in ETModel, so there is still some manual labour involved - this will be difficult to add I expect?
  • Ideally, after this new spec has been merged we would add another increment that checks whether queries refer to outdated nodes, edges, node groups. Or we can add a spec for orphaned queries.

From the first spec run came 15 failures. These have been addressed in individual commits. Goes with the clean up of the investment table: quintel/etmodel#4735.

Review

Ideally, @aaccensi would check the spec and @kndehaan would check the actions for each query. I have added an Excel for this on OneDrive.

I would like to merge this with a merge commit and not squash the commits - to preserve commit history on the addressed queries.

@mabijkerk mabijkerk requested a review from aaccensi June 22, 2026 17:03
@mabijkerk mabijkerk changed the title Add validation spec for missing gqueries Remove outdated queries and update missing query references Jun 22, 2026
@mabijkerk mabijkerk requested a review from kndehaan June 22, 2026 17:34
This query was no longer used anywhere
This query referred p2h for households which is no longer a technology in the model
Bio-oil in output of industry transformation has been changed to bionaphtha but this query had not yet been updated accordingly
The investment table has been hidden in ETModel for several years because of the cost data export introduction. Thetable can be retired. Query regeneration can still be done on modeling experiments repo
This query is no longer used anywhere and contained references to missing queries
@mabijkerk mabijkerk marked this pull request as ready for review June 23, 2026 06:46

@aaccensi aaccensi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new query_validation_spec looks good to me!

Regarding the renamed_functions_spec, I don't think we should delete it.
What it does is to check a list of GQL functions that have been renamed at some point in time are not still being used anywhere with the old name. If we ever make more of this name changes (or we know of a name change that has happened and wasn't included in this test) we should add them to the list:

  • INPUT_LINKS → INPUT_EDGES
  • LG → EDGE_GROUP
  • LINK → EDGE
  • LINK_GROUP → EG
  • OUTPUT_LINKS → OUTPUT_EDGES

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants