FAIR cyber risk quantification toolkits, agent-based control simulation (FAIR-CAM), threat event frequency estimator (PyPI), LLM classification validator (PyPI), Monte Carlo risk engine with IRIS benchmarks.

A comprehensive Python framework for evaluating LLM-extracted structured data against ground truth labels. Supports binary classification, scalar values, and list fields with detailed performance metrics, confidence-based evaluation, and statistical uncertainty quantification via non-parametric bootstrap confidence intervals.

pytest for LLM apps - Test for grounding failures, prompt injection, safety violations, and regressions

Deterministic validation firewall that verifies AI-generated proposals against ground-truth state using immutable rules. Zero dependencies. Patent pending.

Evidence-backed validation framework for AI agents, Cursor skills, MCP orchestrations, and agentic repos. Structured rule catalog, verified evidence standard, and certification output. Free, open-source, MIT.

A new package that helps developers integration-test AI and LLM applications by validating structured outputs. It takes a user's test scenario or prompt as input, sends it to an LLM, and uses pattern

Setting benchmarking standards for the LLM security!

Systematic AI evaluation framework that transforms subjective assessment into objective measurement. Reduce research time by 85% while maintaining 95%+ accuracy through multi-LLM validation.

GSMA telecom documentation dataset creation pipeline with hard negative generation for embedding training. Features concurrent LLM validation, semantic similarity ranking, and DVC-based reproducible data processing.

MCP server for heterogeneous AI validation — T1 structured reasoning, T2 cross-model evaluation, and deterministic checksum. Python stdlib-only.

A unified suite of MCP suppressors that prevent hallucinations, enforce grounding, and stabilise agent behaviour.

MCP-native post-generation validation layer for LLM outputs

GenAI Chatbot Testing Framework using Python & Pytest | Validates intent, context, prompt variations, hallucination risks, safety, and response quality scoring

Production-grade AI quality gates for LLM applications. Automating hallucination detection, semantic accuracy scoring, and adversarial safety testing for RAG & Agentic AI systems.

C# SDK for the Guardrails AI API -- LLM validation, hallucination detection, PII protection, and content safety