Add Telemetry to Application Name#3683
Conversation
There was a problem hiding this comment.
Pull request overview
Adds anonymous, decodable usage telemetry by embedding a DAB-specific token into database connection strings’ Application Name for SQL Server/Azure SQL, DWSQL, and PostgreSQL, with an offline CLI encoder/decoder and buffered debug logging suitable for hosted/late-config and hot-reload scenarios.
Changes:
- Introduces
ApplicationNameTelemetryencoder/decoder and injects telemetry-bearingApplication Nameinto MSSQL/DWSQL/PostgreSQL connection strings during config load and hosted/late-config initialization. - Makes telemetry embedding pool-safe/idempotent, adds bounded
LogBuffer, and flushes buffered logs on hot reload and hosted initialization paths. - Adds new
dab appnameCLI command (encode/decode) plus extensive unit/E2E/integration test coverage for telemetry shape, injection behavior, multi-DB merge behavior, opt-out, and log buffering.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Service.Tests/UnitTests/RuntimeConfigLoaderJsonDeserializerTests.cs | Updates test to new MSSQL-specific Application Name injection helper. |
| src/Service.Tests/UnitTests/LogBufferTests.cs | Adds unit tests for bounded buffering and flush/drain behavior. |
| src/Service.Tests/UnitTests/ApplicationNameTelemetryTests.cs | Adds unit tests for telemetry encoding/decoding, opt-out, host label prefixing, and multi-DB live-vs-default encoding. |
| src/Service.Tests/Configuration/RuntimeConfigLoaderTests.cs | Adds regression tests for multi-DB telemetry merge correctness, idempotency, and log buffer flushing. |
| src/Service.Tests/Configuration/ConfigurationTests.cs | Updates/extends configuration tests for telemetry-bearing Application Name across MSSQL/Postgres/DWSQL and hosted/late-config scenarios. |
| src/Product/ProductInfo.cs | Introduces a stable dab_oss_ marker constant and uses it to build the DAB user agent string. |
| src/Core/Configurations/RuntimeConfigProvider.cs | Ensures telemetry is embedded and logs flushed during hosted/late-config initialization for all data sources. |
| src/Config/Telemetry/ApplicationNameTelemetry.cs | Implements telemetry token encoding/decoding and environment-driven opt-out/label prefixing. |
| src/Config/RuntimeConfigLoader.cs | Reworks post-parse connection string processing to support multi-DB, idempotent telemetry injection, and override behavior. |
| src/Config/ObjectModel/RuntimeConfig.cs | Ensures child config loads skip Application Name injection so the merged/root config injects once globally. |
| src/Config/LogBuffer.cs | Adds a max buffer size and drop-oldest behavior to prevent unbounded growth. |
| src/Config/FileSystemRuntimeConfigLoader.cs | Flushes buffered logs on hot reload; uses base loader flushing via overridden logger property. |
| src/Config/DeserializationVariableReplacementSettings.cs | Adds SkipApplicationNameInjection flag to coordinate multi-DB child/root injection behavior. |
| src/Cli/Program.cs | Registers new appname command options. |
| src/Cli/Commands/AppNameOptions.cs | Adds dab appname command (encode from config without validation; decode tolerant of truncation; optional file output). |
| src/Cli.Tests/EndToEndTests.cs | Adds E2E coverage for appname encode/decode/output behaviors and updates Application Name assertions for telemetry. |
|
|
||
| ## Summary | ||
|
|
||
| Data API builder (DAB) embeds a compact, anonymous **usage-telemetry token** into the `Application Name` property of the connection strings it uses to reach SQL Server, Azure SQL, Azure SQL Data Warehouse (DWSQL), and PostgreSQL. Because `Application Name` is surfaced on the database side (for example in `sys.dm_exec_sessions.program_name`), this lets the team understand — **in aggregate and without any per-customer identifiers** — which DAB version is running and which features are enabled, using telemetry the database already collects. |
There was a problem hiding this comment.
Why not MySQL? And why PostgreSQL? Is sys.dm_exec_sessions.program_name supported on PostgreSQL
|
|
||
| ### Hosted label (`DAB_APP_NAME_ENV`) | ||
|
|
||
| When `DAB_APP_NAME_ENV` is set (DAB's hosted offering sets it to `dab_hosted`), its value is preserved as a **comma prefix**: `dab_hosted,dab_oss_<version>+...+`. Telemetry is always computed from the product version, so the host label **never suppresses** the token and the `dab_oss_` marker stays intact for decoding. |
There was a problem hiding this comment.
in the hosted case, the appname should be dab_hosted_<version>+...+ It should not include dab_oss_ marker.
|
|
||
| // Idempotency guard: if DAB telemetry was already embedded into the Application Name (e.g. by | ||
| // the loader's post-processing), do not append it again — that would duplicate the payload. | ||
| if (connectionStringBuilder.ApplicationName?.Contains(ProductInfo.DAB_USER_AGENT_MARKER, StringComparison.Ordinal) == true) |
There was a problem hiding this comment.
For the hosted scenario, the embedded application name should be dab_hosted_. dab_oss_ should not be present in the embedded application name for that case. The contains check here should verify if is present, to not append again. This is because is the common marker between oss and hosted scenarios.
|
|
||
| // Idempotency guard: if DAB telemetry was already embedded into the Application Name (e.g. by | ||
| // the loader's post-processing), do not append it again — that would duplicate the payload. | ||
| if (connectionStringBuilder.ApplicationName?.Contains(ProductInfo.DAB_USER_AGENT_MARKER, StringComparison.Ordinal) == true) |
There was a problem hiding this comment.
same as above: check should verify
| /// Options for the <c>appname</c> command, which encodes the DAB telemetry Application Name | ||
| /// from a config file, or decodes a telemetry Application Name into a human-readable description. | ||
| /// </summary> | ||
| [Verb("appname", isDefault: false, HelpText = "Show or decode the DAB telemetry 'Application Name' embedded in SQL connections.", Hidden = false)] |
There was a problem hiding this comment.
| [Verb("appname", isDefault: false, HelpText = "Show or decode the DAB telemetry 'Application Name' embedded in SQL connections.", Hidden = false)] | |
| [Verb("appname", isDefault: false, HelpText = "Show or decode the DAB telemetry 'Application Name' embedded in datasource connections.", Hidden = false)] |
| string entity = EncodeSection(_entitySettings, inputs); | ||
|
|
||
| return new StringBuilder() | ||
| .Append(ProductInfo.DAB_USER_AGENT) |
There was a problem hiding this comment.
should be dab_hosted for hosted scenario.
| public static string BuildApplicationNameSegment(RuntimeConfig config, DataSource? liveDataSource) | ||
| { | ||
| string telemetry = IsOptedOut() | ||
| ? ProductInfo.DAB_USER_AGENT |
There was a problem hiding this comment.
if hosted, this should be dab_hosted, not dab_oss
Why make this change?
Closes #3216
DAB ships as an open-source container customers run anywhere, so we have little visibility into how it is configured or which features are used. This change encodes an anonymous usage fingerprint into the connection's
Application Name, which the database side already records (e.g.sys.dm_exec_sessions.program_name), giving aggregate insight with no new endpoints, no network calls, and no per-customer data.What is this change?
Embeds an anonymous, decodable usage-telemetry token into the
Application Nameof the connection strings DAB uses for SQL Server / Azure SQL, DWSQL, and PostgreSQL.dab_oss_<version>+<context>|<runtime>|<entity>+a version plus three flag sections (Context = 4, Runtime = 20, Entity = 14). Values use0/1(off/on),M(missing section),X(not applicable at pool open),?(not yet modeled).Source); per-request facets (protocol/object/role) areXso the connection-pool key, and therefore pool count, is unchanged.RuntimeConfigLoader) and on the hosted/late-configPOST /configurationpaths (RuntimeConfigProvider.Initialize). Embedding is idempotent (skips if thedab_oss_marker is already present).Sourcestill varies per pool.DAB_TELEMETRY_APPNAME_OPT_OUT=1reduces the value to version-only.DAB_APP_NAME_ENVvalue (e.g.dab_hosted) is preserved as a comma prefix and never suppresses the token.dab appnamecommand to encode (--config) or decode (--decode) a token;-o/--outputwrites to a file.LogBufferflushed once the logger is available.docs/design/application-name-telemetry.md.How was this tested?
Added encoder/decoder unit tests (token shape, every flag/section, Source + auth-provider maps, opt-out, host-label prefix, truncation-tolerant decode), connection-string injection tests for MSSQL/DWSQL/PostgreSQL (incl. user-supplied
Application Nameprefix) and no-op cases for MySQL/Cosmos, multi-database and hosted/late-config (/configuration+/configuration/v2) tests, an idempotency test,LogBufferbound/flush tests, and CLI encode/decode/error/opt-out tests.Sample Request(s)
Encode the token for a config (offline, no DB connection):
dab appname --config dab-config.json # dab_oss_2.0.0+XXXX|110000M1M000MMMMMWMM|100?111001110?+Decode a token back to a human-readable legend:
dab appname --decode "dab_oss_2.0.0+XXSX|110000M1M000MMMMMWMM|100?111001110?+"Observe it on the SQL Server side: