feat: remove product config & clean up#716
Conversation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces product-config's writer module. Lives in framework::writer and renders hive-site.xml (Hadoop XML) and security.properties (Java properties), matching product-config's output. Also scaffolds controller/build with placeholders for the per-file builders. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… builders Reshape ValidatedCluster (name, image, cluster_config, role_group_configs) and validate via framework::role_utils::with_validated_config. Build hive-site.xml, security.properties and core-site.xml via dedicated builders + the vendored writer. Switch HiveConfigOverrides to v2 KeyValueConfigOverrides (key-by-key Merge). product-config is now only referenced by the crd Configuration impl. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the Configuration impl for MetaStoreConfigFragment and the product-config crate from the workspace. product-config now only appears transitively via stackable-operator. The deploy/config-spec packaging is left in place to mirror trino-operator (it is operator-templating-managed infrastructure). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve the JDBC driver, Kerberos hive-site entries and the core-site decision into ValidatedClusterConfig during validation, so hive_site/core_site builders consume only ValidatedClusterConfig. config_map keeps an explicit `owner: &HiveCluster` solely for the ObjectMeta owner reference. Rename the reconcile binding `validated` -> `validated_cluster`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the vendored java-properties/Hadoop-XML writer (rust/operator-binary/src/framework/writer.rs) with stackable_operator::v2::config_file_writer, which hosts the same code (moved there verbatim via operator-rs #1217 on the smooth-operator branch; hive's copy was byte-identical to hdfs's, the canonical source). Drop the now-unused java-properties and xml dependencies. The framework module now only contains role_utils. The base dependency tag moves from stackable-operator-0.111.0 to 0.111.1, matching the other operators; cargo only substitutes a [patch] whose package version matches, and the smooth-operator branch carries 0.111.1. No behaviour change; rendered output is byte-identical by construction (same code, new home). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…overy_configmap (provided in ValidatedCluster)
|
…and reduce usage of RoleGroupRef.
|
…bject_meta builder
| if let Some(vector_config) = vector_config { | ||
| cm_builder.add_data(VECTOR_CONFIG_FILE, vector_config); | ||
| } |
There was a problem hiding this comment.
The Vector config could be created here like the other configuration files and the vector_config parameter could be removed:
| if let Some(vector_config) = vector_config { | |
| cm_builder.add_data(VECTOR_CONFIG_FILE, vector_config); | |
| } | |
| let role_group_ref = RoleGroupRef { | |
| cluster: ObjectRef::from_obj(cluster), | |
| role: HiveRole::MetaStore.to_string(), | |
| role_group: role_group_name.to_string(), | |
| }; | |
| if let Some(vector_config) = logging::build_vector_config(&role_group_ref, &rg.config.logging) { | |
| cm_builder.add_data(VECTOR_CONFIG_FILE, vector_config); | |
| } |
There was a problem hiding this comment.
Was a bigger change, introduced ValidatedMetastoreConfig 2732b53
| /// A validated, merged Hive metastore role-group config. | ||
| /// | ||
| /// Built by [`validate::validate_cluster`] from the upstream | ||
| /// [`stackable_operator::v2::role_utils::with_validated_config`] result. Holds only the | ||
| /// fields the build steps consume; `cli_overrides` and the product-specific common config | ||
| /// are intentionally not carried (hive does not use them). | ||
| #[derive(Clone, Debug, PartialEq)] | ||
| pub struct HiveRoleGroupConfig { | ||
| pub replicas: u16, | ||
| pub config: MetaStoreConfig, | ||
| pub config_overrides: v1alpha1::HiveConfigOverrides, | ||
| pub env_overrides: stackable_operator::v2::builder::pod::container::EnvVarSet, | ||
| pub pod_overrides: stackable_operator::k8s_openapi::api::core::v1::PodTemplateSpec, | ||
| pub jvm_argument_overrides: | ||
| stackable_operator::v2::jvm_argument_overrides::JvmArgumentOverrides, | ||
| } |
There was a problem hiding this comment.
The cli_overrides are not used, but this is such a minor difference that I would not create a separate structure for this operator.
| /// A validated, merged Hive metastore role-group config. | |
| /// | |
| /// Built by [`validate::validate_cluster`] from the upstream | |
| /// [`stackable_operator::v2::role_utils::with_validated_config`] result. Holds only the | |
| /// fields the build steps consume; `cli_overrides` and the product-specific common config | |
| /// are intentionally not carried (hive does not use them). | |
| #[derive(Clone, Debug, PartialEq)] | |
| pub struct HiveRoleGroupConfig { | |
| pub replicas: u16, | |
| pub config: MetaStoreConfig, | |
| pub config_overrides: v1alpha1::HiveConfigOverrides, | |
| pub env_overrides: stackable_operator::v2::builder::pod::container::EnvVarSet, | |
| pub pod_overrides: stackable_operator::k8s_openapi::api::core::v1::PodTemplateSpec, | |
| pub jvm_argument_overrides: | |
| stackable_operator::v2::jvm_argument_overrides::JvmArgumentOverrides, | |
| } | |
| /// A validated, merged Hive metastore role-group config. | |
| type HiveRoleGroupConfig = stackable_operator::v2::role_utils::RoleGroupConfig< | |
| MetaStoreConfig, | |
| stackable_operator::v2::role_utils::JavaCommonConfig, | |
| v1alpha1::HiveConfigOverrides, | |
| >; |
|
|
||
| impl ValidatedCluster { | ||
| pub fn new( | ||
| name: stackable_operator::v2::types::operator::ClusterName, |
There was a problem hiding this comment.
Is there a reason that qualified paths are used?
There was a problem hiding this comment.
I like to practice my typing :)
| /// The metrics [`Service`] name, `<cluster>-<role>-<rolegroup>-metrics`. | ||
| /// | ||
| /// [`ResourceNames`](stackable_operator::v2::role_group_utils::ResourceNames) has no metrics | ||
| /// service helper, so the `-metrics` suffix is appended to the qualified role-group name (which is | ||
| /// also the StatefulSet name). | ||
| fn metrics_service_name(cluster: &ValidatedCluster, role_group_name: &RoleGroupName) -> String { | ||
| format!( | ||
| "{qualified}-metrics", | ||
| qualified = cluster.resource_names(role_group_name).stateful_set_name() | ||
| ) | ||
| } |
There was a problem hiding this comment.
There is now a function in stackable-operator: stackabletech/operator-rs#1221
| @@ -40,7 +40,7 @@ spec: | |||
| max: "3" | |||
| memory: | |||
| limit: 3Gi | |||
| resources-from-pod-overrides: | |||
| pod-overrides: | |||
There was a problem hiding this comment.
These changes should not be necessary anymore with stackabletech/operator-rs#1222.
Description
Definition of Done Checklist
Author
Reviewer
Acceptance
type/deprecationlabel & add to the deprecation scheduletype/experimentallabel & add to the experimental features tracker