Skip to content

Develop#126

Open
RahulRengeshOfficial wants to merge 273 commits into
mainfrom
develop
Open

Develop#126
RahulRengeshOfficial wants to merge 273 commits into
mainfrom
develop

Conversation

@RahulRengeshOfficial
Copy link
Copy Markdown
Contributor

No description provided.

coolwater and others added 30 commits September 5, 2025 11:06
xconf logging: add newFWOffer flag to EstbFirmwareService XCONF_LOG
Add rfc_rules and rfc_features in penetrationMetrics table and logs
Updated README and sample configuration with detailed sections and comments.
Added flag for TLS certificate validation.
Doc(swagger): Add XConf DataService device-facing APIs documentation
Implemented support for a distributed lock leveraging Cassandra LWT
Use cassandra for security token instead of group service
Revert "Added flag for TLS certificate validation."
Add configuration option to enable/disable TLS certificate validation.
Added retry logic for DistributedLock
modify logging - xds to xds.token
gravi21 and others added 5 commits April 17, 2026 12:25
- Move GroupServiceConnector test double to dedicated helper file
  (dataapi/group_service_connector_mock_test.go) with Comcast copyright
  header and compile-time interface assertion
- Add TestGetAccountInfoFromGrpService_AccountTypePrecedence: verifies
  non-empty AccountProducts.AccountType overwrites the Group Service value,
  and an empty AccountProducts.AccountType does not discard a valid one
- Extend TestGetFirstElementsInContextMap: include ACCOUNT_TYPE
  comma-separated normalisation case
update go.mod and go.sum to use Go 1.25.0 and upgrade dependencies
Copilot AI review requested due to automatic review settings April 20, 2026 12:46
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 95 out of 206 changed files in this pull request and generated 15 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread http/metrics.go Outdated
Name: "AccountService_empty_response_count",
Help: "A counter for empty 200 responses from AccountService",
},
[]string{"app", "model"},
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The accountServiceEmptyResponseCounter is declared with label names {"app","model"}, but IncreaseAccountServiceEmptyResponseCounter supplies an additional partner label. Prometheus will panic at runtime due to inconsistent label cardinality. Update the counter vec to include "partner" in its label list or remove the "partner" label from the increment call (mandatory).

Suggested change
[]string{"app", "model"},
[]string{"app", "model", "partner"},

Copilot uses AI. Check for mistakes.
Comment thread http/metrics.go
Comment on lines +500 to 503
func IncreaseAccountServiceEmptyResponseCounter(model, partner string) {
if metrics == nil {
return
}
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The accountServiceEmptyResponseCounter is declared with label names {"app","model"}, but IncreaseAccountServiceEmptyResponseCounter supplies an additional partner label. Prometheus will panic at runtime due to inconsistent label cardinality. Update the counter vec to include "partner" in its label list or remove the "partner" label from the increment call (mandatory).

Copilot uses AI. Check for mistakes.
Comment thread http/metrics.go
Comment on lines +508 to +515
if len(partner) == 0 {
partner = "null"
}

labels := prometheus.Labels{
"app": AppName(),
"model": model,
"app": AppName(),
"partner": partner,
"model": model,
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The accountServiceEmptyResponseCounter is declared with label names {"app","model"}, but IncreaseAccountServiceEmptyResponseCounter supplies an additional partner label. Prometheus will panic at runtime due to inconsistent label cardinality. Update the counter vec to include "partner" in its label list or remove the "partner" label from the increment call (mandatory).

Suggested change
if len(partner) == 0 {
partner = "null"
}
labels := prometheus.Labels{
"app": AppName(),
"model": model,
"app": AppName(),
"partner": partner,
"model": model,
labels := prometheus.Labels{
"app": AppName(),
"model": model,

Copilot uses AI. Check for mistakes.
Comment thread http/mock.go
Comment on lines +52 to +54
// Set templates for mock - if config has empty templates, mock provides working ones for test
server.SatServiceConnector.SetTokenUrlTemplate("")
server.SatServiceConnector.SetTokenPartnerUrlTemplate("")
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These setters configure empty format strings for SAT token URLs. With the updated sat_connector.go code, fmt.Sprintf(c.tokenUrlTemplate, c.SatServiceHost()) will produce an empty URL, causing requests to fail. Set these to valid default templates (e.g., the previous hardcoded routes) or avoid overriding templates when the intention is to keep defaults (mandatory).

Suggested change
// Set templates for mock - if config has empty templates, mock provides working ones for test
server.SatServiceConnector.SetTokenUrlTemplate("")
server.SatServiceConnector.SetTokenPartnerUrlTemplate("")
// Use non-empty format strings so SAT connector URL formatting resolves to the mock host.
server.SatServiceConnector.SetTokenUrlTemplate("%s")
server.SatServiceConnector.SetTokenPartnerUrlTemplate("%s")

Copilot uses AI. Check for mistakes.
Comment thread http/sat_connector.go
Comment on lines +105 to +108
tokenUrl := conf.GetString(tokenUrlKey)

tokenPartnerUrlKey := fmt.Sprintf("xconfwebconfig.%v.token_partner_url_template", satServiceName)
tokenPartnerUrl := conf.GetString(tokenPartnerUrlKey)
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

URL templates loaded from config are used directly without validation or defaults. If the config keys are missing/blank, fmt.Sprintf("", ...) returns "", producing an invalid URL and failing requests. Add a blank-check and either (1) fall back to safe defaults (the previous constant templates) or (2) fail fast with a clear configuration error (mandatory).

Suggested change
tokenUrl := conf.GetString(tokenUrlKey)
tokenPartnerUrlKey := fmt.Sprintf("xconfwebconfig.%v.token_partner_url_template", satServiceName)
tokenPartnerUrl := conf.GetString(tokenPartnerUrlKey)
tokenUrl := conf.GetString(tokenUrlKey)
if util.IsBlank(tokenUrl) {
panic(fmt.Errorf("%s is required", tokenUrlKey))
}
tokenPartnerUrlKey := fmt.Sprintf("xconfwebconfig.%v.token_partner_url_template", satServiceName)
tokenPartnerUrl := conf.GetString(tokenPartnerUrlKey)
if util.IsBlank(tokenPartnerUrl) {
panic(fmt.Errorf("%s is required", tokenPartnerUrlKey))
}

Copilot uses AI. Check for mistakes.
Comment on lines +287 to +296
tagging_service {
host = "%s"
tags_mac_address_template = ""
tags_partner_template = ""
tags_partner_and_mac_address_template = ""
tags_partner_and_mac_address_template = ""
tags_account_template = ""
tags_partner_and_mac_address_and_account_template = ""
tags_partner_and_account_template = ""
}
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The config snippet repeats tags_partner_and_mac_address_template twice and appears to omit tags_mac_address_and_account_template (which the connector now reads). This makes the test configuration misleading and may hide missing-template errors. Replace the duplicate key with the correct template key to match NewTaggingConnector’s config reads (mandatory).

Copilot uses AI. Check for mistakes.
Comment thread db/schema.go
Comment on lines 69 to 77
TABLE_LOGS = "Logs2"
TABLE_XCONF_CHANGED_KEYS = "XconfChangedKeys4"
TABLE_APP_SETTINGS = "AppSettings"
TABLE_APPLICATION_TYPES = "ApplicationTypes"
TABLE_TAG = "Tag"
TABLE_LOCKS = "Locks"
)

var AllTables = []string{
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TABLE_LOCKS is added as a constant, but it is not included in AllTables. If AllTables is used for initialization/bootstrapping/validation, the new Locks table may be skipped. Add TABLE_LOCKS to AllTables (mandatory if AllTables is intended to be exhaustive).

Copilot uses AI. Check for mistakes.
Comment thread db/schema.go
Comment on lines 109 to 113
TABLE_LOGS,
TABLE_XCONF_CHANGED_KEYS,
TABLE_TAG,
TABLE_APPLICATION_TYPES,
}
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TABLE_LOCKS is added as a constant, but it is not included in AllTables. If AllTables is used for initialization/bootstrapping/validation, the new Locks table may be skipped. Add TABLE_LOCKS to AllTables (mandatory if AllTables is intended to be exhaustive).

Copilot uses AI. Check for mistakes.
Comment thread http/security_manager.go
Comment on lines 91 to +101
func (s *SecurityTokenPathConfig) getSecurityToken(deviceInfo map[string]string, fields log.Fields) string {
if CanSkipSecurityTokenForClientProtocol(deviceInfo) {
log.WithFields(fields).Debugf("Client protocol type is in token generation skip list, no security token needed")
return ""
}
if len(Ws.SecurityTokenConfig.SecurityTokenModelSet) > 0 && !Ws.SecurityTokenConfig.SecurityTokenModelSet.Contains(strings.ToLower(deviceInfo[SECURITY_TOKEN_MODEL])) {
log.WithFields(fields).Debugf("Model type is not in model list, so no security token needed, model=%s", deviceInfo[SECURITY_TOKEN_MODEL])
return ""
}
if Ws.SecurityTokenConfig.SecurityTokenDevicePercentEnabled && !rulesengine.FitsPercent(deviceInfo[SECURITY_TOKEN_ESTB_MAC], Ws.SecurityTokenConfig.SecurityTokenDevicePercentValue) {
log.WithFields(fields).Debugf("Device mac hash does not fall within security token percent, so no security token needed")
return ""
}
token := ""
if Ws.SecurityTokenConfig.SecurityTokenGroupServiceEnabled {
// token will be set to the mac address without colons if using GroupService to hold other device details
token = util.AlphaNumericMacAddress(deviceInfo[SECURITY_TOKEN_ESTB_MAC])
if util.IsBlank(token) {
log.WithFields(fields).Errorf("Mac address is missing, not generating security token")
return ""
}
deviceInfo[s.TimestampKey] = fmt.Sprintf("%d", time.Now().Unix())
log.WithFields(fields).Debugf("Adding security token to group service")
// removing macAddress from deviceInfo since it's already present as the key
delete(deviceInfo, SECURITY_TOKEN_ESTB_MAC)
err := Ws.GroupServiceSyncConnector.AddSecurityTokenInfo(token, deviceInfo, fields)
if err != nil {
log.WithFields(fields).Errorf("Error adding security token to group service, err=%+v", err)
}
} else {
// if not using GroupService, need to hash the estbIP (and optionally the filename) to create the token
estbIp := deviceInfo[SECURITY_TOKEN_ESTB_IP]
if util.IsBlank(estbIp) {
log.WithFields(fields).Errorf("EstbIP is missing, not generating security token")
return ""
}
input := estbIp
if s.FilenameInTokenEnabled {
input = fmt.Sprintf("%s_%s", input, deviceInfo[SECURITY_TOKEN_FW_LIST])
}
token = generateSecurityToken(input, fields)
// we will use mac address without colons as the token
// 1. if group service is enabled, we will store token info in group service to look up later
// 2. if group service is not enabled, we will get the token info from Cassandra (all fields needed are already in penetration table, so just return token)
// token will be set to the mac address without colons
token = util.AlphaNumericMacAddress(deviceInfo[SECURITY_TOKEN_ESTB_MAC])
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getSecurityToken now returns a deterministic token derived from the MAC address, and the token value is added into log fields. If this “token” is used as a security mechanism (rather than an identifier), this change makes it predictable and easier to replay/guess, and it increases the risk of token leakage via logs. If the intended design is “identifier only,” consider renaming/logging appropriately; otherwise, restore a cryptographic token generation strategy and/or avoid logging the raw token (mandatory to re-evaluate, given the security implications).

Copilot uses AI. Check for mistakes.
Comment thread http/security_manager.go
Comment on lines 110 to +115
func (s *SecurityTokenPathConfig) addTokenToUrl(deviceInfo map[string]string, urlString string, isFqdn bool, fields log.Fields) string {
securityToken := s.getSecurityToken(deviceInfo, fields)
if util.IsBlank(securityToken) {
return urlString
}
fields[fmt.Sprintf("%s.token", SECURITY_TOKEN_KEY)] = securityToken
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getSecurityToken now returns a deterministic token derived from the MAC address, and the token value is added into log fields. If this “token” is used as a security mechanism (rather than an identifier), this change makes it predictable and easier to replay/guess, and it increases the risk of token leakage via logs. If the intended design is “identifier only,” consider renaming/logging appropriately; otherwise, restore a cryptographic token generation strategy and/or avoid logging the raw token (mandatory to re-evaluate, given the security implications).

Suggested change
func (s *SecurityTokenPathConfig) addTokenToUrl(deviceInfo map[string]string, urlString string, isFqdn bool, fields log.Fields) string {
securityToken := s.getSecurityToken(deviceInfo, fields)
if util.IsBlank(securityToken) {
return urlString
}
fields[fmt.Sprintf("%s.token", SECURITY_TOKEN_KEY)] = securityToken
func redactSecurityTokenForLog(token string) string {
if util.IsBlank(token) {
return ""
}
if len(token) <= 4 {
return "****"
}
return fmt.Sprintf("****%s", token[len(token)-4:])
}
func (s *SecurityTokenPathConfig) addTokenToUrl(deviceInfo map[string]string, urlString string, isFqdn bool, fields log.Fields) string {
securityToken := s.getSecurityToken(deviceInfo, fields)
if util.IsBlank(securityToken) {
return urlString
}
fields[fmt.Sprintf("%s.token", SECURITY_TOKEN_KEY)] = redactSecurityTokenForLog(securityToken)

Copilot uses AI. Check for mistakes.
gravi21 and others added 3 commits April 21, 2026 07:31
…143)" (#148)

This reverts commit 9081e96.

Co-authored-by: sding921 <22039486+sding921@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 21, 2026 20:18
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

accountType = xAccountId.GetAccountType()
contextMap[common.ACCOUNT_ID] = accountId
contextMap[common.ACCOUNT_TYPE] = accountType
log.WithFields(fields).Debugf("AddEstbFirmwareContext Successfully fetched AcntId='%s' and AcntType='%s' from Grp Svc", accountId, accountType)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

Sensitive data returned by HTTP request headers
flows to a logging call.

Copilot Autofix

AI 16 days ago

General fix: do not place raw request-header-derived values into logging fields. If needed for diagnostics, log a sanitized/obfuscated value, or omit it entirely.

Best minimal fix without changing behavior: keep using the resolved IP in contextMap, but stop adding it to fields. This preserves business logic (context enrichment and downstream use) while preventing propagation to structured logs.

Change needed:

  • File: dataapi/estb_firmware_context.go
  • Region: NormalizeEstbFirmwareContext(...), inside if shouldAddIp { ... }
  • Replace assignment to fields[common.IP_ADDRESS] with a deletion of any existing IP key (defensive cleanup), so no sensitive IP is logged through WithFields(fields).

No new imports, methods, or dependencies are required.

Suggested changeset 1
dataapi/estb_firmware_context.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/dataapi/estb_firmware_context.go b/dataapi/estb_firmware_context.go
--- a/dataapi/estb_firmware_context.go
+++ b/dataapi/estb_firmware_context.go
@@ -127,7 +127,7 @@
 	if shouldAddIp {
 		ipAddress := util.GetIpAddress(r, contextMap[common.IP_ADDRESS], fields)
 		contextMap[common.IP_ADDRESS] = ipAddress
-		fields[common.IP_ADDRESS] = ipAddress
+		delete(fields, common.IP_ADDRESS)
 	}
 	// check if request is for partner
 	if usePartnerAppType && contextMap[common.APPLICATION_TYPE] == shared.STB {
EOF
@@ -127,7 +127,7 @@
if shouldAddIp {
ipAddress := util.GetIpAddress(r, contextMap[common.IP_ADDRESS], fields)
contextMap[common.IP_ADDRESS] = ipAddress
fields[common.IP_ADDRESS] = ipAddress
delete(fields, common.IP_ADDRESS)
}
// check if request is for partner
if usePartnerAppType && contextMap[common.APPLICATION_TYPE] == shared.STB {
Copilot is powered by AI and may make mistakes. Always verify output.
}
}
xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

Sensitive data returned by HTTP request headers
flows to a logging call.

Copilot Autofix

AI 16 days ago

Best fix: do not log request-derived fields on this debug statement, and avoid dumping full contextMap. Keep behavior intact while preventing sensitive/tainted data from being emitted.

In dataapi/estb_firmware_context.go (around line 311), replace:

  • log.WithFields(fields).Debugf(...)
    with
  • log.Debugf(...) using only explicit non-sensitive values already present in the format args.

This removes the tainted fields from the sink while preserving the debug message and functional behavior. No new imports, methods, or dependencies are required.

Suggested changeset 1
dataapi/estb_firmware_context.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/dataapi/estb_firmware_context.go b/dataapi/estb_firmware_context.go
--- a/dataapi/estb_firmware_context.go
+++ b/dataapi/estb_firmware_context.go
@@ -308,7 +308,7 @@
 					}
 				}
 				xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
-				log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)
+				log.Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)
 			}
 		} else {
 			log.WithFields(log.Fields{"error": err}).Errorf("Error getting accountId information from Grp Service for Mac=%s", macAddress)
EOF
@@ -308,7 +308,7 @@
}
}
xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)
log.Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)
}
} else {
log.WithFields(log.Fields{"error": err}).Errorf("Error getting accountId information from Grp Service for Mac=%s", macAddress)
Copilot is powered by AI and may make mistakes. Always verify output.
}
}
xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

Sensitive data returned by HTTP request headers
flows to a logging call.

Copilot Autofix

AI 16 days ago

General fix: do not log raw sensitive identifiers or entire context maps. Replace with redacted/boolean/summary logging that preserves operational observability.

Best fix here (without changing functionality): in dataapi/estb_firmware_context.go, update the debug/error log lines in AddEstbFirmwareContext that currently print ACCOUNT_ID, ACCOUNT_TYPE, and %v of contextMap. Keep the same control flow and behavior, but emit sanitized metadata (e.g., whether account id exists, account type presence, count of context keys). No new dependencies are needed.

Targeted lines to change:

  • Debug logs at current lines ~271, ~275, ~311.
  • Error log at current line ~278 (remove account id value from message).
Suggested changeset 1
dataapi/estb_firmware_context.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/dataapi/estb_firmware_context.go b/dataapi/estb_firmware_context.go
--- a/dataapi/estb_firmware_context.go
+++ b/dataapi/estb_firmware_context.go
@@ -268,14 +268,14 @@
 			accountType = xAccountId.GetAccountType()
 			contextMap[common.ACCOUNT_ID] = accountId
 			contextMap[common.ACCOUNT_TYPE] = accountType
-			log.WithFields(fields).Debugf("AddEstbFirmwareContext Successfully fetched AcntId='%s' and AcntType='%s' from Grp Svc", accountId, accountType)
+			log.WithFields(fields).Debugf("AddEstbFirmwareContext Successfully fetched account information from Grp Svc (accountIdPresent=%t, accountTypePresent=%t)", accountId != "", accountType != "")
 		}
 
 		if contextMap[common.ACCOUNT_ID] != "" && !util.IsUnknownValue(contextMap[common.ACCOUNT_ID]) {
-			log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' already present,fetching AccntPrds directly from ada", contextMap[common.ACCOUNT_ID])
+			log.WithFields(fields).Debug("AddEstbFirmwareContext accountId already present, fetching account products directly from ada")
 			accountProducts, err = ws.GroupServiceConnector.GetAccountProducts(accountId, fields)
 			if err != nil {
-				log.WithFields(log.Fields{"error": err}).Errorf("Error getting accountProducts information from Grp Service for AccountId=%s", contextMap[common.ACCOUNT_ID])
+				log.WithFields(log.Fields{"error": err}).Error("Error getting accountProducts information from Grp Service")
 			} else {
 				if partner, ok := accountProducts["Partner"]; ok {
 					contextMap[common.PARTNER_ID] = partner
@@ -308,7 +303,7 @@
 					}
 				}
 				xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
-				log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)
+				log.WithFields(fields).Debugf("AddEstbFirmwareContext account products successfully retrieved from xac/ada (contextKeys=%d, accountIdPresent=%t, accountTypePresent=%t)", len(contextMap), contextMap[common.ACCOUNT_ID] != "", contextMap[common.ACCOUNT_TYPE] != "")
 			}
 		} else {
 			log.WithFields(log.Fields{"error": err}).Errorf("Error getting accountId information from Grp Service for Mac=%s", macAddress)
EOF
@@ -268,14 +268,14 @@
accountType = xAccountId.GetAccountType()
contextMap[common.ACCOUNT_ID] = accountId
contextMap[common.ACCOUNT_TYPE] = accountType
log.WithFields(fields).Debugf("AddEstbFirmwareContext Successfully fetched AcntId='%s' and AcntType='%s' from Grp Svc", accountId, accountType)
log.WithFields(fields).Debugf("AddEstbFirmwareContext Successfully fetched account information from Grp Svc (accountIdPresent=%t, accountTypePresent=%t)", accountId != "", accountType != "")
}

if contextMap[common.ACCOUNT_ID] != "" && !util.IsUnknownValue(contextMap[common.ACCOUNT_ID]) {
log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' already present,fetching AccntPrds directly from ada", contextMap[common.ACCOUNT_ID])
log.WithFields(fields).Debug("AddEstbFirmwareContext accountId already present, fetching account products directly from ada")
accountProducts, err = ws.GroupServiceConnector.GetAccountProducts(accountId, fields)
if err != nil {
log.WithFields(log.Fields{"error": err}).Errorf("Error getting accountProducts information from Grp Service for AccountId=%s", contextMap[common.ACCOUNT_ID])
log.WithFields(log.Fields{"error": err}).Error("Error getting accountProducts information from Grp Service")
} else {
if partner, ok := accountProducts["Partner"]; ok {
contextMap[common.PARTNER_ID] = partner
@@ -308,7 +303,7 @@
}
}
xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)
log.WithFields(fields).Debugf("AddEstbFirmwareContext account products successfully retrieved from xac/ada (contextKeys=%d, accountIdPresent=%t, accountTypePresent=%t)", len(contextMap), contextMap[common.ACCOUNT_ID] != "", contextMap[common.ACCOUNT_TYPE] != "")
}
} else {
log.WithFields(log.Fields{"error": err}).Errorf("Error getting accountId information from Grp Service for Mac=%s", macAddress)
Copilot is powered by AI and may make mistakes. Always verify output.
}
}
xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

Sensitive data returned by HTTP request headers
flows to a logging call.

Copilot Autofix

AI 16 days ago

To fix this safely without changing behavior, replace debug/error log statements that print sensitive identifiers or full contextMap with sanitized summaries (presence/boolean/constant text), and avoid printing MAC/account IDs directly.

Best change in dataapi/estb_firmware_context.go:

  • Keep operational logging, but remove raw values from:
    • line with contextMap %v
    • account fetch success logs containing AcntId, AcntType
    • account-products log containing %v of contextMap
    • error log containing Mac=%s
  • Replace with non-sensitive status logs (e.g., “account data retrieved”, “account id present”), optionally include non-sensitive counters/flags only.

No functional logic/path changes are needed; only log message content changes.

Suggested changeset 1
dataapi/estb_firmware_context.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/dataapi/estb_firmware_context.go b/dataapi/estb_firmware_context.go
--- a/dataapi/estb_firmware_context.go
+++ b/dataapi/estb_firmware_context.go
@@ -20,7 +20,6 @@
 
 import (
 	"encoding/json"
-	"fmt"
 	"net/http"
 	"net/url"
 	"regexp"
@@ -224,7 +223,7 @@
 	} else {
 		fields = log.Fields{}
 	}
-	log.Debug(fmt.Sprintf("AddEstbFirmwareContext start ... contextMap %v", contextMap))
+	log.Debug("AddEstbFirmwareContext start")
 	NormalizeEstbFirmwareContext(ws, r, contextMap, usePartnerAppType, shouldAddIp, fields)
 	AddGroupServiceContext(ws, contextMap, common.ESTB_MAC, fields)
 	// getting local sat token
@@ -268,11 +267,11 @@
 			accountType = xAccountId.GetAccountType()
 			contextMap[common.ACCOUNT_ID] = accountId
 			contextMap[common.ACCOUNT_TYPE] = accountType
-			log.WithFields(fields).Debugf("AddEstbFirmwareContext Successfully fetched AcntId='%s' and AcntType='%s' from Grp Svc", accountId, accountType)
+			log.WithFields(fields).Debug("AddEstbFirmwareContext successfully fetched account data from group service")
 		}
 
 		if contextMap[common.ACCOUNT_ID] != "" && !util.IsUnknownValue(contextMap[common.ACCOUNT_ID]) {
-			log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' already present,fetching AccntPrds directly from ada", contextMap[common.ACCOUNT_ID])
+			log.WithFields(fields).Debug("AddEstbFirmwareContext account id present, fetching account products directly from ada")
 			accountProducts, err = ws.GroupServiceConnector.GetAccountProducts(accountId, fields)
 			if err != nil {
 				log.WithFields(log.Fields{"error": err}).Errorf("Error getting accountProducts information from Grp Service for AccountId=%s", contextMap[common.ACCOUNT_ID])
@@ -308,10 +304,10 @@
 					}
 				}
 				xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
-				log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)
+				log.WithFields(fields).Debug("AddEstbFirmwareContext account products successfully retrieved from xac/ada")
 			}
 		} else {
-			log.WithFields(log.Fields{"error": err}).Errorf("Error getting accountId information from Grp Service for Mac=%s", macAddress)
+			log.WithFields(log.Fields{"error": err}).Error("Error getting accountId information from Grp Service")
 			xhttp.IncreaseGrpServiceNotFoundResponseCounter(contextMap[common.MODEL])
 		}
 	}
EOF
@@ -20,7 +20,6 @@

import (
"encoding/json"
"fmt"
"net/http"
"net/url"
"regexp"
@@ -224,7 +223,7 @@
} else {
fields = log.Fields{}
}
log.Debug(fmt.Sprintf("AddEstbFirmwareContext start ... contextMap %v", contextMap))
log.Debug("AddEstbFirmwareContext start")
NormalizeEstbFirmwareContext(ws, r, contextMap, usePartnerAppType, shouldAddIp, fields)
AddGroupServiceContext(ws, contextMap, common.ESTB_MAC, fields)
// getting local sat token
@@ -268,11 +267,11 @@
accountType = xAccountId.GetAccountType()
contextMap[common.ACCOUNT_ID] = accountId
contextMap[common.ACCOUNT_TYPE] = accountType
log.WithFields(fields).Debugf("AddEstbFirmwareContext Successfully fetched AcntId='%s' and AcntType='%s' from Grp Svc", accountId, accountType)
log.WithFields(fields).Debug("AddEstbFirmwareContext successfully fetched account data from group service")
}

if contextMap[common.ACCOUNT_ID] != "" && !util.IsUnknownValue(contextMap[common.ACCOUNT_ID]) {
log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' already present,fetching AccntPrds directly from ada", contextMap[common.ACCOUNT_ID])
log.WithFields(fields).Debug("AddEstbFirmwareContext account id present, fetching account products directly from ada")
accountProducts, err = ws.GroupServiceConnector.GetAccountProducts(accountId, fields)
if err != nil {
log.WithFields(log.Fields{"error": err}).Errorf("Error getting accountProducts information from Grp Service for AccountId=%s", contextMap[common.ACCOUNT_ID])
@@ -308,10 +304,10 @@
}
}
xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)
log.WithFields(fields).Debug("AddEstbFirmwareContext account products successfully retrieved from xac/ada")
}
} else {
log.WithFields(log.Fields{"error": err}).Errorf("Error getting accountId information from Grp Service for Mac=%s", macAddress)
log.WithFields(log.Fields{"error": err}).Error("Error getting accountId information from Grp Service")
xhttp.IncreaseGrpServiceNotFoundResponseCounter(contextMap[common.MODEL])
}
}
Copilot is powered by AI and may make mistakes. Always verify output.
}
}
xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

Sensitive data returned by HTTP request headers
flows to a logging call.

Copilot Autofix

AI 16 days ago

Best fix: stop logging the full contextMap and log only a minimal allowlisted subset already being logged individually (e.g., account ID/type), avoiding request/header-derived bulk context output.

In dataapi/estb_firmware_context.go, replace the debug line at the success path (around line 311) so it no longer includes contextMap (AccntPrd='%v'). Keep functional behavior unchanged (no logic/state changes), only reduce logged payload. No new imports, methods, or dependencies are needed.

Suggested changeset 1
dataapi/estb_firmware_context.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/dataapi/estb_firmware_context.go b/dataapi/estb_firmware_context.go
--- a/dataapi/estb_firmware_context.go
+++ b/dataapi/estb_firmware_context.go
@@ -308,7 +308,7 @@
 					}
 				}
 				xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
-				log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)
+				log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE])
 			}
 		} else {
 			log.WithFields(log.Fields{"error": err}).Errorf("Error getting accountId information from Grp Service for Mac=%s", macAddress)
EOF
@@ -308,7 +308,7 @@
}
}
xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' ,AccntPrd='%v' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE], contextMap)
log.WithFields(fields).Debugf("AddEstbFirmwareContext AcntId='%s' AcntType='%s' successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap[common.ACCOUNT_TYPE])
}
} else {
log.WithFields(log.Fields{"error": err}).Errorf("Error getting accountId information from Grp Service for Mac=%s", macAddress)
Copilot is powered by AI and may make mistakes. Always verify output.
contextMap[common.ACCOUNT_ID] = accountId
contextMap[common.ACCOUNT_TYPE] = accountType
contextMap[common.ACCOUNT_HASH] = util.CalculateHash(accountId)
log.WithFields(fields).Debugf("AddContextForPods Successfully fetched AcntId='%s' and AcntType='%s' from Grp Svc", accountId, accountType)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

Sensitive data returned by HTTP request headers
flows to a logging call.

Copilot Autofix

AI about 1 month ago

To fix this without changing behavior, keep business logic intact and only change logging so it never writes unsanitized fields.

Best approach in this snippet:

  • In dataapi/feature_control_context.go, inside getAccountInfoFromGrpService, create a sanitized copy of fields using existing helper common.FilterLogFields(fields) and use that sanitized map for all WithFields(...) calls in the function.
  • This preserves observability while preventing sensitive/header-derived values from being logged in clear text.
  • No new dependencies are needed; required helper already exists and is used elsewhere in the file.

Concretely:

  • Add safeFields := common.FilterLogFields(fields) near the top of getAccountInfoFromGrpService.
  • Replace:
    • log.WithFields(fields).Debugf(...) (line ~144)
    • log.WithFields(fields).Error(...) (line ~181)
    • log.WithFields(fields).Debugf(...) (line ~187)
      with log.WithFields(safeFields)....
Suggested changeset 1
dataapi/feature_control_context.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/dataapi/feature_control_context.go b/dataapi/feature_control_context.go
--- a/dataapi/feature_control_context.go
+++ b/dataapi/feature_control_context.go
@@ -123,6 +123,7 @@
 func getAccountInfoFromGrpService(ws *xhttp.XconfServer, contextMap map[string]string, fields log.Fields) (*PodData, *AccountServiceData) {
 	var podData *PodData
 	var td *AccountServiceData
+	safeFields := common.FilterLogFields(fields)
 
 	var xAccountId *conversion.XBOAccount
 	var err error
@@ -141,7 +142,7 @@
 		contextMap[common.ACCOUNT_ID] = accountId
 		contextMap[common.ACCOUNT_TYPE] = accountType
 		contextMap[common.ACCOUNT_HASH] = util.CalculateHash(accountId)
-		log.WithFields(fields).Debugf("AddContextForPods Successfully fetched AcntId='%s' and AcntType='%s' from Grp Svc", accountId, accountType)
+		log.WithFields(safeFields).Debugf("AddContextForPods Successfully fetched AcntId='%s' and AcntType='%s' from Grp Svc", accountId, accountType)
 
 		accountProducts, err := ws.GroupServiceConnector.GetAccountProducts(accountId, fields)
 		if err != nil {
@@ -178,13 +179,13 @@
 					contextMap[common.ACCOUNT_STATE] = accountState
 				}
 			} else {
-				log.WithFields(fields).Error("Failed to unmarshall AccountProducts")
+				log.WithFields(safeFields).Error("Failed to unmarshall AccountProducts")
 			}
 
 		}
 
 		xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
-		log.WithFields(fields).Debugf("AddContextForPods AcntId='%s' ,AccntPrd='%v' Successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap)
+		log.WithFields(safeFields).Debugf("AddContextForPods AcntId='%s' ,AccntPrd='%v' Successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap)
 		// Create PodData and AccountServiceData with retrieved information
 		podData = &PodData{
 			AccountId: contextMap[common.ACCOUNT_ID],
EOF
@@ -123,6 +123,7 @@
func getAccountInfoFromGrpService(ws *xhttp.XconfServer, contextMap map[string]string, fields log.Fields) (*PodData, *AccountServiceData) {
var podData *PodData
var td *AccountServiceData
safeFields := common.FilterLogFields(fields)

var xAccountId *conversion.XBOAccount
var err error
@@ -141,7 +142,7 @@
contextMap[common.ACCOUNT_ID] = accountId
contextMap[common.ACCOUNT_TYPE] = accountType
contextMap[common.ACCOUNT_HASH] = util.CalculateHash(accountId)
log.WithFields(fields).Debugf("AddContextForPods Successfully fetched AcntId='%s' and AcntType='%s' from Grp Svc", accountId, accountType)
log.WithFields(safeFields).Debugf("AddContextForPods Successfully fetched AcntId='%s' and AcntType='%s' from Grp Svc", accountId, accountType)

accountProducts, err := ws.GroupServiceConnector.GetAccountProducts(accountId, fields)
if err != nil {
@@ -178,13 +179,13 @@
contextMap[common.ACCOUNT_STATE] = accountState
}
} else {
log.WithFields(fields).Error("Failed to unmarshall AccountProducts")
log.WithFields(safeFields).Error("Failed to unmarshall AccountProducts")
}

}

xhttp.IncreaseGrpServiceFetchCounter(contextMap[common.MODEL], contextMap[common.PARTNER_ID])
log.WithFields(fields).Debugf("AddContextForPods AcntId='%s' ,AccntPrd='%v' Successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap)
log.WithFields(safeFields).Debugf("AddContextForPods AcntId='%s' ,AccntPrd='%v' Successfully retrieved from xac/ada", contextMap[common.ACCOUNT_ID], contextMap)
// Create PodData and AccountServiceData with retrieved information
podData = &PodData{
AccountId: contextMap[common.ACCOUNT_ID],
Copilot is powered by AI and may make mistakes. Always verify output.
accountType := xAccountId.GetAccountType()
contextMap[common.ACCOUNT_ID] = accountId
contextMap[common.ACCOUNT_TYPE] = accountType
log.WithFields(fields).Debugf("AddLogUploaderContext Successfully fetched AcntId='%s' and AcntType='%s' from Grp Svc", accountId, accountType)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

Sensitive data returned by HTTP request headers
flows to a logging call.

Copilot Autofix

AI about 1 month ago

General fix: do not place sensitive/untrusted request-derived values into a shared logging field map that is later logged. If the value is needed for processing, keep it in contextMap, but either omit it from log fields or store an obfuscated/redacted form.

Best minimal fix here (without changing behavior of context processing): in dataapi/log_uploader_context.go, inside NormalizeLogUploaderContext, stop storing raw estbIp in fields. Replace fields[common.ESTB_IP] = estbIp with a redacted marker (or remove key entirely). This preserves use of the real IP in contextMap while preventing clear-text logging via WithFields(fields) at line 96 and elsewhere.

Only one file needs editing:

  • dataapi/log_uploader_context.go in NormalizeLogUploaderContext around current lines 40–42.

No new methods or imports are required.

Suggested changeset 1
dataapi/log_uploader_context.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/dataapi/log_uploader_context.go b/dataapi/log_uploader_context.go
--- a/dataapi/log_uploader_context.go
+++ b/dataapi/log_uploader_context.go
@@ -39,7 +39,7 @@
 	NormalizeCommonContext(contextMap, common.ESTB_MAC_ADDRESS, common.ECM_MAC_ADDRESS)
 	estbIp := util.GetIpAddress(r, contextMap[common.ESTB_IP], fields)
 	contextMap[common.ESTB_IP] = estbIp
-	fields[common.ESTB_IP] = estbIp
+	fields[common.ESTB_IP] = "[REDACTED]"
 	// check if request is for partner
 	if usePartnerAppType && contextMap[common.APPLICATION_TYPE] == shared.STB {
 		if appType := GetApplicationTypeFromPartnerId(contextMap[common.PARTNER_ID]); appType != "" {
EOF
@@ -39,7 +39,7 @@
NormalizeCommonContext(contextMap, common.ESTB_MAC_ADDRESS, common.ECM_MAC_ADDRESS)
estbIp := util.GetIpAddress(r, contextMap[common.ESTB_IP], fields)
contextMap[common.ESTB_IP] = estbIp
fields[common.ESTB_IP] = estbIp
fields[common.ESTB_IP] = "[REDACTED]"
// check if request is for partner
if usePartnerAppType && contextMap[common.APPLICATION_TYPE] == shared.STB {
if appType := GetApplicationTypeFromPartnerId(contextMap[common.PARTNER_ID]); appType != "" {
Copilot is powered by AI and may make mistakes. Always verify output.
Copilot AI review requested due to automatic review settings April 28, 2026 08:11
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

Copy link
Copy Markdown

@github-advanced-security github-advanced-security AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

Copilot AI review requested due to automatic review settings May 18, 2026 08:08
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.