Skip to content

[NO-ISSUE] OpenAI 호출 관측 메트릭 추가 #285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hyoj-dev merged 2 commits into from
Jul 20, 2026
Merged

[NO-ISSUE] OpenAI 호출 관측 메트릭 추가 #285

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
957b470
chore: OpenAI 호출 관측 메트릭 추가
hyoj-dev Jul 20, 2026
15d2967
chore: OpenAI 호출 관측 메트릭 추가 #284
hyoj-dev Jul 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 87 additions & 27 deletions src/main/kotlin/com/didit/adapter/integration/ai/OpenAiClient.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,28 @@ import com.didit.application.retrospect.required.GeneratedDeepQuestion
import com.didit.domain.shared.Job
import com.fasterxml.jackson.annotation.JsonIgnoreProperties
import com.fasterxml.jackson.annotation.JsonProperty
import com.fasterxml.jackson.core.JsonProcessingException
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.kotlin.readValue
import io.micrometer.core.instrument.MeterRegistry
import io.micrometer.core.instrument.Timer
import org.slf4j.LoggerFactory
import org.springframework.beans.factory.annotation.Value
import org.springframework.http.MediaType
import org.springframework.stereotype.Component
import org.springframework.transaction.support.TransactionSynchronizationManager
import org.springframework.web.client.ResourceAccessException
import org.springframework.web.client.RestClient
import org.springframework.web.client.RestClientResponseException
import org.springframework.web.client.body
import java.net.SocketTimeoutException

@Component
class OpenAiClient(
private val restClient: RestClient,
private val objectMapper: ObjectMapper,
private val feedbackPrompts: FeedbackPrompts,
private val meterRegistry: MeterRegistry,
@param:Value("\${openai.api-key}") private val apiKey: String,
@param:Value("\${openai.chat.model}") private val model: String,
) : AIClient {
Expand Down Expand Up @@ -61,35 +69,87 @@ class OpenAiClient(
schemaName: String,
schema: Map<String, Any>,
): OpenAiResponse {
val rawResponse =
restClient
.post()
.uri(URL)
.header("Authorization", "Bearer $apiKey")
.contentType(MediaType.APPLICATION_JSON)
.body(
OpenAiRequest(
model = model,
instructions = SYSTEM_PROMPT,
input = prompt,
maxOutputTokens = 3000,
text =
OpenAiTextFormat(
format =
OpenAiJsonSchemaFormat(
name = schemaName,
schema = schema,
),
),
),
).retrieve()
.body<String>() ?: throw RuntimeException("OpenAI 응답을 받지 못했습니다.")

logger.debug("OpenAI 전체 응답: $rawResponse")

return objectMapper.readValue<OpenAiResponse>(rawResponse)
val operation = if (schemaName == "deep_question") "deep_question" else "summary"
val sample = Timer.start(meterRegistry)
var outcome = "success"

logger.info(
"OpenAI request started - operation: {}, transactionActive: {}",
operation,
TransactionSynchronizationManager.isActualTransactionActive(),
)

try {
val rawResponse =
restClient
.post()
.uri(URL)
.header("Authorization", "Bearer $apiKey")
.contentType(MediaType.APPLICATION_JSON)
.body(
OpenAiRequest(
model = model,
instructions = SYSTEM_PROMPT,
input = prompt,
maxOutputTokens = 3000,
text =
OpenAiTextFormat(
format =
OpenAiJsonSchemaFormat(
name = schemaName,
schema = schema,
),
),
),
).retrieve()
.body<String>() ?: throw RuntimeException("OpenAI 응답을 받지 못했습니다.")

logger.debug("OpenAI 전체 응답: $rawResponse")

return objectMapper.readValue<OpenAiResponse>(rawResponse)

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 20, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Mark malformed OpenAI outputs as errors

When OpenAI returns a 200 response whose envelope deserializes but output_text is missing or the summary JSON is malformed, this returns before those checks run, so the finally records outcome=success and the new error counter is not incremented; parseSummary then throws outside this catch path. That makes the new OpenAI metrics report failed user-facing summary generations as successful, so the output-text/schema parsing should be included in the measured error path or otherwise update the outcome before propagating the failure.

Useful? React with 👍 / 👎.

} catch (exception: Exception) {
outcome = "error"
meterRegistry
.counter(
"didit.openai.request.errors",
"operation",
operation,
"type",
classifyException(exception),
).increment()
throw exception
} finally {
sample.stop(
Timer
.builder("didit.openai.request.duration")
.description("OpenAI API request duration")
.tag("operation", operation)
.tag("outcome", outcome)
.publishPercentileHistogram()
.register(meterRegistry),
)
}
}

private fun classifyException(exception: Exception): String =
when (exception) {
is RestClientResponseException ->
when {
exception.statusCode.value() == 429 -> "rate_limit"
exception.statusCode.is4xxClientError -> "client_error"
exception.statusCode.is5xxServerError -> "server_error"
else -> "http_error"
}

is ResourceAccessException ->
if (exception.causeSequence().any { it is SocketTimeoutException }) "timeout" else "connection_error"

is JsonProcessingException -> "parse_error"
else -> "unknown"
}

private fun Throwable.causeSequence(): Sequence<Throwable> = generateSequence(this) { it.cause }

private fun parseDeepQuestion(response: OpenAiResponse): GeneratedDeepQuestion =
runCatching {
val question = objectMapper.readValue<DeepQuestionDto>(response.outputText).question
Expand Down
5 changes: 5 additions & 0 deletions src/main/resources/application-local.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@ spring:
flyway:
enabled: false

server:
tomcat:
mbeanregistry:
enabled: true

management:
endpoints:
web:
Expand Down
5 changes: 4 additions & 1 deletion src/main/resources/application-prod.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,9 @@ spring:

server:
port: 8080
tomcat:
mbeanregistry:
enabled: true

management:
endpoints:
Expand All @@ -54,4 +57,4 @@ admin:
cors:
allowed-origins: ${ADMIN_CORS_ALLOWED_ORIGINS}
invite:
base-url: ${ADMIN_INVITE_BASE_URL}
base-url: ${ADMIN_INVITE_BASE_URL}
Loading