Skip to content

Commit bc31c9f

Browse files
author
Dylan Huang
committed
Add error handling for evaluator creation in evaluation module
- Implemented a try-except block to handle APIStatusError during evaluator creation. - Added logic to check for existing evaluators and retrieve the existing one if a conflict occurs (status code 409). - Enhanced logging for better traceability of evaluator creation process.
1 parent 060d72c commit bc31c9f

File tree

1 file changed

+14
-6
lines changed

1 file changed

+14
-6
lines changed

eval_protocol/evaluation.py

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -202,12 +202,20 @@ def create(self, evaluator_id, display_name=None, description=None):
202202
logger.info(f"Creating evaluator '{evaluator_id}' for account '{account_id}'...")
203203

204204
try:
205-
# Create evaluator using SDK
206-
result = client.evaluators.create(
207-
evaluator_id=evaluator_id,
208-
evaluator=evaluator_params,
209-
)
210-
logger.info(f"Successfully created evaluator '{evaluator_id}'")
205+
# Try to create evaluator using SDK
206+
try:
207+
result = client.evaluators.create(
208+
evaluator_id=evaluator_id,
209+
evaluator=evaluator_params,
210+
)
211+
logger.info(f"Successfully created evaluator '{evaluator_id}'")
212+
except fireworks.APIStatusError as create_error:
213+
if create_error.status_code == 409:
214+
# Evaluator already exists, get the existing one and proceed to create a new version
215+
logger.info(f"Evaluator '{evaluator_id}' already exists, creating new version...")
216+
result = client.evaluators.get(evaluator_id=evaluator_id)
217+
else:
218+
raise
211219

212220
# Upload code as tar.gz to GCS
213221
evaluator_name = result.name # e.g., "accounts/pyroworks/evaluators/test-123"

0 commit comments

Comments
 (0)