Skip to content

get_performance_metrics() in LLAMA_3_Fine_Tuning_for_Sequence_Classification.ipynb has completely wrong results #1

@chadj2

Description

@chadj2

You are rounding the scores and predictions and this changes all the categories so that { 0.25, 0.5, 0.75 } -> 1.0.

def get_performance_metrics(df_test):
  y_test = df_test.score.round()
  y_pred = df_test.predictions.round()
  print(f"comparing test {y_test} and pred {y_pred}")

  print("Confusion Matrix:")
  print(confusion_matrix(y_test, y_pred))

  print("\nClassification Report:")
  print(classification_report(y_test, y_pred))

You can't pass floats to confusion_matrix() because it will assume you have continuous values. You could pass the category codes but another approach would be to convert the floats to string.

  y_test = df_test.score.astype(str)
  y_pred = df_test.predictions.astype(str)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions