trim cluster output table

[lmullany]

The cluster output table produced by find_clusters() could be reduced to include only columns that are necessary. There may be some redundant or unnecessary columns retained from intermediate steps.

[lmullany]

Current columns include the following:

• target: the cluster center
• observed: the number of cases in the cluster
• spl_threshold: the threshold for this number of observed and expected, given the ingested spline lookup
• date: the start date of the cluster
• test_totals: the total number of cases in the test period from date to detect_date
• location: (one of the locations in the cluster) - this is not particularly helpful. For multiple location clusters, showing only one is not that interesting, and could be confusing
• base_clust_sums: this is the number of cases in this cluster during the baseline period (plus any adjustments related to the baseline_adjustment method)
• distance_value: this is the distance between the cluster center and the center of the furthest location in the cluster (for single-location clusters, this value is therefore zero)
• count this is the number of cases in the cluster center (i.e. in target) on the initial date of the cluster
• count_sum is the total count of cases in the cluster center (i.e in target) from date to detect_date
• detect_date: this is a constant; will be equal to the detect_date passed to the find_clusters() function
• baseline_total: this is a constant and is the total of all cases in the baseline period (plus any adjustments related to the baseline_adjustment parameter
• expected: this is the expected number, given the baseline distribution of cases
• log_obs_exp: this is the log of observed/expected
• min_dist: this is the distance from the cluster center to the furthest cluster center of all locations included in the cluster. Note that it is both redundant (same as distance_value and poorly named
• alert_ratio: this is the ratio of the log_obs_exp to spl_thresh
• alert_gap this is the difference between log_obs_exp and spl_thresh
• id: this is an internally generated id for this row
• nr_locs: this is the number of locations in this cluster
• max_date this is the maximum_date of the cluster, and since we only implement the prospective approach, this date is always equal to detect_date

Candidates to remove:

• test_totals - not useful
• location - only shows one location, so not useful
• base_clust_sums - not helpful
• count - only shows count in the centroid on the first day
• baseline_total (could be attribute)
• min_dist: redundant
• id: internal only
• max_date

Candidates to rename

• target - to cluster_center
• date - to cluster_start_date
• distance_value to max_distance
• count_sum to cluster_center_observed
• detect_date to cluster_end_date
• nr_locs to n_cluster_locations

[howardburkom]

1. alert_ratio can be dropped and has never been used since testing, log_obs_exp/spl_thresh was not as useful in testing as log_obs_exp-spl_thresh
2. Suggesting distance_value -> cluster_radius or max_cluster_distance
3. Suggesting count_sum -> observed_cluster_count
   For variables to remove from the cleaned table:
4. test_totals -> test_period_total_count
5. base_clust_sums -> sum_cluster_locs_in_baseline
6. count -> first_day_clust_center_count
7. location -> farthest_location_from_center
8. baseline_total -> baseline_total_countDrop min_dist and max_date completely, and is there a reason to keep the 'id' variable that compress_cluster_fast uses?