Add VescTool Data Import Format

[scottrcarlson]

First of all, nice tool!

I wrote a quick python script to convert VescTool data output format to match float control csv, but would like to request adding direct support.

VescTool Data for Testing
2024-10-10_12-54-08.csv

Here is the file format, it is semi-colon delimited:

ms_today;input_voltage;temp_mos_max;temp_mos_1;temp_mos_2;temp_mos_3;temp_motor;current_motor;current_in;d_axis_current;q_axis_current;erpm;duty_cycle;amp_hours_used;amp_hours_charged;watt_hours_used;watt_hours_charged;tachometer;tachometer_abs;encoder_position;fault_code;vesc_id;d_axis_voltage;q_axis_voltage;ms_today_setup;amp_hours_setup;amp_hours_charged_setup;watt_hours_setup;watt_hours_charged_setup;battery_level;battery_wh_tot;current_in_setup;current_motor_setup;speed_meters_per_sec;tacho_meters;tacho_abs_meters;num_vescs;ms_today_imu;roll;pitch;yaw;accX;accY;accZ;gyroX;gyroY;gyroZ;gnss_posTime;gnss_lat;gnss_lon;gnss_alt;gnss_gVel;gnss_vVel;gnss_hAcc;gnss_vAcc;

Here is my python script, with most of the fields correlated, I am missing a few.

Output of Converted File format for reference:
2024-10-10_12-54-08_converted.csv

import csv
# Define the mapping from the old header to the new header
field_mapping = {
    'ms_today': 'Time(s)',
    'input_voltage': 'Voltage',
    'temp_mos_max': 'T-Mosfet',
    'temp_motor': 'T-Mot',
    'current_motor': 'I-Motor',
    'current_in': 'I-Battery',
    'duty_cycle': 'Duty%',
    'amp_hours_used': 'Ah',
    'amp_hours_charged': 'Ah Charged',
    'watt_hours_used': 'Wh',
    'watt_hours_charged': 'Wh Charged',
    'erpm': 'ERPM',
    'fault_code': 'Motor-Fault',
    'vesc_id': 'State(num)',
    'roll': 'Roll',
    'pitch': 'Pitch',
    'speed_meters_per_sec': 'Speed(mph)',
    'gnss_lat': 'GPS-Lat',
    'gnss_lon': 'GPS-Long',
    'gnss_hAcc': 'GPS-Accuracy',
    'gnss_alt': 'Altitude(m)',
    # Add other mappings as needed
}

# Define the new header order
new_header = ['Time(s)', 'State', 'Distance(mi)', 'Speed(mph)', 'Duty%', 'Voltage', 'I-Battery', 'I-Motor', 'I-FldWeak',
              'Requested Amps', 'Pitch', 'Roll', 'Setpoint', 'SP-ATR', 'SP-Carve', 'T-Mosfet', 'T-Mot', 'ADC1', 'ADC2',
              'Motor-Fault', 'Ah', 'Ah Charged', 'Wh', 'Wh Charged', 'ERPM', 'Altitude(m)', 'State(num)', 'True Pitch',
              'SP-TrqTlt', 'SP-BrkTlt', 'SP-Remote', 'T-Batt', 'I-Booster', 'GPS-Lat', 'GPS-Long', 'GPS-Accuracy']

def convert_file(input_file_path):
    output_file_path = input_file_path.replace('.csv', '_converted.csv').replace('.txt', '_converted.csv')

    with open(input_file_path, mode='r', newline='', encoding='utf-8') as infile, \
            open(output_file_path, mode='w', newline='', encoding='utf-8') as outfile:

        reader = csv.DictReader(infile, delimiter=';')
        writer = csv.DictWriter(outfile, fieldnames=new_header)

        # Write the new header to the output file
        writer.writeheader()

        for row in reader:
            # Create a new row with the new header and mapped data
            new_row = {new_field: row.get(old_field, '') for old_field, new_field in field_mapping.items()}

            # Scale the duty cycle by 100 if it exists in the row
            if 'Duty%' in new_row and new_row['Duty%']:
                try:
                    new_row['Duty%'] = float(new_row['Duty%']) * 100
                except ValueError:
                    new_row['Duty%'] = ''  # Handle cases where the value is not a number

            writer.writerow(new_row)

    print(f"Conversion completed successfully. Converted file saved as: {output_file_path}")

# Usage example
convert_file('2024-10-10_12-54-08.csv')```

[acheronfail]

Hey, thank you so much! I have been thinking that it would be nice for VESC Tool to be able to record data - I'm fairly new to this stuff.

Please teach me how to record ride data with VESC Tool! How do I do that? Is it the RT data logging on the home page?
I wonder if we can have access to Onewheel only features, like the ADC1&2 voltages, etc 🤔

I'll definitely support this directly in Float View - and your conversion script helps as a starting point!

[scottrcarlson]

On the VescTool Start Page, Every time you check the Enable RT Data Logging box, it writes a new file to disk with a timestamp.

We might have to talk with the VescTool devs and/or take a look at data collection and see about adding those board specific values, currently it doesn't appear to be logging those.

Thanks! Let me know if I can help, if I get anymore useful information on this, I'll share it.

[scottrcarlson]

Added Issue/Request on VescTool repo, it's unclear if balance board related data would live in RT Logging or if the Float/Refloat package would have to add a data logging component? I am also new to the VESC stack.

vedderb/vesc_tool#387