-
Notifications
You must be signed in to change notification settings - Fork 0
Different Results Compared with Vanilla Marabou #3
Description
When running the code with narv and vanilla marabou, I got two different results, which I believe is abnormal. I’m not sure if this is caused by mistakes I made while addressing some errors in your code, or if it stems from issues in the code itself. Could you please check this when you have a chance?
Below are the specific commands I executed and the corresponding outputs:
Command with narv:
python3 narv.py -nn 1_1 -pid property_2 -a global -r global -m marabou_with_ar
Output:[('net_name', 'ACASXU_experimental_v2a_1_1.nnet'), ('property_id', 'property_2'), ('abstraction_time', 0), ('query_result', 'UNSAT'), ('num_of_refine_steps', 0), ('total_ar_query_time', 0.0945885181427002), ('ar_times', '[0.0]'), ('ar_sizes', '[315]'), ('refine_sequence_times', '[]'), ('last_net_data', '{"layer_sizes": {"0": 5, "1": 50, "2": 50, "3": 50, "4": 50, "5": 50, "6": 50, "7": 5, "8": 4, "9": 1}, "num_nodes": 315, "num_layers": 10, "num_hidden_layers": 8}'), ('counter-example', [])]
Command with vanilla marabou:
python3 narv.py -nn 1_1 -pid property_2 -m marabou
Output:[('net_name', 'ACASXU_experimental_v2a_1_1.nnet'), ('property_id', 'property_2'), ('query_result', 'SAT'), ('orig_query_time', 0.22386670112609863), ('net_data', '{"layer_sizes": {"0": 5, "1": 50, "2": 50, "3": 50, "4": 50, "5": 50, "6": 50, "7": 5, "8": 4, "9": 1}, "num_nodes": 315, "num_layers": 10, "num_hidden_layers": 8}')]
Key difference: The query_result is "UNSAT" when using marabou_with_ar, but "SAT" with vanilla marabou.
Thanks for your help on advance.