Skip to content

Different Results Compared with Vanilla Marabou #3

@LIJYann

Description

@LIJYann

When running the code with narv and vanilla marabou, I got two different results, which I believe is abnormal. I’m not sure if this is caused by mistakes I made while addressing some errors in your code, or if it stems from issues in the code itself. Could you please check this when you have a chance?

Below are the specific commands I executed and the corresponding outputs:

Command with narv:

python3 narv.py -nn 1_1 -pid property_2 -a global -r global -m marabou_with_ar

Output:[('net_name', 'ACASXU_experimental_v2a_1_1.nnet'), ('property_id', 'property_2'), ('abstraction_time', 0), ('query_result', 'UNSAT'), ('num_of_refine_steps', 0), ('total_ar_query_time', 0.0945885181427002), ('ar_times', '[0.0]'), ('ar_sizes', '[315]'), ('refine_sequence_times', '[]'), ('last_net_data', '{"layer_sizes": {"0": 5, "1": 50, "2": 50, "3": 50, "4": 50, "5": 50, "6": 50, "7": 5, "8": 4, "9": 1}, "num_nodes": 315, "num_layers": 10, "num_hidden_layers": 8}'), ('counter-example', [])]

Command with vanilla marabou:

python3 narv.py -nn 1_1 -pid property_2 -m marabou

Output:[('net_name', 'ACASXU_experimental_v2a_1_1.nnet'), ('property_id', 'property_2'), ('query_result', 'SAT'), ('orig_query_time', 0.22386670112609863), ('net_data', '{"layer_sizes": {"0": 5, "1": 50, "2": 50, "3": 50, "4": 50, "5": 50, "6": 50, "7": 5, "8": 4, "9": 1}, "num_nodes": 315, "num_layers": 10, "num_hidden_layers": 8}')]

Key difference: The query_result is "UNSAT" when using marabou_with_ar, but "SAT" with vanilla marabou.
Thanks for your help on advance.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions