|
string = re.sub(re.compile("--.*?\n", re.DOTALL ) ,"" ,string) # remove all occurance streamed comments (/*COMMENT */) from string |
an adversarial example would be
>>> string = """
SELECT SPLIT_PART(text_with_double_hyphen_delimiter, '--', 1)
FROM table_with_weird_text
"""
>>> remove_comments(string)
"SELECT SPLIT_PART(text_with_double_hyphen_delimiter, 'FROM table_with_weird_text"
I think you need to truly tokenize the query to not hit this edge case.
py-queryparser/queryParser.py
Line 59 in 8876b33
an adversarial example would be
I think you need to truly tokenize the query to not hit this edge case.