Question about SQL Generation

[ZZZYYYLL]

Thank you for your meaningful work!

I'm interested with this work, however, I have some trouble with sql query in BIGQUERY

When I used retrieved schemas to generate sql for querying in BIGQUERY, the BIGQUERY API return 403 error.
403 GET Quota exceeded: Your project exceeded quota for free query bytes scanned. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas

However, some SQL queries will report this error, but some SQL generation will not have this error. Will you encounter this problem during the process of the experiment?

When I view the console of BIGQUERY, I found that there was no consumption at all when calling BIGQUERY. Is this normal? Will there be no such problem if one upgrades to a paying user?

[Satissss]

We encountered the same error before, likely because some SQL queries involve data that's too large, exceeding the fixed quota limits the BigQuery provided.
Since BigQuery quotas reset after a period of time, so one workaround is to create 2–3 BigQuery accounts and rotate their use to avoid overloading a single account.
We haven’t tried becoming a paid user, but it might help to alleviate the issue.

[ZZZYYYLL]

Thanks for your reply. It is a very meaningful work. However, Do you have any plans to release the code and prompt for the SQL generation part?

[Satissss]

In the SQL generation stage, we applied LinkAlign to the lower-ranked DIN-SQL with only minor adjustments for the new Spider 2 dataset. DIN-SQL is used solely to demonstrate the effectiveness of our approach, rather than an optimal solution for SQL generation. In fact, LinkAlign can be integrated with any exiting methods, such as those ranked top on the BIRD leaderboard, to support large-scale, multi-database scenarios. We plan to open-source an "end-to-end" Text-to-SQL tool in the future, which is based on LinkAlign.

[ZZZYYYLL]

I see in spider2_dev/table_preprocess.json just provide three examples, however, is there any code for generating table_preprocess.json? I didn't find it in the project.

[Satissss]

The table_preprocess.json file is generated by the Spider 2.0 code. You can run spider2_preprocess.py located in Spider2/spider2-lite/baselines/dinsql/preprocessed_data to create it. We only included a small example in the repo to help demonstrate how to run LinkAlign successfully.

[ZZZYYYLL]

Is this released code really the final code? I found a lot of errors during the reproduction process.

In utils.py

1. In function parse_schemas_from_nodes. You save the information of the schema in the following dictionary format

        schema = {
            "Database name": meta_data["db_id"],
            "Table Name": meta_data["table_name"],
            "Field Name": col_info["column_name"],
            'Type': col_info["column_types"],
            'column_descriptions': col_info.get("column_descriptions", None),
            'sample_rows': col_info.get("sample_rows", None),
            'turn_n': node.metadata.get("turn_n", None)
        }

However, when you parse schema from dataframe, you use column_types, table_name, column name

def parse_schema_from_df(df: pd.DataFrame) -> str:
    grouped = df.groupby('table_name')
    output_lines = []

    for table_name, group in grouped:
        columns = []
        for _, row in group.iterrows():
            col_type = row["column_types"]
            if isinstance(col_type, str) and len(col_type) > 150:
                col_type = col_type[:150]
            columns.append(f'{row["column_name"]}(Type: {col_type})')

        line = f'### Table {table_name}, columns = [{", ".join(columns)}]'
        output_lines.append(line)

    return "\n".join(output_lines)

In GenerateSchemas.py

1. In line 27, you declared a variable named external_info_path. Does this refer to the external knowledge provided by spider2-lite? I see that in your load_external_knowledge function, what is loaded is {instance_id}.txt. Did you process the markdown file into a txt file for each instance_id pair? However, I didn't see the relevant code during the preprocess stage.

parser.add_argument("--external_info_path", type=str, required=False, default=r"./spider2_dev/external_knowledge")

2. In line 302

        schema_links = SchemaLinkingTool.generate_by_multi_agent(llm=llm, query=row["question"],
                                                                 context=context,
                                                                 turn_n=1, linker_num=3
                                                                 )

However the get_schema function doesn't input the "row" parameter. Do you want to use query=question

[Satissss]

Thank you, you are right. These are errors from recent updates which have been corrected.

Additionally, load_external_knowledge is not limited to the Spider 2.0 dataset. I have added tools/external.py to extract external knowledge for queries.

[ZZZYYYLL]

Thank you for your reply. Previously, you mentioned that your generation stage referred to the Din-SQL process and only made minor adjustments to it. It would be convenient for you to ask in which processes the adjustments would be made. For example: Din-SQL builds few-shot, but these few-shot are applicable to the BIRD dataset. Will you modify this part of the few-shot? Or are there any specific optimizations for other processes?

[tushika22]

Hey, I wanted to know that we are already giving db_id as input to the get_schema function so where is the database retrieval actually happening please correct me if I am wrong

[russellkim]

From what I have seen in several studies, DIN tends to have lower performance on benchmarks such as BIRD and Spider 2.0. Also, in practice, strong schema linking by itself doesn't always lead to high-quality SQL generation, especially for more complex queries. That said, the schema linking module the authors focus on in the paper is certainly an important component, especially in multi-database environments where scalable and accurate linking is essential. I believe that releasing the SQL generation code would be very helpful for the community to better understand and fairly evaluate the end-to-end performance of LinkAlign, and perhaps also explore how it might be combined with other SQL generation methods.

[Satissss]

Thanks for the comment. Indeed, LinkAlign offers an advantage beyond its multi-database capabilities: it compresses large, redundant database schemas into small subsets (e.g., from 2000+ to 100+ columns) that fit within the LLMs’ context window without omitting key schemas. This addresses major challenges faced by DIN-like baselines in the Spider 2.0 benchmark.

BTW, we have also developed a lightweight, config-driven Text-to-SQL framework called Squrve. It includes end-to-end implementations of LinkAlign along with several other popular open-source baseline methods. You can simply set generate_type to the method identifier (e.g., LinkAlignGenerator) to test and compare their performance.