Issues trying to create YAML device file from example

[clavin-xlnx]

I am attempting to generate a YAML-equivalent version of an Interchange device file. I am following the example shown on the main README.md :

Annotated the xc7a35tcpg236-1 database with constraints:

python3 -mfpga_interchange.patch \
    --schema_dir $RAPIDWRIGHT_PATH/interchange \
    --schema device \
    --patch_path constraints \
    --patch_format yaml \
    xc7a35tcpg236-1.device \
    test_data/series7_constraints.yaml \
    xc7a35tcpg236-1_constraints.device

I recognize that --schema_dir $RAPIDWRIGHT_PATH/interchange is obsolete and used a separate clone of the fpga-internchange-schema repo. I ran into two problems:

1. After building the python-fpga-interchange module and attempting to run the command above, I get this error message:

Traceback (most recent call last):
  File "/group/zircon2/clavin/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/group/zircon2/clavin/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/clavin/tools/python-fpga-interchange/fpga_interchange/patch.py", line 72, in <module>
    main()
  File "/home/clavin/tools/python-fpga-interchange/fpga_interchange/patch.py", line 57, in main
    args.root_schema_path)
  File "/home/clavin/tools/python-fpga-interchange/fpga_interchange/convert.py", line 158, in get_schema
    schemas = Interchange(schema_dir)
  File "/home/clavin/tools/python-fpga-interchange/fpga_interchange/interchange_capnp.py", line 857, in __init__
    imports=search_path)
  File "capnp/lib/capnp.pyx", line 4342, in capnp.lib.capnp.load
  File "capnp/lib/capnp.pyx", line 3572, in capnp.lib.capnp.SchemaParser.load
capnp.lib.capnp.KjException: clavin/workspace/RapidWright/interchange/fpga-interchange-schema/interchange/References.capnp:15: failed: Import failed: /capnp/java.capnp

It seems the directions do not specify how I should communicate the java.capnp file to the tool. I commented out all references to use of the file as they are not needed here, so I moved on.

2. After commenting out the java.capnp references, the tool completes without any output. It does create the expected output file, but the file is binary, I was expecting it to output a YAML equivalent.

I setup python-fpga-interchange with the following (I am using Python 3.7.3 from an Anaconda release):

git clone https://github.com/SymbiFlow/python-fpga-interchange.git
cd python-fpga-interchange/
python3 -mvenv env
source env/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

[gatecat]

1. For the first issue, I believe following the same steps that we do in CI should resolve it:

https://github.com/YosysHQ/nextpnr/blob/0c1ee5fad103b65d0a194596b2006c6622742e05/.github/ci/build_interchange.sh#L13-L17
(I thought this was documented somewhere else but I think those might have been lost)

2. In theory, the command you want is to convert to YAML is:

python -m fpga_interchange.convert --schema_dir $FPGA_INTERCHANGE_SCHEMA/interchange --schema device --input_format capnp --output_format yaml xc7a35t.device xc7a35t.yaml

but this seems to be failing with:

  File "/home/gatecat/python-fpga-interchange/fpga_interchange/converters.py", line 374, in to_writer
    set_value(field_which, deference_fun(value))
  File "/home/gatecat/python-fpga-interchange/fpga_interchange/converters.py", line 326, in <lambda>
    deference_fun = lambda value: writer.dereference_value(
  File "/home/gatecat/python-fpga-interchange/fpga_interchange/rapidyaml_support.py", line 114, in dereference_value
    ref_id = parent.get_list_id(annotation_type.field, value)
  File "/home/gatecat/python-fpga-interchange/fpga_interchange/rapidyaml_support.py", line 94, in get_list_id
    field_id = self.field_ids[field]
KeyError: 'wireTypes'

so I think there's an issue there which needs to be investigated, cc @acomodi

(the patch tool is to patch a binary database with a yaml patch, rather than to convert a database to yaml)

GitHub

nextpnr/build_interchange.sh at 0c1ee5fad103b65d0a194596b2006c6622742e05 · YosysHQ/nextpnr

nextpnr portable FPGA place and route tool. Contribute to YosysHQ/nextpnr development by creating an account on GitHub.

[clavin-xlnx]

Thanks @gatecat for the quick response. I think the error from 1 is that there is a very small Cap'n Proto schema file that is imported but the compiler doesn't seem to find it from my current cap'n proto build. Unfortunately, I'm on a system where I cannot install global packages and must install in local locations, not sure if that is why.

[gatecat]

Unfortunately, when I looked into it it seemed like capnp only looked at a limited number of hardcoded, global, paths:
https://github.com/capnproto/capnproto/blob/a1c25db52979b91b19aae99e358bdbcab5f0e368/c%2B%2B/src/capnp/compiler/capnp.c%2B%2B#L339-L346

but maybe there is another trick.

GitHub

capnproto/capnp.c++ at a1c25db52979b91b19aae99e358bdbcab5f0e368 · capnproto/capnproto

Cap'n Proto serialization/RPC system - core tools and C++ library - capnproto/capnp.c++ at a1c25db52979b91b19aae99e358bdbcab5f0e368 · capnproto/capnproto

[mithro]

https://github.com/capnproto/capnproto/blob/a1c25db52979b91b19aae99e358bdbcab5f0e368/c%2B%2B/src/capnp/compiler/capnp.c%2B%2B#L287-L295

GitHub

capnproto/capnp.c++ at a1c25db52979b91b19aae99e358bdbcab5f0e368 · capnproto/capnproto

Cap'n Proto serialization/RPC system - core tools and C++ library - capnproto/capnp.c++ at a1c25db52979b91b19aae99e358bdbcab5f0e368 · capnproto/capnproto