Skip to content

Error Retrieval calculation error & Fix some detailed problems - #11

Open
ztxtech wants to merge 2 commits into
salesforce:mainfrom
ztxtech:main
Open

Error Retrieval calculation error & Fix some detailed problems#11
ztxtech wants to merge 2 commits into
salesforce:mainfrom
ztxtech:main

Conversation

@ztxtech

@ztxtech ztxtech commented Nov 24, 2024

Copy link
Copy Markdown

Change outdated methods and fix some modules that default to specifying CUDA as the device.

@salesforce-cla

Copy link
Copy Markdown

Thanks for the contribution! Before we can merge this, we need @ztxtech to sign the Salesforce Inc. Contributor License Agreement.

@ztxtech

ztxtech commented Nov 25, 2024

Copy link
Copy Markdown
Author

image

There is a calculation error here. The Attention value should be weighted and memorized instead of the corresponding index.

Original code at fsnet_.py (line 105-108)

...
            v, idx = torch.topk(att, 2)
            ww = torch.index_select(self.W, 1, idx)
            idx = idx.unsqueeze(1).float()
            old_w = ww @ idx
...

Corrected code

...
            v, idx = torch.topk(att, 2)
            ww = torch.index_select(self.W, 1, idx)
            old_w = ww @ v.clone().detach().unsqueeze(1).float()
...

@ztxtech ztxtech changed the title Fix some detailed problems Error Retrieval calculation error & Fix some detailed problems Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant