Thanks for the great implementation!
I am considering having the fgram model leverage embeddings from the main model to produce fgram embeddings, rather than using the original IDs. Looking at the current implementation, it seems to use fgram IDs to generate fgram embeddings, which are then added to the base embedding. Does this align with the original design of SCONE?
Or it is a kind of better variant of SCONE?
Thanks for the great implementation!
I am considering having the fgram model leverage embeddings from the main model to produce fgram embeddings, rather than using the original IDs. Looking at the current implementation, it seems to use fgram IDs to generate fgram embeddings, which are then added to the base embedding. Does this align with the original design of SCONE?
Or it is a kind of better variant of SCONE?