Skip to content

[RFC] Include opensource industrial-level data #2

Description

@HaFred

Recently, I have been testing the recall agent and the ranking agent with the DeepSeek backend. However, I found that the current mock data corpus is indeed too simple -- Our framework here will be a total overkill. Therefore, I suggest that we can involve industrial-level data for better validation of the framework's real-world effectiveness and further demonstration.

Kuaishou's https://huggingface.co/datasets/OpenOneRec/OpenOneRec-RecIF will be a good start.

  • onerec_bench_release.parquet contains users' video/ad/goods history, engagement labels, user profiles, CoT reasoning
  • video_ad_pid2sid.parquet, product_pid2sid.parquet, pid2caption.parquet contains ID mappings

I can go on to develop this feature. Please kindly assign this to me if it looks ok to you, @guoxun , thank you!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions