RUMS: Response Utility optimized Memory Selection

This repository provides the code and scripts to reproduce experiments from the paper "Beyond Similarity for Personalization: User Memory Selection via Response-Utility Optimization". It introduces RUMS: Response-Utility (optimized) Memory Selection, a framework designed to evaluate the informativeness of user-specific memory items by estimating mutual information (MI) between model responses generated with and without selected features.

---

Overview

RUMS enables the analysis of which user memory items meaningfully influence model responses in personalization or controllable generation tasks.

---

Quickstart

Setting Up the Environment

To set up the environment to run the code, ensure you have Python 3.10+ installed, then run:

python3 -m venv venv
source venv/bin/activate  # On Windows, use `venv\Scripts\activate`
pip install -r requirements.txt

Running RUMS Algorithm

We include the Python file RUMS_alg.py to run the RUMS algorithm. This script selects the best subset of candidate memory items for each input and user profile pair. It does this by evaluating the informativeness of user-specific items by estimating mutual information between model responses generated with and without selected features.

To run the script:

python RUMS_alg.py \
  --model_name meta-llama/Llama-3.1-8B \
  --input_csv /path/to/your/input.csv \
  --output_json /path/to/save/results.json \
  --profile_json /path/to/profiles.json \
  --max_features 5 \
  --n_samples 5 \
  --n_users 5

Citation

If you find this repository useful, please consider citing our paper:

@misc{your2025rums,
  title={RUMS: Reproducible Uncertainty Modeling for Style},
  author={Your Name and Co-author Name},
  year={2025},
  eprint={arXiv:xxxx.xxxxx},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/xxxx.xxxxx},
}