Film grain levels are too high when using noise_model to generate a film grain table

[aom-bot]

I used hallmonitor from xiph, and ran the following commands on research-v13.0.0:

ffmpeg -i ~/Movies/cif/hall_monitor_cif.y4m -vf nlmeans -vframes 15 /tmp/hm.y4m
ffmpeg -i /tmp/hm.y4m /tmp/hm.yuv
ffmpeg -i ~/Movies/cif/hall_monitor_cif.y4m -vframes 15 /tmp/hm.noise.yuv
examples/noise_model  --input=/tmp/hm.noise.yuv --input-denoised=/tmp/hm.yuv --width=352 --height=288 --output-grain-table=/tmp/hm.tbl
avmenc /tmp/hm.y4m -o /tmp/hm.obu --cpu-used=1 --qp=120 --enable-keyframe-filtering=0 --deltaq-mode=0 --film-grain-table=/tmp/hm.tbl

I then decoded the generated file using avmdec. I got the following result:

{width=352 height=288}

This doesn't look correct, the grain scale values are way too high. For comparison, when running comparable commands on aomenc, this works correctly.

Note: This issue was migrated from GitLab. See original issue for discussion history.