Participant level model fitting notes

[ndphillips]

When we do participant level model fitting, I recommend using a softmax choice rule that contains a temperature parameter which changes over trials rather than a fixed one. This will allow for random choice in the beginning of the task (reflecting exploration), and impression weighted choice towards the end (reflecting exploitation).

A common rule used by Yechiam and Busemeyer is to take the typical softmax, but instead of using a fixed temperature Theta, set Theta to be a function of trial. In Yechiam and Busemeyer (2005), they use Theta(t) = (t / 10) ^ c, where c is an individual level choice sensitivity parameter. However, when I don't like about this rule (see attached image) is that it forces Theta(t = 10) = 1. Instead, I suggest the rule Theta(t) = log(t) ^ c.

Psychological Science-2005-Yechiam-973-8.pdf

[mdsteiner]

done. However now that I've added this to the choice rule fitting the parameters for the reinforcement learning model doesn't work for some reason (it returns either inf or NAs at some point and then optim breaks, have to check this out).

[mdsteiner]

I just had a meeting with Renato and also mentioned the somewhat disappointing modeling results and when we discussed the model selection we looked at the code and he found that we incorrectly calculate the g2 values. In the g2 calculation we do -2 * log(Lik) + 2 * n_Parameter * log(observations) which is a mixture between BIC (-2 * log(Lik) + n_Parameter * log(observations)) and AIC (-2 * log(Lik) + 2 * n_Parameter). So I subtracted (n_Parameter * log(observations)) to get BICs and it got a little better:

Run1:

	Goal	NoGoal
Random	79	78
RL	21	54
SampEx_Heur_Goal	7	13
SampEx_Heur_NoGoal	10	7
SampEx_Int_Goal	72	69

Run 1 probabilities

	Goal	NoGoal
Random	0.418	0.353
RL	0.111	0.244
SampEx_Heur_Goal	0.037	0.059
SampEx_Heur_NoGoal	0.053	0.032
SampEx_Int_Goal	0.381	0.312

Run 1 Separated for variance conditions

Low:

	Goal	NoGoal
Random	0.424	0.246
RL	0.136	0.275
SampEx_Heur_Goal	0.030	0.058
SampEx_Heur_NoGoal	0.076	0.029
SampEx_Int_Goal	0.333	0.391

High

	Goal	NoGoal
Random	0.492	0.403
RL	0.098	0.234
SampEx_Heur_Goal	0.033	0.078
SampEx_Heur_NoGoal	0.016	0.039
SampEx_Int_Goal	0.361	0.247

Equal

	Goal	NoGoal
Random	0.339	0.400
RL	0.097	0.227
SampEx_Heur_Goal	0.048	0.040
SampEx_Heur_NoGoal	0.065	0.028
SampEx_Int_Goal	0.452	0.307

Run 2:

	Goal	NoGoal
Random	79	78
RL	21	51
SampEx_Heur_Goal	7	13
SampEx_Heur_NoGoal	10	10
SampEx_Int_Goal	72	69

Run 3:

	Goal	NoGoal
Random	79	78
RL	20	50
SampEx_Heur_Goal	7	13
SampEx_Heur_NoGoal	11	11
SampEx_Int_Goal	72	69

[ndphillips]

yes you are absolutely correct! Thanks for catching this. I've also gone ahead and changed the name g2 to bic throughout the code.

[ndphillips]

I just had a thought. As I have the feeling from my own simulations that the memory_N parameter doesn't improve model fits very much, what would happen if you fixed memory_N to 25 to all participants? This would reduce the number of parameters in the SampEx models by 1, and might improve their performance.

Let me know if you can re-run the model fits on the server using this adjustment

[mdsteiner]

Sure I can do that... It would however a bit of the heuristic notion from the model, I think. But yes of course if it doesn't help much then it's not appropriate to use an additional parameter. I'll let you know once I have the results...

[mdsteiner]

Ooops I only just noticed that we actually fitted two parameters in the SampEx_Int_Goal, so I didn't correct the BICs of the initial fits enough, so here's the new table:

Run1:

	Goal	NoGoal
Random	62	69
RL	19	54
SampEx_Heur_Goal	7	13
SampEx_Heur_NoGoal	10	7
SampEx_Int_Goal	91	78

Run 1 probabilities

	Goal	NoGoal
Random	0.328	0.312
RL	0.101	0.244
SampEx_Heur_Goal	0.037	0.059
SampEx_Heur_NoGoal	0.053	0.032
SampEx_Int_Goal	0.481	0.353

Run 1 Separated for variance conditions

Low:

	Goal	NoGoal
Random	0.348	0.188
RL	0.136	0.275
SampEx_Heur_Goal	0.030	0.058
SampEx_Heur_NoGoal	0.076	0.029
SampEx_Int_Goal	0.409	0.449

High

	Goal	NoGoal
Random	0.410	0.377
RL	0.082	0.234
SampEx_Heur_Goal	0.033	0.078
SampEx_Heur_NoGoal	0.016	0.039
SampEx_Int_Goal	0.459	0.273

Equal

	Goal	NoGoal
Random	0.226	0.360
RL	0.081	0.227
SampEx_Heur_Goal	0.048	0.040
SampEx_Heur_NoGoal	0.065	0.027
SampEx_Int_Goal	0.581	0.347

The nice thing is, that now, in the High and Equal conditions we actually see a relatively big difference between Goal and NoGoal in the SampEx_Int_Goal condition. One thing about the parameters: The mean of the best parameters were SampEx_Heur_Goal = 7.55, SampEx_Heur_NoGoal = 6.11 and SampEx_Int_Goal = 3.19. So the best values are rather low.

[ndphillips]

Got it. Ok so the Heuristic models seem to really suck. In a way that's good as they are models we didn't pre-register. However, it's still weird that so many people in the nogoal condition are being classified as SampEx_Int_Goal.

I wonder if SampEx_Int_Goal is also capturing a simple "mean" model (Hertwig called this the 'natural mean heuristic'), where impressions are based solely on the sample means of options. Now, the RL model should also be able to approximate this, but it requires an additional updating parameter, which may cause it to lose out to SampEx_Int_Goal.

I would propose the following

0 Parameter Model

1- Random

1 Parameter models

2- Mean (Choice Sensitivity)

2 Parameter models

3- RL (Updating Rate, Choice Sensitivity)
4- SampEx_Int_Goal (Memory_N, Choice Sensitivity)

I predict that a lot of people in the NoGoal condition would fall out of the SampEx_Int_Goal classification

[mdsteiner]

So just to clarify, we get rid of the SampEx_heur_Goal model and see how participants divide among the others (for the mean model I'd just use the SampEx_heur_NoGoal model and fix Memory_N to 25)?

I think I'm done with the model recovery (I'm currently running it with 10 subjects to see whether the code works and if it does will put it on the server).

Do you also have a server account? then we could run the simulation fitting and the fitting with the new models simultaneously. If not I'll do it sequentially (or just give 8 cores to each session), should still run through until next week...

[ndphillips]

Yep that sounds good, but I think it would be clearer coding wise to create a new model called "NaturalMean" that explicitly only returns the sample mean. That way, we'll never get confused.

I had a server account a while ago but I have no idea if it still works and I don't really want to deal with it. I'll leave the fitting to you :)

[mdsteiner]

ok, yes you are right, I'll add a mean model... Ok, sure, I'll put it up then once I'm done with the coding.

[mdsteiner]

New Model fitting results are here (only real data, Parameter/ Model Recovery is not yet through):

Overall

	Goal	NoGoal
NaturalMean	0.143	0.217
Random	0.169	0.140
RL	0.360	0.421
RLGoal	0.026	0.050
SampEx_Int_Goal	0.302	0.172

Low Environment

	Goal	NoGoal
NaturalMean	0.182	0.261
Random	0.167	0.072
RL	0.455	0.420
RLGoal	0.030	0.014
SampEx_Int_Goal	0.167	0.232

High Environment

	Goal	NoGoal
NaturalMean	0.115	0.195
Random	0.180	0.208
RL	0.328	0.364
RLGoal	0.016	0.104
SampEx_Int_Goal	0.361	0.130

Equal Environment

	Goal	NoGoal
NaturalMean	0.129	0.200
Random	0.161	0.133
RL	0.290	0.480
RLGoal	0.032	0.027
SampEx_Int_Goal	0.387	0.160

I think this looks quite good. Not only do we have fewer participants that are classified as random but also the results of two environments (High and Equal) suggest, that the model that accounts for the goal is better in classifying people from the Goal than from the NoGoal condition. I thinks this, together with the plots with pRisky on Y and Points Earned on X axis and the new regression you suggested are nice results to make our point that a goal affects how people make decisions. I would suggest, that we fully focus on these analyses (of course we can still use the good stuff from what we've had so far) and report all the rest in a supplement. The good thing is that we wouldn't really have problems with the preregistration since what we've done additionally is in line with the content of the preregistration, just not the exact code we've used there (and importantly we didn't change the code that does the exclusions and data preparation).

[ndphillips]

Great stuff! I agree with your suggestions.

I would suggest two changes to simply things and make the results clearer:

1. Remove the RLGoal model, clearly it bombed. Reclassify these (few) participants to the next best strategy

2. Put the "Random" classification percentages at the top of the table. Then, the order should be: "random", "NaturalMean", "RL", "SampEx_Int_Goal"

[mdsteiner]

Here we go:

Overall

	Goal	NoGoal
Random	0.180	0.154
NaturalMean	0.148	0.244
RL	0.360	0.421
SampEx_Int_Goal	0.312	0.181

Low Environment

	Goal	NoGoal
Random	0.182	0.072
NaturalMean	0.197	0.275
RL	0.455	0.420
SampEx_Int_Goal	0.167	0.232

High Environment

	Goal	NoGoal
Random	0.197	0.234
NaturalMean	0.115	0.247
RL	0.328	0.364
SampEx_Int_Goal	0.361	0.156

Equal Environment

	Goal	NoGoal
Random	0.161	0.147
NaturalMean	0.129	0.213
RL	0.290	0.480
SampEx_Int_Goal	0.419	0.160

[ndphillips]

great! It is a bit 'unfortunate' that the modal strategy for the goal condition across conditions is RL and not SampEx_Int_Goal. But hey that's how it goes.

Can you also calculate summary statistics for MLE parameter estimates for those people classified by each model?

Also in reporting the classification results in the paper, the table should include both the absolute frequencies, and the proportions. Just something for us to remember

[mdsteiner]

model_best	bic_mean	bic_sd	par_Imp_mean	par_Imp_sd	par_Choice_mean	par_Choice_sd	N
NaturalMean	284.6419	57.11349	NA	NA	0.1021640	0.0743124	82
Random	346.5740	0.00000	NA	NA	NA	NA	68
RL	271.3271	60.62504	0.7193733	0.2604189	0.0953607	0.1166225	161
SampEx_Int_Goal	307.5428	30.28930	2.8787879	3.6902318	1.7736558	0.7426561	99