http://vis.stanford.edu/files/2013-SemanticColor-EuroVis.pdf
I found this paper on matching semantics with colors. We might, for instance, associate blue with sky or money with green, and the question of how to algorithmically match these to better generate content specific visualization designs is the focus of this paper. To validate their algorithmic approach, the researchers had participants look at different visualizations and have their response time recorded. One motivational observation from previous studies is that people take longer to name words when they are colored in a conflicting way. For instance, if fire were to be colored blue or grass as red (note as opposites).
They had two experiments. The general goal of both of them was to take a set of colors (20 of the tableau preset), and figure out how to best categorically assign them to a set of labels based on people's response times and accuraces of answering a set of questions about a colored bar plot. They first identified a set of related images to the semantic labels, then applied a series of filtering algorithms in order to attain histograms of each color's relative appearence in corresponding images. These color frequencies were then applied towards their experiment.
The goal of their first experiment was to validate whether colors matching the semantics actually mattered. They found that colors chosen by data viz experts + algorithm improved participant responses. They also found the experts did a better job than the algorithms at choosing semantic-resonant colors for concrete categories. They found that question type also influenced how much the color improved, suggesting that time spent thinking about the answer was more important than how resonant the colors were. Interestingly, the algorithm performed worse than the expert on concrete (fruit/vege) labels but about the same as expert on iconic (drink/company). This could be because the relative variance in color along the images they scraped was lower for a company logo.
I think this is a very interesting focus of research. I would be interested in learning more about what the tradeoffs might be of choosing one color over another to represent a label. For instance, besides response time, can we more accutately intuit a data type of it has one color versus another? If a secondary color is used, is this any better than using an unrelated color or worse (for instance brown or green for banana vs blue)? More broadly, I would also be interested in seeing whether or not certain variable types affect other areas of perception. For instance, if the label is apple vs house, how might we anticipate the relative discrepincies in quantity.