Lessons from the dress: The fundamental ambiguity of visual perception

The brain lives in a bony shell. The completely light-tight nature of the skull renders this home a place of complete darkness. So the brain relies on the eyes to supply an image of the outside world, but there are many processing steps between the translation of light energy into electrical impulses that happens in the eye and the neural activity that corresponds to a conscious percept of the outside world. In other words, the brain is playing a game of telephone and – contrary to popular belief – our perception corresponds to the brains best guess of what is going on in the outside world, not necessarily to the way things actually are. This has been recognized for at least 150 years, since the time of Hermann von Helmholtz. As there are many parts of the brain that contribute to any given perception, it should not be surprising that different people can reconstruct the outside world in different ways. This is true for many perceptual qualities, including form

Rubin’s vase: A classical example of figure/ground segmentation. The image is fundamentally ambiguous. People perceive a vase or faces, but not both at the same time.

and motion. While this guessing game is going on all the time, it is possible to generate impoverished stimulus displays that are consistent with different mutually exclusive interpretations, so in practice the brain will not commit to one interpretation, but switch back and forth. These are known as ambiguous or bistable stimuli, and they illustrate the point that the brain is ultimately only guessing when perceiving the world. It usually just has more information to go by and disambiguate the interpretation.

A bistable motion stimulus. Do you see the dots moving from left to right or up and down?

This is also true for color vision. The fundamental challenge in the perception of color is to identify an object despite changing illumination conditions. The mixture of wavelengths that reaches our eye will be interpreted by the brain as color, but which part is due to the reflectance of the object and which part is due to the illumination?

This is a inherently ambiguous situation, so the brain has to make a decision whether to take the appearance of an object at face value or whether it should try to discount the part of the information that stems from the illumination. As the organism is not primarily interested in the correct representation of hues, but rather the identification of objects in light of dramatically varying conditions (e.g. a predominance of long wavelengths in the early morning and late afternoon vs. more short wavelengths at noon), it is commonly accepted that the brain strives for “color constancy” and is doing a pretty good job at that.   But in this tradeoff towards discounting, something has to give, and that is that we are bad at estimating the apparent absolute hue of objects. For instance, a white surface illuminated by red light will objectively look reddish. The same white surface illuminated by blue light will objectively look blueish. In order to recognize both as the same white surface, the subjective percept needs to discount the color of the light source.

So it should not be surprising that inference of hue can be dramatically influenced by context. The same shade of grey can look almost black on a bright background but almost white on a dark one.

Lightness illusions are common. The shade of grey above and below the horizon are the same. You can see this by covering the strip in the middle

Note that this is not a bug. It is a necessary tradeoff in the quest to achieve a stable appearance of the same object, regardless of context or illumination.

So far, so good. Now where does the dress come in? The latest sensation to sweep social media has sharply divided observers. Some see the dress as gold on white, others as black on blue.

White/gold vs. Black/blue: Some people perceive the image on the left, others the one on the right. Others switch back and forth.

As noted before, this kind of divergence of interpretation might be rather common with complex stimuli. The importance of the “dress” stimulus is the extent to which intersubjective interpretation differs in the color domain. To my knowledge, this is by far the most extreme such stimulus. Of course one has to allow for the fact that not everyone’s monitor will be calibrated in the same way and viewing angles might differ, but this doesn’t account for the different subjective experience of people viewing the exact same image on the same monitor from the same position. And of course the reason why the “true colors” of the dress are in dispute in the first place is the color constancy phenomenon we alluded to above. This was likely a black/blue dress that was photographed with poor white balance, giving it an ambiguous appearance. But that doesn’t change the fact that some people sincerely perceive it as white/gold.

That the interpretation of the color values itself depends on context can readily be seen if the context is taken away. In the image below, some stripes were extracted from the original image without altering it in any other way. The “white/blue” stripe can now be identified as light blue and the “gold/black” stripe as brown.

Stripes, out of context: One now unambiguously looks like blue, the other like brown.

But why the difference in interpretation? That is where things get interesting. If the ambiguity derives from color constancy (and it looks like it does), the most plausible explanation is that people differ in their interpretation of what the illumination source is. Those who interpret the dress as illuminated by a blue light will discount for this and see it as white/gold whereas those who interpret the illumination as reddish will tend to see it as black/blue. Interestingly, the image itself does allow for both interpretations, the illumination looks blueish in the top of the image, but yellowish/reddish in the bottom. On a more fundamental level, a blue/black dress illuminated by a white light source might be indistinguishable from a white/gold one, but with a blueish shadow falling onto it.

But if this is the case, one should be able to consciously override this interpretation once it is pointed out, but – for most people – this does not seem to be the case, in contrast to most other such ambiguous duckrabbit displays. People are able to willfully control what they see.

Inherently ambiguous stimuli. Interpretations switch, they don’t blend. But people can consciously override interpretations once they are pointed out.

This raises several intriguing possibilities. For instance, it has been recognized for quite a while that the human “retinal mosaic” – the distribution of short-, medium- and long-wavelength cones in humans is radically different between observers, but this seems to only have a minute impact on the actual perception of color. Perhaps it is the case that differences in retinal mosaic can account for difference in the perception of this kind of “dress” stimulus. Moreover, there is another type of context to consider and that is temporal context. We don’t just perceive visual stimuli naively, but we perceive them in the context of what we have encountered before – not all stimuli are equally likely. This is known as a “prior”. It is quite conceivable that some people (e.g. larks, owls) have a different prior as to what kind of illumination conditions they encounter more frequently. Or a complex interaction between the two.

While we must confess that we currently do not know why some people consistently see the dress one way, others consistently in another way and some switch, it is remarkable that the switching happens on very long timescales. Usually, switching is fast, for instance in the Rubin’s vase stimulus above. This could be particular to color vision. There are no shortcuts other than to do research as to the underlying reason that accounts for this striking difference in subjective perception.

Meanwhile, one lesson that we can take from all of this is that it is wise to assume a position of epistemic humility. Just because we see something in a certain way doesn’t mean that everyone else will see it in the same way. Moreover, it doesn’t mean that our percept necessarily corresponds to anything in the real world. A situation like this calls for the hedging of one’s bets, and that means to keep an open mind.  Something to remember next time you disagree with someone.

Who is right? Does the question even make sense? Or is it important to see the bigger picture?