The visual sources of Sensory History
Seeing, hearing, smelling, tasting and feeling often have a visual component. Therefore the iconography of representations where the five senses play a part, however implicitly, also often has a sensory component. Participating in a two-day workshop about the olfactory aspect of historical sources confirmed the idea that our visual heritage is extremely rich in sensory information and connotations. That information can be made quite explicit in pictures, but more often it is implied and can only be exposed through the interpretation of complex visual data. Making that information available for historical research is therefore not an easy task.
Iconclass, the classification that is most widely used for cultural content, does not seem to do justice to the richess of sensory connotations. Olfactory or auditory aspects of biblical, mythological and historical stories go unmentioned or remain more or less implicit. In many cases they are not easily retrievable. It is easy, for instance, to tag representations of paradise with Iconclass concepts, but in its present state the Iconclass system does not provide olfactory information about its “lost smell”. Neither does it refer to the sense of touch that plays a part in the depiction of the institution of marriage, another pre-Expulsion theme. This variant of the iconography, with Adam and Eve holding - and touching - each other’s hand, has an undeniable tactile aspect.
In a similar way, Iconclass is often silent when sound is implied by a picture. Here is a random example, showing soldiers shooting at a pack of dogs attacking them. A picture of soldiers using firearms will not be retrieved by the implied auditory aspect of its iconography. Nor will it be retrieved by its olfactory aspect, for example by a reference to the smell of gunpowder, a result of the firing of rifles.
Thinking about ways to improve the performance of Iconclass when it comes to the sensory aspect of visual information, my first instinct was to focus on the tools the Iconclass system itself has on board. Those can be broadly subdivided into the words of the concept definitions, the keywords of the entry vocabulary, the cross references between concepts, and the references between related keywords.
In its present state, for example, the smell of the venison prepared for Isaac, the feeling of the goatskin Jacob has put on his arms, nor the blindness of Isaac, are made explicit in the Iconclass concept definition nor in the keywords with which you can find the concept.
If we want to alert researchers of sensory history to those aspects, we must also tag it with concepts located in a different branch of the Iconclass tree. The senses involved here – sight, smell and feeling – are found in the Iconclass category of the “five senses” 31A3 the (five) senses and they can be combined with the concept actually identifying the story of Jacob’s deceit
In this specific case it will be easy to change the definition to incorporate the sensory information, or add keywords like “smell” or “touch” to retrieve the concept.
However, surveying the content of the Iconclass schedules from this perspective, it dawned on me that there is a real risk that we shall overshoot our mark when we try to solve the problem with the internal instruments of Iconclass.
I had been looking for ways to enrich the Iconclass toolkit to improve its performance for olfactory information with the help of the information gathered by the Odeuropa team in an Excel sheet. As the list of concepts with potential sensory information grew and the web of cross references increased in density I realized that this approach could create so many references to the sensory aspect of our visual heritage, that to many users it would lose its meaning.
Iconclass contains some 40,000 concepts, but the combinatory explosion unleashed when we include the keys (secondary hierarchies) in the equation, will boost the tally up to some 1,5 million concepts.
So, even if only a small percentage of Iconclass concepts has a sensory connotation – and I have a feeling that the percentage will actually be quite significant – the number of hits that would be produced for words like smell, sound, touch, taste or sight would be massive. Massive, that is, if by adapting the Iconclass system itself searching with those words would always produce hits.
To grasp the consequences it might help to have a look at how Iconclass deals with the iconography of saints. Here is an example:
You do not have to be a specialist in Christian iconography to realize that not every depiction of saint Augustine will show the attributes mentioned in the concept definition. Still, if a user searches for “flaming heart” - included in the definition and in the keywords (in italics) - the Iconclass system will always respond with a reference to saint Augustine. To alert the user to the fact that this reference can be – and often will be –a false positive, we decided to add the word “possible” to the concept definition. The original printed format of Iconclass, of which this is a snippet, used italics for attributes. and relied on the assumption that a reader would know that not all representations of saint Augustine would always show all attributes. In a computerized environment that is simply too implicit.
An Iconclass-wide adaptation of definitions and keywords with potential sensory content along these lines will therefore in all likelihood indeed overshoot its target.
In addition, the small army of cataloguers that have built the corpus of Iconclass-tagged images will not have taken into account that the images they were cataloguing may have sensory connotations. Those connotations will in most cases not have been perceived as relevant. So, to enrich the concepts they have used for the iconography of their collections with explicit references to sensory content would have an enormous impact on the information they thought they were providing when they selected the Iconclass concepts. We would be changing the rules midway through the game…
The adaptation of the vocabulary, in other words, must take into account that the vocabulary has been used for over fifty years and millions of images have been tagged with it.
So, our task is twofold:
- To make it possible to alert researchers of Sensory History to aspects of iconography relevant to their research, taking into account that this may cover an extremely wide range of historical sources
- To avoid creating so many references to sensory experiences that the information will be perceived as self-evident and trivial; from “smell” to “smoking tobacco” … duh…
The solution I am inviting you now to consider is a simple one, seen from the perspective of the basic Iconclass toolset. However, it is also dependent on the way Iconclass information is offered in local implementations of the system. This is a much more general issue: while all institutions using Iconclass information for their iconographic information by definition use a common vocabulary, the experience they offer to their end users can be wildly divergent, up to the point that it obscures the fact that they are all using the same vocabulary …
Keep that in mind when evaluating the following proposal.
Combining concepts
Even though in very specific cases, like the blessing of Jacob, it will be useful and easy to enrich the specific definition, the main focus of the adaptation of the schedules will be on category 31A3 the (five) senses.
In the appendix you will find the concepts now included in this category. It is easy to see that the category has not expanded to great detail. As a result there is ample room for new concepts. I have now added a handful of concepts – the ones in red – to give you an idea of where we could go with this. You will recognize the main menu of the Odeuropa tools site in the proposed new section for 31A333 olfactory concepts. I’ve provisionally copied that subdivision for sounds as well, to illustrate the principle for other senses as well.
Now, used in isolation these concepts do not contribute much. However, if they are combined in the tagging of a source with concepts from other parts of the schedules, they will have an effect.
An obvious example is provided by this Aesop illustration. In addition to the concepts identifying the scene as an Aesopian story, the picture could be tagged with the following concepts:
31A3332 smell sources
31A623 faeces, excrement
If so, a word like “smell” or “stench”, but also a phrase like “source of a smell” can be combined – at retrieval time – with “faeces”.
In a similar setup a picture such as this one of the Fall of Jericho could be tagged with
31A3332 sound sources
48C7352 horn, trumpet, cornet, trombone, tuba
Which would then lead – potentially - to searches like “sound” or “source of a sound” combined with “horn”.
Do note that because of the hierarchical construction of Iconclass, this would also mean that “sound” and “music” would then retrieve a picture of Jericho’s destruction.
I have deliberately used the word potentially because for this strategy to really benefit the end user of the information, two conditions must be met.
- The cataloguer/researcher who decides that the sensory aspect of the visual sources deserves to be documented, must tag the picture with a combination of concepts. So, yes, it does require an effort.
- The way searching with Iconclass is implemented must make it possible to retrieve records from a database by simultaneously searching for a combination of notations, or of words that belong to different definitions. (i.e. the Arkyves model…)
I have limited myself for now to the expansion of only two of the candidate concepts, i.e. 31A32 and 31A33. The actual expansion and phrasing (and translation!) would require input from specialists of Sensory History.
The proposed expansion is printed in red. The wording is copied from the online Odeuropa tools and adapted for sounds.
In blue and preceded by an arrow are suggested, hand-picked cross references. Again, just to give you an idea of what would be possible. The cross references at 31A31 sight, however, are already part of the online Iconclass system
Appendix
