1. Disguised reality
In this intervention, I will try to introduce a reflection on the status that images have acquired within the complex system of contemporary media, often referred to as digital capitalism, data capitalism or platform society.[1] My speech, therefore, moves between media studies and visual culture studies – though, in the final part, I will propose a dialogue between these disciplines and the political economy of material and symbolic resources.
To conduct this reflection, I will use an example that seems to me profoundly revealing: the video filters for face distortion or manipulation effects that are spreading with impressive speed on various social platforms and particularly on Instagram. So, let me first introduce this case.
In August 2019, an update to Instagram (a social media enterprise owned and controlled by Facebook) permitted users to submit their own filters to the app’s Effects Gallery. Some of these (such as the trendy one FixMe), when applied to photos, would mimic the effects of facelifts, Botox injections, and other surgeries. In October 2019, Instagram banned the publication of photographs (especially selfies) that contained distortion effects on the subject’s face. The decision was put forward by the concern that these images could contribute to the spread of unrealistic beauty standards, linked to negative body image among users – in particular after some claims that images depicting self-harm and depression on the site had contributed to a 14-year-old British teenager’s suicide.
However, the decision was short-lived. On August 6, 2020, the platform decided to reintroduce face distortion effects; as stated in a post on Facebook by Spark AR creators, the community gathering the Instagram creators of effects, “We want AR effects to be a positive and safe experience for our community while allowing creators to express their artistic perspectives. We recognize that creators predominantly use face alteration and feature augmentation to create artistic, surreal, fantasy effects that many enjoy and that these effects are widely available across other platforms”.[2]
After all, we must recognize that the ban was an unrealistic decision: the business of face manipulation effects is widespread and growing. They are used by more than 1 billion people a month on Instagram alone. Other social media such as TikTok and Snapchat offer similar effects. The above-mentioned Spark AR creators community counts more than 600,000 members from 190 countries, for a total of more than 2 million effects available for users. AR effects are presently expanding on other messaging platforms of the Facebook galaxy, such as Messenger, Instagram and Portal.[3] Above all, they involve a considerable flow of money: AR filters are partly free (and as such contribute to fuel the flow on platforms), partly for sale, and partly (increasingly) provided in forms of branded content by fashion, cosmetic and plastic surgery companies (“try the product before purchasing it!”).
Of course, some of them are just funny and amusing; yet, many others (and I would say most of them) aim to “beautify” the bodies and especially the faces of the portrayed subjects: “Today, … more and more young people – and especially teenage girls – are using filters that ‘beautify’ their appearance and promise to deliver modelesque looks by sharpening, shrinking, enhancing, and recolouring their faces and bodies”.[4]
How to define this kind of image? The effect creators and many commentators speak of a new frontier of augmented or expanded reality – and declare that their mission is that of redefining reality.[5] This terminology is not entirely precise: these effects technically correspond to what is defined as mixed reality: in this case, indeed, a digitized moving image is automatically combined with elements and determinations artificially produced by the machine to produce a hybrid entity in which the visual data “extracted” by reality are intimately blended with “artificial” ones. In this respect, we can take a step forward. If we look at reality, we realize that it is neither augmented, nor expanded or even redefined, but more properly rigged: consequently, we should more correctly define these effects as devices producing disguised reality.
As I mentioned at the beginning, I intend to use the example of disguised reality effects to introduce a reflection on the status that images have acquired within the platform society and digital capitalism. I will try to grasp this status by identifying three fundamental characteristics of contemporary images: I will consider them in turn as technological objects, as practical objects and as economic-political objects. We will see that in each of the three cases, the example of the disguised reality effects will prove to be a good ground for analysis and exemplification of my assumptions.
2. Images as technological objects
In the 80s, the Czech philosopher Vilém Flusser defined “modern” pictures, provided by film and television, as technical images.[6] Flusser also heralded the advent of images more directly linked to the management of information; yet, what subsequently happened exceeded Flusser’s predictions. Today, images do not simply arise from the “digitization” of previous photographic or electronic images; instead, they entirely derive from data management processes, according to a scheme that includes three major steps.
(1) The first step is the extraction/capture/ingestion of data which can take place either through the acquisition of patterns of photons (intended as components of the electromagnetic band, not necessarily visible to the human eye) by employing sensors, or through the acquisition of information through the use of interfaces.
(2) The second step consists of cleaning and sorting the data in data sets; starting from here, the data can be manipulated (as when we modify the parameters of a selfie), combined (as in the case of disguised reality), or subjected to extraction processes (as when our face is used to enrich a database for biometric recognition).
(3) This set of processes, often applied jointly, gives rise to more complex data-cubes, which are at the core of the third step. Indeed, they can be translated into concrete manifestations employing devices generically called “actuators”; these can be of two kinds. The first are practical ones, such as when a mechanical arm retrieves a piece that the sensors and recognition algorithms have identified as defective on the assembly line. The second are sensory actuators: they correspond to the various types of screens (from the huge ones in big city squares, to the microscopic ones used in smart glasses and headsets for augmented or virtual reality) translating the data cubes into light and sound patterns. It is only at this point that we can speak of the constitution of an image.
Based on this framework, it is immediately evident that contemporary digital images have a different status from photographic or electronic technical ones, for two reasons. First, they no longer constitute the “immediate” and partially automated appearance of a distant portion of the world; rather, images should be presently considered as the interactive and dynamic display of a nearby portion of a data cube. Secondly, they do not constitute the “digitization” of previously existing photographic or electronic images, since the data cube derives only minimally from photographic traces (i.e. from the translation into data of photon patterns): they are mostly “original” products deriving from the interaction between data gathered from different sources. Within the data cubes, photon patterns extracted from the “real” visible world cannot claim any status of superiority. Therefore, this new type of images is not technical but technological; scholars often refer to them as “algorithmic images”, but since the computational process component is prevalent in their constitution, I think it is better to talk about visual algorithms.[7]
The images resulting from the application of disguised reality effects are a perfect example of visual algorithms. Indeed, they arise from the combination of moving photographic traces and adjustments introduced “live” by computer vision, facial recognition and Artificial Intelligence. From this perspective, we must be careful not to confuse them with the “old” Photoshop effects. This new generation of filters uses sophisticated biometric tools to apply the effects to the face in motion in completely realistic videos showing subjects talking, moving and assuming different expressions.[8] Their way of acting is somewhat similar to the so-called deep fakes, which indeed use the same algorithms.[9]
3. Images as practical objects
This new way of thinking about images as visual algorithms, that is, technological objects intimately linked to data management, leads us to the second salient feature of contemporary images: we must consider them as practical objects. Visual algorithms are not intended for contemplation but for action, and are themselves equipped (thanks to the devices that allow and discipline their uses) with a specific agency. Let me explain this point with some practical examples, still linked to my introductory case.
The web is full of facial analysis services. For example, the Qoves studio platform (based in Australia) provides a free “facial aesthetics consultancy” service (https://www.qoves.com/): an AI analyzes your face starting from your photograph and predicts its aesthetical flaws and their probability of manifestation. The platform suggests appropriate cosmetic surgery and cosmetic products that help prevent or fix these failures. Similarly, the Face++ platform (https://www.faceplusplus.com/) offers a series of free services related to facial recognition and manipulation: for example, the Beauty scoring system, which like Qoves uses AI to examine your face. But instead of detailing what it sees in clinical language, it boils down its findings into a percentage grade of likely attractiveness. In fact, it returns two results: one score that predicts how men might respond to a picture, and the other that represents a female perspective.[10]
These practices can be considered frivolous; however, they are based on very complex and advanced technologies that sometimes derive from and lead to less innocent uses. First, there are economic implications: the analyzes of one’s face lead to hyper-targeted and tailor-made advertising proposals; moreover, the images of faces are a rather valuable commodity in the data market, since they are used to train artificial intelligences for facial recognition and for the generation of synthetic faces to be used for false profiles on social networks.
Moreover, algorithms of facial recognition and evaluation hide a series of social entanglements. For example, professional moderators of social platforms are beginning to apply beauty scoring algorithms to ban some faces marked as ugly and unpleasant:
The makers of TikTok, the Chinese video-sharing app with hundreds of millions of users around the world, instructed moderators to suppress posts created by users deemed too ugly, poor, or disabled for the platform, according to internal documents obtained by The Intercept… TikTok moderators were told to suppress users with “abnormal body shape”, “ugly facial looks”, “too many wrinkles”, or in “slums, rural fields” and “dilapidated housing”.[11]
Not only: these types of images are also involved in political dynamics. The Face++ algorithmic engine is implemented by one of the giants of the biometric recognition sector, the Chinese company Megvii, the largest third-party authentication software provider globally. In May 2019, the NGO Human Rights Watch reported that parts of Face++ code was used by the Chinese Government to collect data on and track the Uighur community in Xinjiang. Even though the involvement of Megvii in the Uighur case is still controversial (in June 2019, Human Rights Watch released a correction to its report stating that Megvii did not appear to have collaborated on IJOP), nonetheless the involvement of Megvii in the Chinese government’s surveillance network is a proven fact.
These episodes allow us to grasp how, behind the light and frivolous uses of visual algorithms, more serious, practical and sometimes disturbing uses emerge. In this sense, the boundary between art and media on the one side and practical and extra medial uses of images on the other one, becomes very blurred – and, in many cases, it turns out to be only a type of window dressing. For this reason I talk of a “postmedia” condition with regard to the use of contemporary image production and management devices. To sum up, as technological objects, visual algorithms are practical and operational tools, equipped with specific forms of agency: they operate in the real world with very concrete and immediate effects.
4. Images as economic-political objects
A third conceptual key that can help us understand the role of images in the contemporary context is considering them as economic-political objects. Obviously, images are part of a political economy of the media in the traditional sense, as they feed a global market of massive financial dimensions. However, I intend to propose a vision of a political economy that is not linked in the first instance to the market and finance, and instead possesses a broader scope: I, therefore, consider the economy any regulated management of extraction, production, circulation, exchange, accumulation, deprivation and disposal of resources; since these resources are continually shared or shareable, and since they are managed within a common space, economics is always a political economy.[12]
This interpretation allows us to understand that visual algorithms derive (also in historical terms) from the connection and conjunction of three types of resources and that consequently, they cross and link three kinds of economies: that of images as material objects, that is (moving) pictures; that of the light necessary to produce and in some cases transmit the images; and that of information and data. Each of these kinds of resources is measured by a minimal unit: respectively, the pixel, the photon, and the bit. Furthermore, the joint management of these three types of resources through visual algorithms allows the administration of a large number of other resources: of immaterial type (for example, the reputational resources linked to the beauty of the face; or the attentional ones linked to time and concentration spent on the use of devices); of agentive type (the possibilities of action or their foreclosure related to facial recognition in various situations and in particular in surveillance regimes); and of material type (the sale of beauty products or plastic surgery operations related to beauty assessment; the expense for the purchase of particular effects of disguised reality; the accumulation of datified faces and the refinement of recognition algorithms to be resold on the big data market).
5. Conclusions
In conclusion, if images circulate so widely within contemporary society, it is because they are no longer just “pictures” – that is, objects to be observed or contemplated from a distance. Contemporary technological images, which I have called visual algorithms, are practical devices capable of acting in and on the world, regulating the flows of and exchanges between different and multiple types of resources.
Therefore, we will have to study images more and more as tools of power that play an actual and fundamental role in distributing, redistributing, and accumulating common resources. It seems that a meeting between visual studies and political-economic ones can no longer be postponed.
END NOTES
[1] Dan Schiller, Digital Capitalism, The MIT Press, Cambridge (Mass.), London 2000; José van Dijck, Thomas Poell, Martijn de Waal, The Platform Society. Public Values in a Connective World, Oxford University Press, Oxford – New York, 2018; Nick Couldry, Ulises A. Mejias, The Costs of Connection. How Data Is Colonizing Human Life and Appropriating It for Capitalism, Stanford University Press, Stanford (Cal.) 2019. For a quick introduction to visual studies see Alexis L. Boylan, Visual Culture, The MIT Press, Cambridge (Mass.), London 2020.
[2] Spark AR Community, “Policy Update: Face-Altering Effects on Instagram”, August 6, 2020, https://www.facebook.com/groups/SparkARcommunity/permalink/950326835379359
[3] Amanda Silberling, “Facebook’s Spark AR platform expands to video calling with Multipeer API”, Tech Crunch, June 2, 2021, https://techcrunch.com/2021/06/02/facebooks-spark-ar-platform-expands-to-video-calling-with-multipeer-api/
[4] Tate Ryan-Mosley, “Beauty filters are changing the way young girls see themselves”, MIT Technology Review, April 2, 2020, https://www.technologyreview.com/2021/04/02/1021635/beauty-filters-young-girls-augmented-reality-social-media/. See also, among others interventions, Anna Haines “From ‘Instagram Face’ To ‘Snapchat Dysmorphia’: How Beauty Filters Are Changing The Way We See Ourselves”, in Forbes, Apr 27, 2021, https://www.forbes.com/sites/annahaines/2021/04/27/from-instagram-face-to-snapchat-dysmorphia-how-beauty-filters-are-changing-the-way-we-see-ourselves/; Maghan McDowell, “The ethics and future of flattering AR filters”, Vogue Business, 2 March 2021, https://www.voguebusiness.com/technology/the-ethics-and-future-of-flattering-ar-filters.
[5] Spark AR Team, “Women Redefining Reality Ten creators share their insights and advice for women exploring AR”, 4 December 2020, https://sparkar.facebook.com/blog/women-redefining-reality/
[6] Vilém Flusser Into the Universe of Technical Images (1985), Introduction by Mark Poster, Translated by Nancy Ann Roth, University of Minnesota Press, Minneapolis – London 2011.
[7] The literature of visual studies on digital imaging is boundless. See only Jacques Khalip, Robert Mitchell (eds.), Releasing the Image. From Literature to New Media, Stanford University Press, Stanford (Calif.) 2011; Liv Hausken (ed.), Thinking Media Aesthetics. Media Studies, Film Studies and the Arts, Peter Lang, Frankfurt am Main 2013; Steve F. Anderson, Technologies of Vision, The War Between Data and Images, The MIT Press, Cambridge (Mass.) – London 2017; Krešimir Purgar, Pictorial Appearing. Image Theory After Representation, Transcript Verlag, Bielefeld 2019; Johanna Drucker, Visualization and Interpretation. Humanistic Approaches to Display, The MIT Press, Cambridge (Mass.) – London 2020.
[8] “Beauty filters are essentially automated photo editing tools that use artificial intelligence and computer vision to detect facial features and change them. They use computer vision to interpret the things the camera sees, and tweak them according to rules set by the filters’ creator. A computer detects a face and then overlays an invisible facial template consisting of dozens of dots, creating a sort of topographic mesh. Once that has been built, a universe of fantastical graphics can be attached to the mesh. The result can be anything from changing eye colours to planting devil horns on a person’s head”. Tate Ryan-Mosley, “Beauty filters are changing the way young girls see themselves”.
[9] Samuel Woolley, The Reality Game. How the Next Wave of Technology Will Break the Truth and What We Can Do About It, Public Affairs, London 2020.
[10] Tate Ryan-Mosley, “I asked an AI to tell me how beautiful I am”, MIT Technology Review, March 5, 2021 https://www.technologyreview.com/2021/03/05/1020133/ai-algorithm-rate-beauty-score-attractive-face/
[11] Sam Biddle, Paulo Victor Ribeiro, Tatiana Dias, “Invisible censorship”, The Intercept, March 16 2020, https://theintercept.com/2020/03/16/tiktok-app-moderators-users-discrimination/
[12] Ruggero Eugeni, Capitale algoritmico. Cinque dispositivi postmediali (più uno) [Algorithmic capital. Five post-media devices (plus one)], Morcelliana Scholè, Brescia 2021.