Elsevier

Computers in Human Behavior
人类行为中的计算机

Volume 134, September 2022, 107305
第 134 卷,2022 年 9 月,107305
Computers in Human Behavior

Users’ emotional and behavioral responses to deepfake videos of K-pop idols
用户对 K-pop 偶像深度伪造视频的情感与行为反应

https://doi.org/10.1016/j.chb.2022.107305Get rights and content  获取权利和内容
Full text access  全文访问

Highlights  要点

  • Deepfake is a process and outcome of artificial intelligence (AI) to create hyper-realistic manipulated media content.
    深度伪造是人工智能(AI)创建超逼真操纵性媒体内容的过程和结果。
  • Significant attention is paid to deepfake pornographic content, which is a big threat to media figures and the public.
    深度伪造色情内容受到广泛关注,这对媒体人物和公众构成了重大威胁。
  • This study explains how online users feel about and react to sexual deepfakes, focusing explicitly on K-pop idols.
    本研究探讨了网络用户对性深伪视频的感受及反应,特别关注 K-pop 偶像。
  • The results indicated the positive effects of anger on the coping behaviors of problem solving and emotional support.
    结果表明,愤怒对问题解决和情感支持等应对行为具有积极影响。
  • There is the necessity of collaboration among government, industry, and education to address deepfakes of a sexual nature.
    政府、产业和教育部门之间有必要合作应对性深伪视频问题。

Abstract  摘要

Deepfake is a process and outcome of artificial intelligence (AI) to create hyper-realistic manipulated media content. Despite its benefits, great attention is paid to deepfake pornographic content, which is a tremendous threat to media figures and the public. This study thus explains how general online users feel about and react to sexual deepfakes, focusing explicitly on K-pop idols. We conducted an online survey with individuals who have experience watching any deepfake content, and we analyzed the model using the PLS-SEM method by SmartPLS 3.0. The results showed that previous perceptions about sexual harassment and K-pop idols, experiences viewing pornographic content, and gender were crucial predictors of viewers' emotions toward K-pop idols' deepfake porn videos. Media exposure and age, however, did not have significant effects on users' emotional responses. The results also indicated the substantial positive effects of anger on the coping behaviors of problem solving and emotional support. Guilt, on the other hand, was not associated with either behavioral response measured. This study represents the first attempt to empirically investigate individuals’ negative feelings toward and reactions to content generated by novel and maliciously exploited technology. It also highlights the necessity of collaboration among government, industry, and education to address deepfakes of a sexual nature. Retry    Reason

Keywords  关键词

Deepfake
Pornography
Artificial intelligence
Negative emotions
Coping behavior

深度伪造色情艺术人工智能负面情绪应对行为

1. Introduction  1. 引言

Recent advances in artificial intelligence (AI) technologies and deep learning algorithms have improved users’ ability to create, transform, and manipulate digital content (Chesney & Citron, 2019; Öhman, 2019, pp. 1–8). This has given rise to deepfakes, which are face-swap videos counterfeited using AI (Gosse & Burkell, 2020). Deepfakes are widely applied in the media and entertainment industry, making special effects, dubbing, and reproducing the past less complicated. For example, deepfake technology enabled Robert DeNiro to play a character younger than himself in The Irish Man, and it helped to create a digital version of deceased actor Peter Cushing in Rogue One: A Star Wars Story (Suciu, 2020).
近年来,人工智能(AI)技术和深度学习算法的进步提高了用户创建、转换和操纵数字内容的能力(Chesney & Citron, 2019; Öhman, 2019, 第 1-8 页)。这催生了深度伪造,即使用人工智能伪造的面部交换视频(Gosse & Burkell, 2020)。深度伪造在媒体和娱乐产业中得到广泛应用,使得特效制作、配音和重现历史变得更加简单。例如,深度伪造技术使罗伯特·德尼罗能够在《爱尔兰人》中扮演一个比他年轻的角色,并帮助在《星球大战:侠盗一号》中创造已故演员彼得·库辛的数字版本(Suciu, 2020)。
Despite the effectiveness of deepfake technology, it can have adverse effects on media figures and the public because of its ability to concoct facts and attack innocent individuals. For instance, individuals are abusing deepfake technology in the form of nonconsensual pornographic videos in which celebrity faces are synthesized on the body of porn stars. According to a report by deepfake research company Deeptrace Labs, almost 96% of all deepfake videos that they discovered online were deepfake pornographic videos (DPVs) (Ajder et al., 2019). Among them, 46% included women entertainers from the US or UK and 25% included female K-pop stars. Specifically, their results showed that DPVs of hundreds of K-pop girl group members are being fabricated only for distribution on pornographic websites, private webpages, and even SNSs (Ajder et al., 2019). The existence of chatrooms solely for sharing DPVs on Telegram was recently uncovered, with one of the most popular chatrooms having more than 500 posts of DPVs with over 2000 subscribers (Hwang, 2020).
尽管深度伪造技术效果显著,但由于其编造事实和攻击无辜个体的能力,它会对媒体人士和公众产生负面影响。例如,一些人正利用深度伪造技术制作非自愿色情视频,将名人面部合成到色情明星的身上。根据深度伪造研究公司 Deeptrace Labs 的报告,他们在线上发现的几乎所有深度伪造视频都是深度伪造色情视频(DPVs)(Ajder 等人,2019)。其中,46%包含来自美国或英国的女性演艺人员,25%包含女性 K-pop 明星。具体而言,他们的结果显示,数百名 K-pop 女子团体成员的 DPVs 正被专门制作并在色情网站、私人网页甚至社交网络(SNS)上传播(Ajder 等人,2019)。最近在 Telegram 上发现了一些专门用于分享 DPVs 的聊天室,其中最受欢迎的聊天室有超过 500 条 DPV 帖子,订阅者超过 2000 人(黄,2020)。
As DPVs that inherently harass, humiliate, and victimize female celebrities have become pervasive with increasing technological development, the public is more likely to be exposed to such deceptive media. Consequently, there has been widespread discussion among Koreans on the presidential e-petition website about the investigation and severe punishment of DPVs (Kang, 2021). Although the online petition case indicates general online users’ attitudes toward and behavioral reactions to the abuse of deepfake technology, there have been no empirical studies examining who is reacting to DPVs and how. Recent studies on deepfakes have concentrated mainly on explaining the concept of deepfake technology (Bates, 2018; Fletcher, 2018; Kietzmann et al., 2020), algorithmic approaches for detecting malicious deepfake content (Bappy et al., 2019; Korshunov & Marcel, 2018; Nguyen et al., 2019; Westerlund, 2019), or legal issues pertaining to deepfakes (Bae, 2019; Gieseke, 2020; Maras & Alexandrou, 2019). Some studies have focused on deepfake manipulation in the context of pornography (Bae, 2019; Delfino, 2019; Dickson, 2019; Öhman, 2019, pp. 1–8) or in the political and economic sectors (Chesney & Citron, 2019), but not enough relevant studies have investigated user perspectives.
随着深度伪造视频(DPVs)因其固有的骚扰、羞辱和陷害女性名人而随着技术发展日益普遍,公众更有可能接触到这种欺骗性媒体。因此,韩国人在总统电子请愿网站上就调查和严厉惩处 DPVs 展开了广泛讨论(Kang,2021)。尽管在线请愿案件表明了普通网络用户对深度伪造技术滥用的态度和行为反应,但还没有实证研究考察谁在反应 DPVs 以及如何反应。最近关于深度伪造的研究主要集中在解释深度伪造技术的概念(Bates,2018;Fletcher,2018;Kietzmann 等人,2020)、检测恶意深度伪造内容的算法方法(Bappy 等人,2019;Korshunov 和 Marcel,2018;Nguyen 等人,2019;Westerlund,2019)或与深度伪造相关的法律问题(Bae,2019;Gieseke,2020;Maras 和 Alexandrou,2019)。一些研究关注深度伪造在色情内容背景下的操纵(Bae,2019;Delfino,2019;Dickson,2019;Öhman,2019,第...页)。 1–8)或是在政治和经济领域(Chesney & Citron, 2019),但针对用户视角的相关研究还不足。
In this sense, exploring user responses to DPVs is crucial to address the gap in deepfake-related literature. Moreover, this investigation would be valuable in identifying individuals' primary attitudes toward deepfake as a new information technology (IT), which could help in explaining successful acceptance or resistance of emerging IT (Park et al., 2016). For example, if general online users believe that deepfake technology threatens to victimize themselves or others online, it may reduce their acceptance of the technology. The extent of users' influences on the current online ecosystem—as potential creators, distributors, or detectors of deepfake content—also highlights the importance of such a study. Additionally, understanding the reasons behind the creation of deepfakes and users' reactions to them is important for preventing abuse of this technology. Thus, this study explores users’ emotional and behavioral responses to DPVs of K-pop idols, one of the most prominent targets of DPVs globally. Ultimately, the results call for comprehensive actions from individuals, industry, and government to resolve the issue of DPVs.
从这个意义上说,探索用户对深度伪造视频(DPVs)的反应,对于填补深度伪造相关文献的空白至关重要。此外,这项调查在识别个人对深度伪造作为一种新技术(IT)的主要态度方面具有重要价值,这有助于解释新兴 IT 的成功接受或抵制(Park 等人,2016)。例如,如果普通网络用户认为深度伪造技术威胁到他们自己或他人在线上的安全,可能会降低他们对这项技术的接受度。用户对当前在线生态系统的影响程度——作为深度伪造内容的潜在创作者、传播者或检测者——也突出了这项研究的重要性。此外,了解深度伪造产生的原因以及用户对它们的反应,对于防止这项技术的滥用至关重要。因此,本研究探讨了用户对 K-pop 偶像深度伪造视频(DPVs)的情感和行为反应,而 K-pop 偶像是全球深度伪造(DPVs)最突出的目标之一。最终,研究结果呼吁个人、行业和政府采取全面行动来解决深度伪造(DPVs)的问题。
This study aims to investigate users' complex reactions to DPVs by measuring their coping behaviors while taking personal backgrounds and psychology into account. Particularly, the present study tries to examine the role of negative emotions (e.g., anger, guilt) as a bridge between user characteristics and DPV-related direct and indirect coping behaviors (Bechara et al., 1997). To test this model, we collected quantitative online survey data. Based on previous literature, we conceptualized sexual harassment attitudes, celebrity involvement, pornography and media exposure, and general demographics as personal perceptions that may affect emotions toward DPVs. Consequently, this study suggests that negative emotions significantly influence online users' direct or indirect coping behaviors with respect to DPVs. It makes theoretical and practical contributions as one of the first studies to examine online users’ responses to deepfake technology.

2. Theoretical background

2.1. Overview of deepfakes

“Deepfake” is a compound word of the terms “deep learning” and “fake,” originating from an anonymous user's post on the r/deepfake forum of social news aggregator Reddit. Definitions of deepfake vary among scholars. While some focus on the AI technology behind the deepfakes, others emphasize the hyper-realistic content produced by deepfake technology (see Table 1). In the context of this study, we examined mainly K-pop idols' DPVs to determine the impact of deepfakes as content on society, thus aligning with the latter conception. Therefore, we consider deepfakes to be hyper-realistic media content altered or manipulated with AI technology that is difficult to detect.

Table 1. Definitions of deepfakes.

Definitions
Emphasis on technology (software)Bates (2018)AI-based software that can superimpose someone's face onto an existing image or video.
Kietzmann et al. (2020)Techniques from machine learning and artificial intelligence to manipulate or generate visual and audio content with strong potential to deceive.
Nguyen et al. (2019)A technique that can superimpose facial images of a target person onto a video of a source person, generating a video of the target person doing or saying things that the source person did or said.
Emphasis on content (videos)Chesney and Citron (2019)Highly realistic and difficult-to-detect digital manipulations of audio or video.
Fletcher (2018)Videos are practically indistinguishable from authentic media.
Franks and Waldman (2018)Audio or visual material digitally manipulated to make it appear that a person is saying or doing something that they have not really said or done
Maras and Alexandrou (2019)Product of AI applications that merge, combine, replace, or superimpose images and videoclips to create fake videos that appear authentic.
Öhman (2019)Hyper-realistic videos in which a person's face has been analyzed by a deep learning algorithm and then superimposed onto the face of an actor in a video.
Strickland (2019)Videos that have been modified using a specific face-swapping technique but now a catch-all term for manipulated videos.
Wagner and Blewer (2019)Videos resulting from feeding information into a computer and allowing that computer to learn from this corpus over time and generate new content.
Westerlund (2019)Hyper-realistic videos digitally manipulated to depict people saying and doing things that never actually happened.
Deepfakes are widely produced by autoencoders (Kingma & Welling, 2013) and generative adversarial networks (GAN) (Goodfellow et al., 2014). In particular, deepfakes can be created by a series of deep learning processes in which pairs of two artificial networks, the generator and the discriminator, pit against each other in generative adversarial networks (GANs; Creswell et al., 2018). With several facial images from different angles, the generator can produce authentic-looking digital manipulations while the discriminator attempts to tease the authentic and fake images apart (Creswell et al., 2018; Westerlund, 2019). In this simultaneous and competitive environment of two networks, the authenticity gradually improves (Öhman, 2019, pp. 1–8), ultimately creating fake, but hyper-realistic media content (Fletcher, 2018; Maras & Alexandrou, 2019).
Deepfake algorithms have become democratized with open Internet-based platforms or applications such as FaceSwap, FakeApp, DeepFaceLab, and DeepNude, contributing to the widespread production of deepfakes by making the quality deepfakes production process surprisingly easy (Gosse & Burkell, 2020; Kolagati et al., 2022). This development of digital technology under the rapidly changing media environment exacerbates the difficulty of distinguishing between real and fake, potentially misleading audiences (Maras & Alexandrou, 2019). Particularly, videos represent the largest proportion of the content manipulated by deepfakes worsens the problem, as it completely reverses the conventional notion that videos depict the absolute truth (Bates, 2018; Gosse & Burkell, 2020; Köbis et al., 2021; Wagner & Blewer, 2019). Bates (2018) explains that deepfake technology threatens individuals' traditional beliefs about the trustworthiness of live scenes that they have observed. Wagner and Blewer (2019) also indicate that the emergence of deepfakes has disrupted the public's trust in moving images as pure, unalterable truth. Similarly, Gosse and Burkell (2020) point out deepfakes' potential to interfere with any aspect of a shared framework to understand reality. In particular, one study conducted an online experiment to test whether people can detect one of the most difficult deepfake videos, proving that people cannot reliably detect deepfakes but overestimate their abilities to identify them (Köbis et al., 2021). The findings of the experiment confirmed that a lot of online users may fall victim to digital disinformation in the current “post-truth era.”

2.2. Deepfake pornography featuring K-pop idols

One of the notorious consequences of deepfake technology is the creation of nonconsensual, hardcore deepfake pornographic videos (DPVs) that superimpose the faces of celebrities onto the bodies of different pornographic actors. Unfortunately, the characteristics of the K-pop industry and idols tend to increase demand for DPVs. To illustrate, K-pop is a world-renowned South Korean pop music style that has become part of the dominant music culture in recent years (Kang, 2016). Idols refer to the K-pop singers raised from an early age under the systematic training provided by entertainment agencies (Cha & Choi, 2011; Shim, 2013). Thus, K-pop idols must work on a strict schedule according to the investments they receive from their agencies, and therefore they are commercially molded and exploited. Specifically, celebrity sex scandals hurt not only the image of the idols themselves but also that of the agency and eventually can affect the company's stock prices, providing a plausible excuse for agencies to control idols' dating lives. Therefore, considering the vulnerability of perfectly imaged idols in the K-pop industry to sex scandals, DPV, which disparages female celebrities, gives the malevolent actors the pleasure of fulfilling undesirable desires (van der Nagel, 2020). For example, Kim (2018) reveals that the sexual objectification of K-pop idols could satisfy men's hidden urges by perpetuating women's subordinate image. Dickson (2019) also claims that anti-fans are likely to feel pleasure or reward from tarnishing innocent female celebrities' reputations. As such, the attributes of the K-pop industry amplify the demand for DPVs.
However, the DPVs cause long-lasting harm to the victims, including emotional distress, physical illness, job loss, and reputational degradation (Bae, 2019; Chesney & Citron, 2019; Melville, 2019). This damage is exacerbated by the nature of cyberspace and deepfake methods. First, it is difficult to control the speed and scope of content production and distribution online. Additionally, the anonymity of online platforms allows DPV creators to produce content without moral accountability. Finally, rudimentary and restrictive regulations on online sex crimes enable creators to commit digital forgery without any legal liability. For example, a partial amendment to the Special Act on punishing sexual violence crimes was approved at a Cabinet meeting, and a law strengthening discipline for deepfake videos began to take effect on June 25, 2020, in South Korea. Still, the law is ineffective for overseas creators (Lee, 2020). In addition to the online environment's problems, the uniqueness of deepfake technology complicates the problem. DPVs are created by deep learning algorithms that are freely accessible anywhere online, thus hindering the political process, allowing anyone to attack cybersecurity systems that are not yet prepared, and creating false beliefs or objectionable sexual content (Gosse & Burkell, 2020; Kolagati et al., 2022). As such, the online environment's characteristics and unique deepfake techniques increase the threat of DPV victimization.

2.3. Negative emotions: Anger and guilt

As DPVs are inherently abusive contents and deceptively alter videos without consent, they may elicit negative emotions among general online users. Prior studies have investigated negative feelings about perpetrators’ moral violations, referred to as negative moral emotions (Carver & Harmon-Jones, 2009; Haidt, 2003; Tangney et al., 2007). Negative moral emotions can be categorized into several clusters, including two principal clusters of other-condemning emotions cluster (e.g., anger, contempt, disgust) and self-conscious emotions cluster (e.g., guilt, shame, embarrassment). The first cluster reflects moral emotions toward others with disapproval, while the second cluster includes an ongoing self-evaluation process of the moral worth (Rozin et al., 1999; Tangney et al., 2007). To be specific, contempt often arises from hierarchical relationships, which is felt by one group of members to another group who are slightly inferior, thus linked to racism or prejudice (Rozin et al., 1999). Rozin et al. (1999) found that emotion of disgust is related to violations of divine ethics, such as problems with hygiene or assaults on human dignity; in addition, it involves a tendency to avoid or expel (Haidt, 2003). Moreover, Haidt (2003) contends that shame and embarrassment are triggered by any painful revelation of the self to be flawed or defective, especially that he or she has violated moral standards of action, thus corresponding with motivation to deny, escape, or hide. The difference between the two emotions of shame and embarrassment often relies on the fact that embarrassment occurs more in hierarchical interactions. DPV may not be related to negative emotions elicited upon the hierarchical interactions (e.g., contempt, embarrassment) as well as negative emotions associated with avoidance or hiding tendencies (e.g., disgust, shame, embarrassment) because those emotions may not mediate active behavioral responses in the DPV context.
In the meantime, individuals may experience anger in various situations, such as when affronted, frustrated, or injured by offending others (Carver & Harmon-Jones, 2009; Tangney et al., 2007). Particularly, Tangney et al. (2007) describe anger as an emotion that arises when a perpetrator who has violated basic moral standards is viewed as acting in a morally repulsive manner toward a third party. Likewise, Rozin et al. (1999) illustrate that anger motivates bystanders to rectify injustices that they have observed, similar to other studies’ results suggesting that anger is associated with confrontational coping behaviors that can change stressful situations (Lazarus, 1991). Because DPV is morally reprehensible, fake content generated by an ill-intentioned creator, online users may experience anger if they encounter it. As predicted in previous research, anger can stimulate users to take several coping actions.
On the other hand, guilt results from evaluation of one's own behavior, usually when the individual has violated moral rules (Tangney et al., 2007). Individuals who experience guilt focuses on their bad behavior rather than evaluating themselves as people (Haidt, 2003). Baumeister et al. (1994) explain that guilt is more commonly induced when users believe that their behavior may pose a severe threat to the victim. Moreover, research has shown guilt to foster empathy and motivate individuals to modify or improve situations using constructive strategies (Sheikh & Janoff-Bulman, 2010; Tangney et al., 2007). Further, empirical results indicate that guilt is most effective in stimulating individuals to follow a moral direction in their lives by taking reparative actions such as apologies, confessions, and prosocial behavior (Haidt, 2003; Sheikh & Janoff-Bulman, 2010; Tangney et al., 2007). As such, internet users may feel guilty about their actions after watching DPVs. They are likely to be motivated to correct such actions and situations by evaluating them negatively, feeling empathy for DPV victims, and taking moral countermeasures.

2.4. Coping behavior

Coping behavior is the process of taking action to address stressful situations (Carver et al., 1989). Previous research has identified problem-focused coping and emotion-focused coping as the two major types of coping behavior (Lazarus & Folkman, 1984). According to Carver et al. (1989), there are variations in problem-focused and emotion-focused coping. While problem-focused coping seeks to actively resolve or change the subject to overcome the stress, emotion-focused coping aims to reduce or control individuals’ stressful emotions. More specifically, problem-focused responses include activities like taking actions directed toward other people to blame, increasing stepwise efforts, or seeking social support through advice, information, and help from higher institutions (Carver et al., 1989). Exercised caution as holding back actions until an opportunity comes is also the problem-focused strategy (Aldwin & Revenson, 1987). Emotion-focused coping involves responses such as seeking social support emotionally, especially for moral approval or understanding (Carver et al., 1989).
In this regard, general online users who watch DPVs may perform two types of coping behaviors after experiencing negative emotions (Carver et al., 1989). In fact, given the anonymity of DPVs creators, a direct confrontation with any DPVs creator is not feasible. In terms of problem-focused coping, users may report DPVs to the police or relevant investigation agencies to seek social help from higher institutions. Moreover, they can ask celebrity management agencies to take corrective measures or ask platform providers to delete DPVs. In terms of emotion-focused coping, users can engage in social advocacy for celebrities with online petitions or emotional support. Therefore, this study considers two types of coping responses in the context of DPV: immediate problem solving and indirect emotional support.

2.5. DPV-related coping behaviors mediated by negative emotions

Emotions have an adaptive function related to behavioral tendencies (Plutchik, 1980). Notably, prior literature has revealed that negative moral emotions play an important role in determining individuals' moral decision making and coping processes in the real world (Haidt, 2001, 2003; Monin et al., 2007; Shweder & Haidt, 1993). Accordingly, negative emotions are considered as important predictors of individuals’ moral judgments (Haidt et al., 1993; Shweder & Haidt, 1993). For example, Haidt (2001) highlights that people engage in moral behaviors based on moral emotions. Along the same lines, we also argue that anger and guilt are associated with specific coping responses to malicious deepfake content.
According to prior studies, anger is related to witnessing morally offensive behavior aimed at third parties, inducing bystanders to take action to redress the injustice (Rozin et al., 1999; Tangney et al., 2007). Thus, users who feel anger are likely to enact direct, immediate, and overt punishment for immoral behaviors (Carver & Harmon-Jones, 2009; Fischer & Roseman, 2007). Correspondingly, individuals who may encounter DPVs are likely to engage in active coping behaviors to alter the situation. Furthermore, anger will positively affect the emotion-focused coping behavior of seeking emotional support, as most negative emotions elicit both types of coping (Carver et al., 1989). For this reason, we propose the following hypotheses:

H1a

Online users' anger over DPVs positively affects direct problem-solving behavior.

H1b

Online users' anger over DPVs positively affects indirect emotional supporting behavior.
Research has proven that guilt tends to function as a moral barometer and provide immediate and critical feedback on everyday behavior (Haidt, 2003). Guilt thus strongly influences moral behavior (Tangney et al., 2007). Extant research about guilt has illustrated its association with constructive and proactive pursuits that can change or compensate for guilt-inducing behavior (Haidt, 2003; Sheikh & Janoff-Bulman, 2010; Tangney et al., 2007). More specifically, guilt may inhibit pornography use or hypersexual behavior (Gilliland et al., 2011). Consequently, guilt is likely to be associated with DPV observers’ direct problem-solving responses online. Additionally, as guilt appears to be linked to empathy, it may encourage emotion-focused coping behaviors in DPV contexts (Tangney et al., 2007).

H2a

Online users' guilt over DPVs positively affects direct problem-solving behavior.

H2b

Online users' guilt over DPVs positively affects indirect emotional supporting behavior.

2.6. Influences of individual characteristics on emotion and behavior

The extant studies have illustrated how individual differences may influence moral sentiments (Shweder & Haidt, 1993; Wang & Kim, 2022) and subsequent behaviors (Carver et al., 1989; Haidt, 2001, 2003; Monin et al., 2007). Moreover, perceptions and acceptance of new information technology may vary depending on individual experiences and characteristics (Gal & Berente, 2008; Xu, 2007). We hypothesized that personal attributes would influence emotions in response to witnessing DPVs online. DPVs are content incorporating new information technologies, but they are inauthentic, illegal, and target individuals sexually without their consent. Thus, DPVs can evoke negative moral emotions and subsequent reactions, the degree of which depends on individual characteristics. Consequently, this study analyzes relevant experiences and features, such as previous tolerance for sexual harassment, prior fondness of celebrities, media exposures, and demographics, to prove the relationship of these variables with anger and guilt, the most representative emotions that we would expect users to feel.

2.6.1. Sexual harassment attitudes

While the history of sexual harassment (SH) is very long, it has only recently been recognized as a social problem. Mazer and Percival (1989) were pioneers in redefining SH as unacceptable and controversial behavior, arguing that SH can have a negative effect on people who have experienced it. Since then, SH has been widely considered an act of violating or demeaning an individual based on sex or gender (Lopez et al., 2009). Some studies have explored several individual features that may determine tolerance for SH situations. For example, Mazer and Percival (1989) confirm that men have higher tolerance for SH behavior and are more partial to traditional sex roles than relative to women. Ford and Donis (1996) indicate that age has a significant relationship with individuals' tolerance of SH. For instance, women became more tolerant of SH until age 50 but became less tolerant after 50, while men tended to exhibit the opposite pattern. Tang et al. (1995) argue that individuals' intolerance of SH is related to their support for gender equality movements and objection to traditional gender roles. As such, this study investigates how individuals’ sensitivities to SH that may be affected by their own characteristics and circumstances may influence their feelings about DPVs. Insofar as DPVs constitute sexual harassment, SH tolerance could predict emotional responses to DPVs. Thus, users with greater tolerance for SH would be expected to exhibit relatively less anger and guilt about DPVs.

H3a

SH tolerance is negatively related to anger over DPVs.

H3b

SH tolerance is negatively related to guilt over DPVs.

2.6.2. Celebrity involvement: affinity

Celebrity involvement (CI) is a psychological process and refers to individuals' thoughts and feelings about and reactions to celebrities (Brown & de Matviuk, 2010). The term is related to public consumption of media, a multidimensional construct that includes affinity (Giles, 2002), parasocial relationship (Brown et al., 2003), and identification (Brown et al., 2003). Giles (2002) describes affinity as individuals' initial and general fondness of celebrities, stimulated in situations when viewers observe physical attractiveness, credibility, or perceived similarity with certain stars (Veer et al., 2010). In accordance with Giles' (2002) definition and the uniqueness of affinity, this study applies affinity in the DPV context to examine individuals' overall liking of media figures. Existing literature has examined that involvement is a meaningful predictor of attitudinal and behavioral changes among individuals (Brown et al., 2003; Wirth & Schramm, 2005). For instance, Wirth and Schramm (2005) demonstrate that individuals' positive feelings toward a media figure whom they like have a significant relationship with symmetric co-emotions and the will to act on the celebrities' behalf. Therefore, those who feel closer to celebrities are likely to feel angrier and guiltier about DPVs because they are more likely to empathize with these celebrities’ pain. More specifically, fans of K-pop celebrities known for their enthusiastic, supportive, and cohesive nature are expected to feel the most negatively about the proliferation of DPVs.

H4a

Celebrity involvement (affinity) is positively related to anger over DPVs.

H4b

Celebrity involvement (affinity) is positively related to guilt over DPVs.

2.6.3. Media exposure: Typical and pornographic video watching

Desensitization has been widely studied, especially in the context of online video games and media violence (Anderson & Dill, 2000; Fanti et al., 2009; Krahé et al., 2011). The term generally refers to the gradual reduction of responses to an arousal-eliciting stimulus due to repeated exposure (Krahé et al., 2011). In the context of media consumption, Fanti et al. (2009) observe that participants' enjoyment of a given humorous film gradually decreased with repeated viewing. Similarly, Anderson and Dill (2000) demonstrate that viewers often lose their ability to experience negative emotions when exposed to violent media repeatedly. Krahé et al. (2011) also find a significant negative relationship between individuals' habitual consumption of violent media content and emotional responsivity to it. Desensitization even occurred when individuals watched videos conveying sadness. For example, participants’ emotional responsiveness declined with repeated viewing of two film clips about the death of a loved one (Krahé et al., 2011). As such, researchers suggest that habitual exposure to emotion-laden media content (particularly videos) results in a reduction in psychological reactivity.

H5a

Typical video watching is negatively related to anger over DPVs.

H5b

Typical video watching is negatively related to guilt over DPVs.
Some investigations of desensitization to sexually violent films have indicated that those who had regularly been exposed to filmed violence against women experienced a significant decrease in anxious and depressed feelings stemming from the related content (Linz et al., 1989). This experimental group judged that sexual harassment victims were less damaged than those in the control groups who were not exposed to these videos (Linz et al., 1984). Accordingly, we posit that online users who had watched sexually offensive films or pornographic content habitually would feel less angry and guilty about DPVs because of their desensitization to similar kinds of videos.

H6a

Pornographic video watching is negatively related to anger over DPVs.

H6b

Pornographic video watching is negatively related to guilt over DPVs.

2.6.4. Demographic factors: Gender and age

Demographic factors may influence individuals' responses to DPVs. Gender appears to be an especially strong predictor of individuals' reactions. For example, existing literature has demonstrated gender differences in reactions to sexually violent media content (Biber et al., 2002; Hald, 2006; Lindsay et al., 2016; Linz et al., 1989; Malovich & Stake, 1990; Rupp & Wallen, 2007). Some studies have revealed that women report strong negative feelings of disgust, guilt, and shame toward hardcore pornographic videos (Rupp & Wallen, 2007). However, men have been found to enjoy pornography even if the videos lack a relational context or emotional attachment (Hald, 2006). Other extant studies of online SH have shown that women are less tolerant of and more uncomfortable with sexually harassing online pictures and jokes on the internet relative to men (Biber et al., 2002). Similarly, more frequent exposure to SH in real life may lead women to be less patient, angrier, and more empathetic to the situations of women sexually exploited online (Lindsay et al., 2016; Malovich & Stake, 1990). Thus, considering that DPVs’ main target is women and that attitudes toward pornography and SH vary according to gender, we expected that emotional responses to DPVs would also differ depending on gender. Specifically, we propose that women would feel angrier and guiltier about this issue.

H7a

Relative to men, women experience greater anger over DPVs.

H7b

Relative to men, women experience greater guilt over DPVs.
Age may also play an important role in determining viewers’ responses. Prior literature has explained that younger generations have gained both frequent and free access to the Internet with the widespread use of computers and mobile devices (Chan et al., 2016; Greenfield, 2004). According to the Pew Research Center, 95% of all US teens (13–17 years) had accessed to smartphones in 2018, and 90% went online at least several times a day. Teens also constantly use social networking sites (SNSs), among which YouTube, Instagram, and Snapchat are the most popular (Anderson & Jiang, 2018). As younger individuals use the Internet more, it is more likely for them to be sexually targeted or to encounter explicit content, including sexual content (Chan et al., 2016). Therefore, in the DPV context, younger individuals likely exhibit less emotion toward DPVs relative to older individuals because of greater prior exposure to sexually degrading media content on SNSs or other internet platforms. We expect greater exposure to provocative content would entail greater desensitization among younger individuals.

H8a

Age is positively related to anger over DPVs.

H8b

Age is positively related to guilt over DPVs.
Based on the foregoing hypotheses, Fig. 1 presents the research model used in this study.
Fig. 1
  1. Download: Download high-res image (522KB)
  2. Download: Download full-size image

Fig. 1. Research model.

3. Methodology

3.1. Data collection

Prior to the actual survey, two pilot surveys were conducted to identify internet users' emotions toward DPVs. The first survey included open-ended questions about feelings after watching DPVs, and the second survey had a quantitative questionnaire. Throughout two pilot surveys, step-by-step, we have tried to identify some negative emotions which are more likely to related to users' coping behavior. In line with literature review on negative emotions, we found that negative emotions related to users’ coping behavior might be anger and guilt. Accordingly, we selected and used anger and guilt as emotional variables in the actual survey.
We collected data for the actual survey from August 27 to September 1, 2020, via the professional online survey agency Macromill Embrain in Korea to test the research hypotheses. The subjects were randomly selected using stratified sampling based on the proportion of age and gender in Korea. Overall, the selected sample was well balanced. The targeted survey participants were those who have experience watching any deepfake video content on an online platform. To participate in the survey, individuals were required to have that experience. Since we could not show DPVs directly to the participants under IRB regulations, we provided a brief explanation of deepfake technology and deepfake pornography at the beginning of the survey to ensure participants' clear understanding of the survey context. Participants read text scenarios portraying accidental DPV encounters and encountered relevant screenshots of DPVs, then we measured their emotional and behavioral responses to the DPVs. All questionnaires were answered anonymously. After excluding unreliable responses, we used 293 valid questionnaires in the final data analysis. Table 2 reports the participants’ demographic characteristics.

Table 2. Participant demographics.

MeasuresFrequencyPercent
GenderMale14850.5
Female14549.5
Age18–297625.9
30–396421.8
40–497625.9
50–597726.3
Highest level of educationHigh school3210.9
College3311.3
University18864.1
Graduate school4013.6
Video watching (Per day)Less than 30 min6421.8
30 minutes–1 hour7625.9
1–2 h7023.9
2–3 h4314.7
3–4 h217.2
4–5 h72.4
More than 5 h124.1
Porn watchingOnce a week289.6
Twice or three times a week3913.3
Once a day134.4
Once or twice a month3612.3
Intermittently9532.4
I used to watch it, but now I do not8228.0
Total293100.0

3.2. Measures

The survey contained four parts, measuring individuals' preexisting perceptions, negative emotions, coping behaviors, and demographic characteristics. We derived all measures from prior literature, with some modifications to fit our specific research context. All items utilized 5-point Likert response scales ranging from 1 (“strongly disagree”) to 5 (“strongly agree”). We adopted measures of tolerance for sexual harassment from Beauvais (1986) and Mazer and Percival (1989), modifying the Sexual Harassment Attitude Scale (SHAS) to fit the DPV context. We derived items related to celebrity involvement from Wen and Cui (2014) that measured the level of affinity with media figures. To measure the negative emotions of anger and guilt, we used a modified version of Izard et al.’s (1993) Differential Emotions Scale, which has been widely used for understanding individuals' multivariate mood states. We developed behavioral measures from studies by Bowes-Sperry and Powell (1999) and Benavides-Espinoza and Cunningham (2010), which assessed the extent to which participants would take action when observing SH. After eliminating similar items, we modified the questionnaires to fit the DPV and online contexts. We also included demographic information, including gender, age, and average daily video consumption, in the analysis.

3.3. Measurement model

We employed partial least squares structural equation modeling (PLS-SEM), using SmartPLS 3.0 as the analytic tool, to test the hypotheses. PLS-SEM is particularly appropriate when dealing with a relatively small sample size (Hair et al., 2011). Further, this approach can simultaneously estimate the structural model with high predictive accuracy, although typically in situations where applications have little available theoretical background (Wong, 2013). We adopted PLS-SEM given our small sample size and the relative insufficiency of existing theory, as discussions on deepfake technology are in their early stages and factors that may influence the individuals’ responses to DPVs have yet to be examined. We also used SPSS 25.0 to enter and code the survey data.
We have employed a reflective approach of constructs from the structural model. In PLS-SEM, the measurement model is assessed based on the reliability and validity of the measurement constructs. To assess reliability, we computed Cronbach's alphas and composite reliabilities. As shown in Table 3, all constructs' coefficients (range = 0.755–0.932) exceeded the minimum required value (0.70), meaning that they were internally consistent. We evaluated convergent validity using factor loadings, composite reliability (CR), and average variance extracted (AVE). The suggested threshold is 0.5 for AVE values and 0.7 for factor loadings and CR values. Table 3 indicates that all values demonstrated acceptable convergent validity, with AVE values ranging between 0.723 and 0.931, CR values ranging between 0.891 and 0.964, and all factor loadings exceeding 0.790. To evaluate discriminant validity, we compared the constructs' correlation coefficients and the square root of the AVE, the latter of which should be greater (Fornell & Larcker, 1981). Moreover, outer loading values should be larger than cross-loadings. As shown in Table 4, Table 5, the square roots of the AVE values exceeded the correlation coefficients of the different constructs, and each item provided strong evidence of discriminant validity.

Table 3. Reliability and validity results.

ConstructFactor loadingAVECRCronbach's alpha
SHSH1.790.723.912.872
SH2.853
SH3.889
SH4.866
CICI1.866.804.943.922
CI2.925
CI3.938
CI4.856
VWVW1.0001.0001.0001.000
PWPW1.0001.0001.0001.000
GENGEN1.0001.0001.0001.000
AGEAGE1.0001.0001.0001.000
ANAN1.919.860.949.919
AN2.939
AN3.922
GUGU1.966.931.964.926
GU2.964
DPSDPS1.915.830.951.932
DPS2.885
DPS3.951
DPS4.892
IESIES1.899.803.891.755
IES2.893
Note: AVE = average variance extracted; CR = composite reliability; SH = sexual harassment tolerance; CI = celebrity involvement; VW = video watching; PW = pornography watching; GEN = gender; AN = anger; GU = guilt; DPS = direct problem solving; IES = indirect emotional supporting.

Table 4. Correlations among measured variables.

Empty CellNo.QsMSD12345678910
SH41.824.928.850
CI43.4291.071-.018.897
VW12.8291.522-.045.1591.000
PW12.6111.523.260-.001.2101.000
GEN11.495.500-.209.318.047-.3391.000
AGE12.5261.137.323-.052-.222-.130-.0131.000
AN33.7861.040-.288.233-.029-.479.450-.008.927
GU23.6591.044-.291.151-.016-.431.386-.091.837.965
DPS43.2991.189-.190.243.005-.315.376-.029.676.606.911
IES23.7011.057-.294.291.044-.291.296-.070.661.587.773.896
Note: Abbreviations are same as in Table 2.

Table 5. Factor loadings for measured variables.

Empty CellSHCIVWPWGENAGEANGUDPSIES
SH1.790.010-.041.262-.263.134-.256-.262-.157-.215
SH2.853-.014-.064.141-.057.386-.198-.232-.126-.244
SH3.889-.034-.043.221-.161.299-.244-.237-.171-.275
SH4.866-.025-.008.243-.208.298-.270-.255-.186-.266
CI1-.109.866.151.002.231-.081.156.084.161.234
CI2-.032.925.130-.005.270-.005.219.127.208.289
CI3.020.938.139-.020.333-.053.272.203.296.314
CI4.047.856.179.059.291-.071.110.054.122.117
VW-.045.1591.000.210.047-.222-.029-.016.005.044
PW.260-.001.2101.000-.339-.130-.479-.431-.315-.291
GEN-.209.318.047-.3391.000-.013.450.386.376.296
AGE.323-.052-.222-.130-.0131.000-.008-.091-.029-.070
AN1-.297.245-.013-.472.412-.041.921.767.604.621
AN2-.261.167-.052-.437.413.021.939.799.616.628
AN3-.242.236-.015-.425.428-.001.922.761.660.590
GU1-.306.137-.035-.411.361-.117.784.966.586.577
GU2-.255.155.003-.422.384-.057.831.964.583.555
DPS1-.183.208.018-.285.324-.024.622.533.915.688
DPS2-.102.207-.035-.276.271-.032.540.516.885.675
DPS3-.163.206.032-.263.371-.026.636.586.951.719
DPS4-.234.262.000-.323.393-.023.655.570.892.731
IES1-.213.263.002-.259.289.001.599.535.787.899
IES2-.316.258.077-.264.240-.129.585.516.595.893
Note: Abbreviations are same as in Table 2.

4. Results

We used 5000 bootstrap subsamples to test the structural path's significance with respect to the measured variables (Wong, 2013). In terms of the relationship between negative emotions and coping behavior, anger was a significant predictor of both direct problem solving (β = 0.564, t = 6.445, p < .001) and indirect supporting behavior (β = 0.570, t = 7.057, p < .001). However, guilt was not significantly related to either kind of coping behavior. While H1a and H1b were supported, H2a and H2b were not (see Table 6).

Table 6. Results of hypothesis testing.

HypothesisβtResult
H1a: Anger → Direct reporting.564∗∗∗6.445Supported
H1b: Anger → Indirect supporting.570∗∗∗7.057Supported
H2a: Guilt → Direct reporting.1341.393Rejected
H2b: Guilt → Indirect supporting.1101.259Rejected
H3a: SH tolerance → anger-.1442.388Supported
H3b: SH tolerance → guilt-.1342.135Supported
H4a: Celebrity involvement → anger.157∗∗2.611Supported
H4b: Celebrity involvement → guilt.0801.189Rejected
H5a: Video watch → anger.0060.101Rejected
H5b: Video watch → guilt.0080.131Rejected
H6a: Porn watch → anger-.359∗∗∗6.195Supported
H6b: Porn watch → guilt-.336∗∗∗5.463Supported
H7a: Gender → anger.248∗∗∗4.581Supported
H7b: Gender → guilt.217∗∗∗3.679Supported
H8a: Age → anger.0040.059Rejected
H8b: Age → guilt-.0831.587Rejected
Note: ∗p < .05, ∗∗p < .01, ∗∗∗p < .001.
Moreover, this study aimed to understand the determinants of online users' negative emotions toward K-pop idols' DPVs and subsequent behaviors. The results showed that individuals' prior perceptions, experience with pornographic content, and gender significantly affected their emotions in response to DPVs. Particularly, H3 predicted that SH tolerance would be negatively related to the two negative emotions. We found statistically significant negative relationships between levels of SH tolerance and anger (β = −0.144, t = 2.388, p < .05) as well as between SH tolerance and guilt (β = −0.134, t = 2.135, p < .05). H4 predicted that celebrity involvement would be positively related to both anger and guilt. While there was a significant effect of CI on anger (β = 0.157, t = 2.611, p < .01), there was no significant effect on guilt. Thus, the results supported H4a but not H4b. H5 and H6 predicted that video watching experience would affect individuals’ negative angry and guilty reactions to DPVs. The results showed that general video exposure had no significant impact on either anger or guilt. Pornography exposure, on the other hand, significantly affected anger (β = −0.359, t = 6.195, p < .001) and guilt (β = −0.336, t = 5.463, p < .001), supporting both H6a and H6b. H7 and H8 predicted that gender and age would significantly influence anger and guilt. However, only gender had significant effects on anger (β = 0.248, t = 4.581, p < .001) and guilt (β = 0.217, t = 3.679, p < .001). Thus, H7a and H7b were supported, but H8a and H8b were not. Table 6 and Fig. 2 summarize the hypothesis testing results.
Fig. 2
  1. Download: Download high-res image (459KB)
  2. Download: Download full-size image

Fig. 2. Results of the research model.

5. Discussion and conclusion

5.1. Key findings

This study sheds light on the recent phenomenon of creating surrealistically crafted media content using AI algorithms. By placing media figures' faces on others' nude bodies, this sophisticated deepfake fabrication technology is causing great harm. However, most previous work has concentrated only on the technological and legal ramifications of deepfakes, with little to no research on online users' responses. To address this gap, this study explored general online users' responses to deepfake pornographic content by analyzing the relationship between individual characteristics, negative emotions, and coping strategies. The results demonstrate that anger is a strong predictor of users' coping behavior in response to DPVs. The PLS-SEM analysis shows that anger has a profound impact on direct reporting of DPVs' existence to related stakeholders, including the police, idols' management agencies, broadcasting deliberation committees, or platform providers. Anger is also associated with users' emotional support of the victims of DPVs. Users tend to socially encourage DPV victims or raise awareness of the issue via online petitions. For example, more than 390,000 South Koreans signed a nationwide online petition to punish DPV creators for sexual crimes. This could have directed public attention to sexually demeaning deepfake videos because the online petition was directly operated by the Blue House, the executive office in South Korea. Consequently, as suggested by prior research results, anger appears to elicit both types of coping responses, direct problem-focused coping behavior and indirect emotion-focused coping behavior, in the DPV context (Carver et al., 1989). However, the findings reveal that guilt is not significantly related to either type of coping behavior. These results indicate that guilt may correct an individual's bad behavior (Haidt, 2003) but may not be sufficiently powerful to induce coping behavior among online users who have encountered DPVs. According to previous literature, guilt exerts a strong influence on individual compensatory, prosocial, and moral actions (Sheikh & Janoff-Bulman, 2010). However, users feeling guilt may not cope directly because the moral behavior of such individuals involves apologizing or confessing, which tends to occur in one-on-one, personal relationships (Haidt, 2003; Sheikh & Janoff-Bulman, 2010; Tangney et al., 2007). Moreover, other empirical results support that guilt alone may not be sufficient to motivate individuals to change, although guilt is related to a desire to rectify socially undesirable behavior (Gilliland et al., 2011). Thus, nonsignificant relationships between guilt and the two coping behaviors are plausible.
Regarding the relationship between individual traits and negative emotions, anger and guilt are negatively related to SH tolerance, while celebrity involvement is positively related to anger only. Specifically, the results imply that individuals' previous opinions of SH are the salient factor generating anger and guilt in the DPV context. This finding suggests that users with more sensitivity to SH are more likely to be angry about deepfake content and even feel guilty when they encounter DPVs. One possible explanation is that this group presumably perceives DPVs as a cruel digital sex crime. Another explanation is that these individuals likely consider the female celebrities in the DPVs as victims of sexual crimes. These perceptions of DPVs and DPV victims may have influenced users' emotional responses. Personal preference for celebrities is also an important predictor of negative emotional reactions to DPVs. Particularly, individuals exhibiting more robust emotional responses tend to be enthusiastic and supportive fans interested in celebrities' happiness and welfare. Therefore, if these individuals witness their favorite entertainer becoming a victim of DPVs and learn that the celebrity is suffering from not only the initial humiliating comments but also from rapid dissemination of content and false information, they are likely to feel angry over the DPV that provoked this situation. Thus, greater affection for K-pop idols tends to yield angrier responses among users to DPVs (Wirth & Schramm, 2005). However, CI does not have a significant relationship with guilt. Guilt over watching DPVs can be expressed regardless of whether individuals like the idols or not. Intuitively speaking, someone with strong affection for idols may feel more guilty about not being able to protect their beloved idols from DPVs, but others may feel less guilty because they believe that DPVs’ production is beyond their control.
While differences in video watching experience do not influence anger or guilt over DPVs, pornography exposure is most strongly, and negatively related to anger and guilt among online users. These results indicate that individuals with more experience with pornographic content tend to respond less emotionally in the DPV context. Consistent with prior research, frequent exposure to hypersexual content can desensitize individuals to similar content (Linz et al., 1984), regardless of who was victimized or what was used to fabricate the video. In contrast, neither anger nor guilt is related to prior video consumption. Given the widespread consumption of videos among the participants—over 80% watched videos for more than 4 h a week (Table 2)—exposure to typical videos may not have been a suitable variable to predict negative emotions. Moreover, in past studies, as repeated exposure to similar content genres (Fanti et al., 2009; Krahé et al., 2011) or violent media (Anderson & Dill, 2000) predicted desensitization, overall media consumption may not be reliably related to emotion.
Gender turns out to be one of the strongest predictors of anger and guilt in the context of DPVs. The findings show that women are more likely to exhibit powerful negative feelings toward the DPVs compared to men. One explanation is that female celebrities are the most frequent targets of DPVs. Furthermore, the memories of women who have experienced SH in real life may increase their empathy for victims and reduce their tolerance for similar online SH content. On the contrary, age does not seem to be related to emotional responses to DPVs. We expected younger individuals to be less sensitive to sexual content because they are frequently exposed to such videos on various online platforms and SNSs, and they can have relatively open conversations about sexual issues. However, in the South Korean context, it is not just the younger generation who have access to the Internet or provocative content. Advances in technology have enabled older individuals to access manipulated online content with SH characteristics. In sum, individuals’ feelings toward DPVs may be similar regardless of age; hence, meaningful relationships may not appear.

5.2. Implications

This study makes important contributions to both theory and practice. Theoretically, this study sheds new light on the potential coping behaviors exhibited by online users in response to DPVs. The results show that anger is an essential antecedent to both immediately reporting of DPVs and providing emotional support to DPV victims. This aligns with past studies showing that the moral emotion of anger is associated with virtuous coping responses that can change the situation (Carver et al., 1989; Carver & Harmon-Jones, 2009; Fischer & Roseman, 2007). Moreover, this study is one of the timely attempts to examine general online user behavior in the context of deepfake technology. Although several previous studies have addressed deepfake technology's non-technical aspects, there has been little research interest in users who may have encountered DPVs on the Internet. The study of Wang and Kim (2022) is one of the earliest works to identify meaningful relationships between internet users' characteristics and their negative emotions to DPVs. In addition to this, this study expands the discussion of user responses to behaviors. Thus, it may fill the research gap by proposing a research model (Fig. 1) to validate users' different emotional and behavioral responses, which stem from distinct traits and experiences. Moreover, this study classifies personal traits, negative moral emotions, and potential coping behaviors into more specific variables.
This study's results also provide practical implications for platform operators, government, and academia. First, this study highlights the need for stronger governmental efforts to penalize DPV creation. No matter how often users report DPVs to related government organizations, it becomes meaningless if there are no applicable rules to punish DPV creators. Thus, establishing legal foundations for digital sex crimes and fake manipulation videos is an urgent task. Although the South Korean government recently established new DPV-related regulations, they apply only to those who produce or distribute DPVs and not to those who consume or watch them (Kang, 2021). However, it seems necessary to punish consumers because female celebrities are suffering from excessive distribution and production of DPVs, and this supply often stems from the corresponding demand. Besides, the fact that deepfake perpetrators can remain anonymous and that most of them are agitators from abroad suggests that the current regulations are ineffective. Therefore, the government should continuously supplement and revise DPV-related legislation.
Second, this study suggests that platform operators should assume social responsibility for controlling malicious deepfake content. Although netizens must cope with the DPVs, the issue can be resolved only with platforms' effective policies or technological developments that prevent the creation, dissemination, and distribution of illegal content. Even showing a willingness to block DPVs, platform operators can discourage content production and prevent serious consequences. For example, Facebook is one of the leading companies taking preemptive actions to prohibit malicious deepfake content. Facebook first announced that it would ban the posting of deepfakes and immediately delete related content ahead of the 2020 US Presidential Election (Room et al., 2020). Technically, Facebook held a Deepfake Detection Challenge (DFDC) with Microsoft, AWS, and Partnership on AI to facilitate the development of new AI technologies that can identify media manipulations. However, there also are limits to the platforms’ endeavors because certain conditions must be met for the content to be blocked. Thus, attempts at platform-level solutions are meaningful but not sufficient to address the DPV problem. That is why media literacy, which allows users to accurately understand the context of the content and to view media critically, seems ever more important.
Such a need for media literacy is associated with the necessity of strategic digital literacy education. Digital literacy education may help online users understand the opportunities and threats in the online environment and practice digital technology ethics. Such education may include programs that enhance individuals' abilities to discern fake content or predict the possible intentions behind it. This process appears to be vital in the so-called post-truth era, where the line between truth and falsehood largely has become blurred. In addition, the current study's findings reveal that less tolerance for SH and less exposure to pornographic content are likely to elicit more anger, the emotion associated with individuals' prosocial behaviors in the DPV context. These results support the argument that we must improve sex education programs for media users. Curricula on gender sensitivity could help students fully understand gender differences and pursue gender equality. Better explanations about sex crimes in the digital environment and various coping strategies related to online SH, whether users are victims or bystanders, are also required. Furthermore, there should be educational efforts to make people feel uncomfortable with, realize the seriousness of, and understand the appropriate measures to handle digital sex crimes from an early age. By understanding online sex crimes and the potential victims of DPVs, this study anticipates the growth of active grassroots participation in reporting, detecting, and banning DPVs featuring K-pop idols. These potential programs we have listed may be conducted not only by schools but also by IT companies or celebrity management agencies. In sum, through collaborative efforts among schools, online platforms, and the government, we expect that the harm resulted from DPV proliferation can be alleviated.

5.3. Limitations and future research

Despite its meaningful implications, there are several limitations to this study. As an exploratory study that attempted to identify the antecedents of emotional and behavioral responses to DPVs among general online users, this study investigated significant impacts of individuals' perceptions, media experiences, and demographic characteristics on emotions and behaviors. However, the predictors suggested here may not completely explain users' experiences with DPVs. Other predictors could be included or excluded to analyze users' behaviors more comprehensively. Specifically, it would be necessary to examine how individual's personality or other psychological factors influence negative emotions and coping behavior. Future research is required to verify and improve this exploratory model by narrowing the possible influential factors and focusing on specific ones. In addition, this study provided a text scenario and related screenshots of DPVs to measure online users' responses via an online survey. Although the estimations may be limited, this method was the next-best option that still met the institutional review board's research ethical guidelines. Thus, future experimental studies may provide a fuller picture of users' responses to DPVs. Experiments may expose respondents to real deepfake pornographic videos, not images or text. Furthermore, the present study took place in South Korea, a country with one of highest internet penetration rate as of January 2022 (Johnson, 2022), as well as where the K-pop idol industry originated; it is also still a conservative and patriarchal society. Thus, this study's participants may not only be technologically more aware of DPVs but also more hostile to them. Comparative studies conducted with viewers in other countries could validate and generalize this study's proposed model.

Acknowledgments

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF- 2019S1A3A2099973) and by the MSIT (Ministry of Science and ICT), Korea, under the ITRC(Information Technology Research Center) support program (IITP-2020-0-01749) supervised by the IITP (Institute of Information & Communications Technology Planning & Evaluation).

References

View Abstract