arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2407.12876

Exploring the Use of Abusive Generative AI Models on Civitai

Yiluo Wei·Yiming Zhu·Pan Hui·Gareth Tyson
View original source ↗Read source document ↗Export metadata ↓

Abstract

The rise of generative AI is transforming the landscape of digital imagery, and exerting a significant influence on online creative communities. This has led to the emergence of AI-Generated Content (AIGC) social platforms, such as Civitai. These distinctive social platforms allow users to build and share their own generative AI models, thereby enhancing the potential for more diverse artistic expression. Designed in the vein of social networks, they also provide artists with the means to showcase their creations (generated from the models), engage in discussions, and obtain feedback, thus nurturing a sense of community. Yet, this openness also raises concerns about the abuse of such platforms, e.g., using models to disseminate deceptive deepfakes or infringe upon copyrights. To explore this, we conduct the first comprehensive empirical study of an AIGC social platform, focusing on its use for generating abusive content. As an exemplar, we construct a comprehensive dataset covering Civitai, the largest available AIGC social platform. Based on this dataset of 87K models and 2M images, we explore the characteristics of content and discuss strategies for moderation to better govern these platforms.

Explore related subjects

cs.SIcs.AIcs.CL

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yiluo Wei, Yiming Zhu, Pan Hui, Gareth Tyson. 2024-07-20. Exploring the Use of Abusive Generative AI Models on Civitai. https://arxiv.org/abs/2407.12876

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Higher-order Network phenomena of cascading failures in resilient cities

Modern urban resilience is threatened by cascading failures in multimodal transport networks, where localized shocks trigger widespread paralysis. Existing models, limited by their focus on pairwise interactions, often underestimate this systemic risk. To address this, we introduce a framework that confronts higher-order network theory with empirical evidence from a large-scale, real-world multimodal transport network. Our findings confirm a fundamental duality: network integration enhances static robustness metrics but simultaneously creates the structural pathways for catastrophic cascades. Crucially, we uncover the source of this paradox: a profound disconnect between static network structure and dynamic functional failure. We provide strong evidence that metrics derived from the network's static blueprint-encompassing both conventional low-order centrality and novel higher-order structural analyses-are fundamentally disconnected from and thus poor predictors of a system's dynamic functional resilience. This result highlights the inherent limitations of static analysis and underscores the need for a paradigm shift towards dynamic models to design and manage truly resilient urban systems.

cs.SI↗
arXiv2026

Measuring Time-Horizon Engagement Effectiveness: Persistence, Recency, and Re-Emergence

Online attention is commonly summarized using cumulative volume, peak activity, or arithmetic averages, but such measures can obscure differences between activity that is sustained over time, concentrated near the present, or renewed after dormancy. This paper introduces the Time-Horizon Engagement Effectiveness (TH-EE) framework, which constructs interpretable temporal profiles of online attention by distinguishing three related but non-equivalent properties: persistence, recency, and re-emergence. We evaluate the framework through controlled engagement traces, a proof-of-concept application to 1,850 YouTube videos across 37 topics (18 cohorts selected a priori as exemplars of persistent, acute, cyclical, and recently originating attention, and 19 cohorts corresponding to authoritatively debunked claims), and an event-level validation on eleven years of daily Wikipedia pageview series for the same topics. The controlled analyses show that the framework distinguishes distributed activity from concentrated bursts, introduces temporal-order sensitivity through recency weighting, and identifies renewed activity after a defined dormant interval. On the event-level series, the framework's reactivations co-locate with Kleinberg burst onsets, and PELT change points far more often than chance. The YouTube application shows that debunked-claim cohorts do not occupy a unique region of temporal-profile space; they exhibit heterogeneous patterns that overlap substantially with benign topics. These results support treating persistence, recency, and re-emergence as separate dimensions of online attention. TH-EE is a descriptive and comparative measurement framework, not a classifier of misinformation, coordination, intent, or content veracity.

cs.SI↗
arXiv2026

Enhancing Human Mobility Prediction with Spatially Aware LLM-based Multi-Agent Systems

Predicting a user's next POI is a task in human mobility modeling, yet LLM-based approaches focus on semantic reasoning from previous mobility records, while neglecting real-world spatial context. However, human mobility is inherently shaped by spatial cognition, including geographic distance and neighborhood context. This issue is further compounded by prior evidence that LLMs often struggle with spatial reasoning tasks, including distance estimation and geographically biased prediction. To address these limitations, we propose our framework, a multi-agent LLM framework that decomposes next-POI prediction into three stages: Firstly, a Pattern Extraction Agent that captures temporal and categorical mobility patterns from trajectory history; Secondly, a Spatial Reasoning Agent that structures candidate activity choices by combining behavioral preferences with real-world spatial constraints, including geographic distance, road network distance, and neighborhood affiliation; and Thirdly, a Decision Synthesis Agent that integrates behavioral patterns and spatial reasoning for final prediction. Experiments on the NYC benchmark dataset with two LLM backbones show improvements over baseline methods, with up to 493% Hit@1 improvement and 37% relative improvement in Hit@5. Ablations show that combining neighborhood affiliation with distance-based features generally outperforms distance-only settings, and that the Spatial Reasoning Agent plays a crucial role in final prediction by integrating behavioral preferences with real-world spatial constraints, especially for smaller models. Overall, the results highlight the importance of spatial reasoning in mobility prediction. Accurate next-POI prediction requires combining behavioral patterns with explicit real-world spatial constraints, and multi-agent decomposition provides an effective structure for organizing these forms of context.

cs.SI↗