arXiv subjects

Haojian Jin

Publications and source records attributed to Haojian Jin.

25 records · Page 2Linked to original sources

arXiv2024

Moderator: Moderating Text-to-Image Diffusion Models through Fine-grained Context-based Policies

We present Moderator, a policy-based model management system that allows administrators to specify fine-grained content moderation policies and modify the weights of a text-to-image (TTI) model to make it significantly more challenging for users to produce images that violate the policies. In contrast to existing general-purpose model editing techniques, which unlearn concepts without considering the associated contexts, Moderator allows admins to specify what content should be moderated, under which context, how it should be moderated, and why moderation is necessary. Given a set of policies, Moderator first prompts the original model to generate images that need to be moderated, then uses these self-generated images to reverse fine-tune the model to compute task vectors for moderation and finally negates the original model with the task vectors to decrease its performance in generating moderated content. We evaluated Moderator with 14 participants to play the role of admins and found they could quickly learn and author policies to pass unit tests in approximately 2.29 policy iterations. Our experiment with 32 stable diffusion users suggested that Moderator can prevent 65% of users from generating moderated content under 15 attempts and require the remaining users an average of 8.3 times more attempts to generate undesired content.

cs.CR↗

arXiv2023

On the Feasibility of Reasoning about the Internal States of Blackbox IoT Devices Using Side-Channel Information

Internet of Things (IoT) devices are typically designed to function in a secure, closed environment, making it difficult for users to comprehend devices' behaviors. This paper shows that a user can leverage side-channel information to reason fine-grained internal states of black box IoT devices. The key enablers for our design are a multi-model sensing technique that fuses power consumption, network traffic, and radio emanations and an annotation interface that helps users form mental models of a black box IoT system. We built a prototype of our design and evaluated the prototype with open-source IoT devices and black-box commercial devices. Our experiments show a false positive rate of 1.44% for open-source IoT devices' state probing, and our participants take an average of 19.8 minutes to reason the internal states of black-box IoT devices.

cs.HC↗

arXiv2022

Peekaboo: A Hub-Based Approach to Enable Transparency in Data Processing within Smart Homes (Extended Technical Report)

We present Peekaboo, a new privacy-sensitive architecture for smart homes that leverages an in-home hub to pre-process and minimize outgoing data in a structured and enforceable manner before sending it to external cloud servers. Peekaboo's key innovations are (1) abstracting common data pre-processing functionality into a small and fixed set of chainable operators, and (2) requiring that developers explicitly declare desired data collection behaviors (e.g., data granularity, destinations, conditions) in an application manifest, which also specifies how the operators are chained together. Given a manifest, Peekaboo assembles and executes a pre-processing pipeline using operators pre-loaded on the hub. In doing so, developers can collect smart home data on a need-to-know basis; third-party auditors can verify data collection behaviors; and the hub itself can offer a number of centralized privacy features to users across apps and devices, without additional effort from app developers. We present the design and implementation of Peekaboo, along with an evaluation of its coverage of smart home scenarios, system performance, data minimization, and example built-in privacy features.

cs.CR↗

arXiv2021

Sensor as a Company: On Self-Sustaining IoT Commons

Beyond the "smart home" and "smart enterprise", the Internet of Things (IoT) revolution is creating "smart communities", where shared IoT devices collectively benefit a large number of residents, for transportation, healthcare, safety, and more. However, large-scale deployments of IoT-powered neighborhoods face two key socio-technical challenges: the significant upfront investment and the lack of information on local IoT needs. In this paper, we present SensorInc, a new IoT deployment paradigm that incentivizes residents to design and manage sensor deployment through sensor liquefaction. By turning shared sensors into liquid (i.e. tradeable) assets akin to company stock or bond, users can design and invest in promising IoT deployments and receive monetary rewards afterward. We present the detailed design of SensorInc and conduct two case studies (parking occupancy sensors and air pollution sensors) to study the self-sustainability and deployment challenges of such a paradigm.

cs.NI↗

arXiv2021

The Design of the User Interfaces for Privacy Enhancements for Android

We present the design and design rationale for the user interfaces for Privacy Enhancements for Android (PE for Android). These UIs are built around two core ideas, namely that developers should explicitly declare the purpose of why sensitive data is being used, and these permission-purpose pairs should be split by first party and third party uses. We also present a taxonomy of purposes and ways of how these ideas can be deployed in the existing Android ecosystem.

cs.CR↗

arXiv2019

Enhancing Email Functionality using Late Bound Content

Email is one of the most successful computer applications yet devised. Communication features in email, however, have remained relatively static in years. We investigate one way of expanding email functionality without modifying the existing email infrastructure. We introduce email late bound content, a simple and generalizable technique that defers message content binding through image lazy-loading. Parts of an email are converted into external images embedded in HTML code snippets, making it so that email clients will defer the image download (i.e. content binding) until the moment users open the email. This late bound content allows email senders and third party services to update delivered emails. To illustrate the utilities of late bound content, we present four new example features and discuss the tradeoffs of email content late binding.

cs.HC↗

arXiv2017

ElasticPlay: Interactive Video Summarization with Dynamic Time Budgets

Video consumption is being shifted from sit-and-watch to selective skimming. Existing video player interfaces, however, only provide indirect manipulation to support this emerging behavior. Video summarization alleviates this issue to some extent, shortening a video based on the desired length of a summary as an input variable. But an optimal length of a summarized video is often not available in advance. Moreover, the user cannot edit the summary once it is produced, limiting its practical applications. We argue that video summarization should be an interactive, mixed-initiative process in which users have control over the summarization procedure while algorithms help users achieve their goal via video understanding. In this paper, we introduce ElasticPlay, a mixed-initiative approach that combines an advanced video summarization technique with direct interface manipulation to help users control the video summarization process. Users can specify a time budget for the remaining content while watching a video; our system then immediately updates the playback plan using our proposed cut-and-forward algorithm, determining which parts to skip or to fast-forward. This interactive process allows users to fine-tune the summarization result with immediate feedback. We show that our system outperforms existing video summarization techniques on the TVSum50 dataset. We also report two lab studies (22 participants) and a Mechanical Turk deployment study (60 participants), and show that the participants responded favorably to ElasticPlay.

cs.MM↗