arXiv subjects

Jiawei Wang

Publications and source records attributed to Jiawei Wang.

At least 127 records · Page 7Linked to original sources

arXiv2024

Uniform Anisotropic Regularity and Low Mach Number Limit of Non-isentropic Ideal MHD Equations with a Perfectly Conducting Boundary

We prove the low Mach number limit of non-isentropic ideal magnetohydrodynamic (MHD) equations with general initial data in the half-space whose boundary satisfies the perfectly conducting wall condition. By observing a special structure contributed by Lorentz force in vorticity analysis, we establish uniform estimates in suitable anisotropic Sobolev spaces with weights of Mach number determined by the number of material derivatives. We also observe that the entropy has the enhanced regularity in the direction of the magnetic field. These two observations help us get rid of the loss of derivatives and weights of Mach number in vorticity analysis caused by the simultaneous appearance of entropy, general initial data and the magnetic field, which is one of the major difficulties that do not appear in Euler equations or the isentropic problems. By utilizing the technique of Alinhac good unknowns, the anti-symmetric structure is preserved in the tangential estimates for the system differenetiated by high-order material derivatives.

math.AP↗

arXiv2024

DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding

We present DeepSeek-VL2, an advanced series of large Mixture-of-Experts (MoE) Vision-Language Models that significantly improves upon its predecessor, DeepSeek-VL, through two key major upgrades. For the vision component, we incorporate a dynamic tiling vision encoding strategy designed for processing high-resolution images with different aspect ratios. For the language component, we leverage DeepSeekMoE models with the Multi-head Latent Attention mechanism, which compresses Key-Value cache into latent vectors, to enable efficient inference and high throughput. Trained on an improved vision-language dataset, DeepSeek-VL2 demonstrates superior capabilities across various tasks, including but not limited to visual question answering, optical character recognition, document/table/chart understanding, and visual grounding. Our model series is composed of three variants: DeepSeek-VL2-Tiny, DeepSeek-VL2-Small and DeepSeek-VL2, with 1.0B, 2.8B and 4.5B activated parameters respectively. DeepSeek-VL2 achieves competitive or state-of-the-art performance with similar or fewer activated parameters compared to existing open-source dense and MoE-based models. Codes and pre-trained models are publicly accessible at https://github.com/deepseek-ai/DeepSeek-VL2.

cs.CV↗

arXiv2024

Fixing Security Vulnerabilities with AI in OSS-Fuzz

Critical open source software systems undergo significant validation in the form of lengthy fuzz campaigns. The fuzz campaigns typically conduct a biased random search over the domain of program inputs, to find inputs which crash the software system. Such fuzzing is useful to enhance the security of software systems in general since even closed source software may use open source components. Hence testing open source software is of paramount importance. Currently OSS-Fuzz is the most significant and widely used infrastructure for continuous validation of open source systems. Unfortunately even though OSS-Fuzz has identified more than 10,000 vulnerabilities across 1000 or more software projects, the detected vulnerabilities may remain unpatched, as vulnerability fixing is often manual in practice. In this work, we rely on the recent progress in Large Language Model (LLM) agents for autonomous program improvement including bug fixing. We customise the well-known AutoCodeRover agent for fixing security vulnerabilities. This is because LLM agents like AutoCodeRover fix bugs from issue descriptions via code search. Instead for security patching, we rely on the test execution of the exploit input to extract code elements relevant to the fix. Our experience with OSS-Fuzz vulnerability data shows that LLM agent autonomy is useful for successful security patching, as opposed to approaches like Agentless where the control flow is fixed. More importantly our findings show that we cannot measure quality of patches by code similarity of the patch with reference codes (as in CodeBLEU scores used in VulMaster), since patches with high CodeBLEU scores still fail to pass given the given exploit input. Our findings indicate that security patch correctness needs to consider dynamic attributes like test executions as opposed to relying of standard text/code similarity metrics.

cs.CR↗

arXiv2024

UAV Path Planning for Object Observation with Quality Constraints: A Dynamic Programming Approach

This paper addresses a UAV path planning task that seeks to observe a set of objects while satisfying the observation quality constraint. A dynamic programming algorithm is proposed that enables the UAV to observe the target objects with the shortest path while subjecting to the observation quality constraint. The objects have their own facing direction and restricted observation range. With an observing order, the algorithm achieves $(1+ε)$-approximation ratio in theory and runs in polynomial time. The extensive results demonstrate that the algorithm produces near-optimal solutions, the effectiveness of which is also tested and proved in the Airsim simulator, a realistic virtual environment.

cs.RO↗

arXiv2024

Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generation

This paper introduces a novel approach using Large Language Models (LLMs) integrated into an agent framework for flexible and effective personal mobility generation. LLMs overcome the limitations of previous models by effectively processing semantic data and offering versatility in modeling various tasks. Our approach addresses three research questions: aligning LLMs with real-world urban mobility data, developing reliable activity generation strategies, and exploring LLM applications in urban mobility. The key technical contribution is a novel LLM agent framework that accounts for individual activity patterns and motivations, including a self-consistency approach to align LLMs with real-world activity data and a retrieval-augmented strategy for interpretable activity generation. We evaluate our LLM agent framework and compare it with state-of-the-art personal mobility generation approaches, demonstrating the effectiveness of our approach and its potential applications in urban mobility. Overall, this study marks the pioneering work of designing an LLM agent framework for activity generation based on real-world human activity data, offering a promising tool for urban mobility analysis.

cs.AI↗

arXiv2024

Decentralized Robust Data-driven Predictive Control for Smoothing Mixed Traffic Flow

In a mixed traffic with connected automated vehicles (CAVs) and human-driven vehicles (HDVs) coexisting, data-driven predictive control of CAVs promises system-wide traffic performance improvements. Yet, most existing approaches focus on a centralized setup, which is not computationally scalable while failing to protect data privacy. The robustness against unknown disturbances has not been well addressed either, causing safety concerns. In this paper, we propose a decentralized robust DeeP-LCC (Data-EnablEd Predictive Leading Cruise Control) approach for CAVs to smooth mixed traffic flow. In particular, each CAV computes its control input based on locally available data from its involved subsystem. Meanwhile, the interaction between neighboring subsystems is modeled as a bounded disturbance, for which appropriate estimation methods are proposed. Then, we formulate a robust optimization problem and present its tractable computational solutions. Compared with the centralized formulation, our method greatly reduces computation burden with better safety performance, while naturally preserving data privacy. Extensive traffic simulations validate its wave-dampening ability, safety performance, and computational benefits.

math.OC↗

arXiv2024

Sub-100 Hz Intrinsic Linewidth 852 nm Silicon Nitride External Cavity Laser

We demonstrate an external cavity laser with intrinsic linewidth below 100 Hz around an operating wavelength of 852 nm, selected for its relevance to laser cooling and manipulation of cesium atoms. This system achieves a maximum CW output power of 24 mW, wavelength tunability over 15 nm, and a side-mode suppression ratio exceeding 50 dB. This performance level is facilitated by careful design of a low-loss integrated silicon nitride photonic circuit serving as the external cavity combined with commercially available semiconductor gain chips. This approach demonstrates the feasibility of compact integrated lasers with sub-kHz linewidth centering on the needs of emerging sensor concepts based on ultracold atoms and can be further extended to shorter wavelengths via selection of suitable semiconductor gain media.

physics.optics↗

arXiv2024

Robust Data-EnablEd Predictive Leading Cruise Control via Reachability Analysis

Data-driven predictive control promises model-free wave-dampening strategies for Connected and Autonomous Vehicles (CAVs) in mixed traffic flow. However, its performance relies on data quality, which suffers from unknown noise and disturbances. This paper introduces a Robust Data-EnablEd Predictive Leading Cruise Control (RDeeP-LCC) method based on reachability analysis, aiming to achieve safe and optimal CAV control under bounded process noise and external disturbances. Precisely, the matrix zonotope set technique and Willems' Fundamental Lemma are employed to derive the over-approximated system dynamics directly from data, and a data-driven feedback control technique is utilized to obtain an additional feedback input for stability. We decouple the mixed platoon into an error system and a nominal system, where the error system provides data-driven reachability sets for the enhanced safety constraints in the nominal system. Finally, a data-driven predictive control framework is formulated in a tube-based control manner for robustness guarantees. Nonlinear simulations with noise-corrupted data demonstrate that the proposed method outperforms baseline methods in mitigating traffic waves.

eess.SY↗

arXiv2024

Tarsier: Recipes for Training and Evaluating Large Video Description Models

Generating fine-grained video descriptions is a fundamental challenge in video understanding. In this work, we introduce Tarsier, a family of large-scale video-language models designed to generate high-quality video descriptions. Tarsier employs CLIP-ViT to encode frames separately and then uses an LLM to model temporal relationships. Despite its simple architecture, we demonstrate that with a meticulously designed two-stage training procedure, the Tarsier models exhibit substantially stronger video description capabilities than any existing open-source model, showing a $+51.4\%$ advantage in human side-by-side evaluation over the strongest model. Additionally, they are comparable to state-of-the-art proprietary models, with a $+12.3\%$ advantage against GPT-4V and a $-6.7\%$ disadvantage against Gemini 1.5 Pro. When upgraded to Tarsier2 by building upon SigLIP and Qwen2-7B, it further improves significantly with a $+4.8\%$ advantage against GPT-4o. Besides video description, Tarsier proves to be a versatile generalist model, achieving new state-of-the-art results across nine public benchmarks, including multi-choice VQA, open-ended VQA, and zero-shot video captioning. Our second contribution is the introduction of a new benchmark -- DREAM-1K (https://tarsier-vlm.github.io/) for evaluating video description models, consisting of a new challenging dataset featuring videos from diverse sources and varying complexity, along with an automatic method specifically designed to assess the quality of fine-grained video descriptions. We make our models and evaluation benchmark publicly available at https://github.com/bytedance/tarsier.

cs.CV↗

arXiv2024

Static Analysis Driven Enhancements for Comprehension in Machine Learning Notebooks

Jupyter notebooks enable developers to interleave code snippets with rich-text and in-line visualizations. Data scientists use Jupyter notebook as the de-facto standard for creating and sharing machine-learning based solutions, primarily written in Python. Recent studies have demonstrated, however, that a large portion of Jupyter notebooks available on public platforms are undocumented and lacks a narrative structure. This reduces the readability of these notebooks. To address this shortcoming, this paper presents HeaderGen, a novel tool-based approach that automatically annotates code cells with categorical markdown headers based on a taxonomy of ML operations, and classifies and displays function calls according to this taxonomy. For this functionality to be realized, HeaderGen enhances an existing call graph analysis in PyCG. To improve precision, HeaderGen extends PyCG's analysis with support for handling external library code and flow-sensitivity. The former is realized by facilitating the resolution of function return-types. The evaluation on 15 real-world Jupyter notebooks from Kaggle shows that HeaderGen's underlying call graph analysis yields high accuracy (95.6% precision and 95.3% recall). This is because HeaderGen can resolve return-types of external libraries where existing type inference tools such as pytype (by Google), pyright (by Microsoft), and Jedi fall short. The header generation has a precision of 85.7% and a recall rate of 92.8%. In a user study, HeaderGen helps participants finish comprehension and navigation tasks faster. To further evaluate the type inference capability of tools, we introduce TypeEvalPy, a framework for evaluating type inference tools with a micro-benchmark containing 154 code snippets and 845 type annotations. Our comparative analysis on four tools revealed that HeaderGen outperforms other tools in exact matches with the ground truth.

cs.SE↗

arXiv2024

Nonlinear Eigen-approach ADMM for Sparse Optimization on Stiefel Manifold

With the growing interest and applications in machine learning and data science, finding an efficient method to sparse analysis the high-dimensional data and optimizing a dimension reduction model to extract lower dimensional features has becoming more and more important. Orthogonal constraints (Stiefel manifold) is a commonly met constraint in these applications, and the sparsity is usually enforced through the element-wise L1 norm. Many applications can be found on optimization over Stiefel manifold within the area of physics and machine learning. In this paper, we propose a novel idea by tackling the Stiefel manifold through an nonlinear eigen-approach by first using ADMM to split the problem into smooth optimization over manifold and convex non-smooth optimization, and then transforming the former into the form of nonlinear eigenvalue problem with eigenvector dependency (NEPv) which is solved by self-consistent field (SCF) iteration, and the latter can be found to have an closed-form solution through proximal gradient. Compared with existing methods, our proposed algorithm takes the advantage of specific structure of the objective function, and has efficient convergence results under mild assumptions.

math.OC↗

arXiv2024

DLAFormer: An End-to-End Transformer For Document Layout Analysis

Document layout analysis (DLA) is crucial for understanding the physical layout and logical structure of documents, serving information retrieval, document summarization, knowledge extraction, etc. However, previous studies have typically used separate models to address individual sub-tasks within DLA, including table/figure detection, text region detection, logical role classification, and reading order prediction. In this work, we propose an end-to-end transformer-based approach for document layout analysis, called DLAFormer, which integrates all these sub-tasks into a single model. To achieve this, we treat various DLA sub-tasks (such as text region detection, logical role classification, and reading order prediction) as relation prediction problems and consolidate these relation prediction labels into a unified label space, allowing a unified relation prediction module to handle multiple tasks concurrently. Additionally, we introduce a novel set of type-wise queries to enhance the physical meaning of content queries in DETR. Moreover, we adopt a coarse-to-fine strategy to accurately identify graphical page objects. Experimental results demonstrate that our proposed DLAFormer outperforms previous approaches that employ multi-branch or multi-stage architectures for multiple tasks on two document layout analysis benchmarks, DocLayNet and Comp-HRDoc.

cs.CV↗

arXiv2024

AMCEN: An Attention Masking-based Contrastive Event Network for Two-stage Temporal Knowledge Graph Reasoning

Temporal knowledge graphs (TKGs) can effectively model the ever-evolving nature of real-world knowledge, and their completeness and enhancement can be achieved by reasoning new events from existing ones. However, reasoning accuracy is adversely impacted due to an imbalance between new and recurring events in the datasets. To achieve more accurate TKG reasoning, we propose an attention masking-based contrastive event network (AMCEN) with local-global temporal patterns for the two-stage prediction of future events. In the network, historical and non-historical attention mask vectors are designed to control the attention bias towards historical and non-historical entities, acting as the key to alleviating the imbalance. A local-global message-passing module is proposed to comprehensively consider and capture multi-hop structural dependencies and local-global temporal evolution for the in-depth exploration of latent impact factors of different event types. A contrastive event classifier is used to classify events more accurately by incorporating local-global temporal patterns into contrastive learning. Therefore, AMCEN refines the prediction scope with the results of the contrastive event classification, followed by utilizing attention masking-based decoders to finalize the specific outcomes. The results of our experiments on four benchmark datasets highlight the superiority of AMCEN. Especially, the considerable improvements in Hits@1 prove that AMCEN can make more precise predictions about future occurrences.

cs.LG↗

arXiv2024

Scalable single-microring hybrid III-V/Si lasers for emerging narrow-linewidth applications

Silicon photonics, compatible with large-scale silicon manufacturing, is a disruptive photonic platform that has indicated significant implications in industry and research areas (e.g., quantum, neuromorphic computing, LiDAR). Cutting-edge applications such as high-capacity coherent optical communication and heterodyne LiDAR have escalated the demand for integrated narrow-linewidth laser sources. To that effect, this work seeks to address this requirement through the development of a high-performance hybrid III-V/silicon laser. The developed integrated laser, utilizes a single microring resonator (MRR), demonstrating single-mode operation with a side mode suppression ratio (SMSR) exceeding 40 dB, with laser output power as high as 16.4 mW. Moving away from current hybrid/heterogeneous laser architectures that necessitate multiple complex control, the developed laser architecture requires only two control parameters. Importantly, this serves to streamline industrial adoption by reducing the complexity involved in characterizing these lasers, at-scale. Through the succinct structure and control framework, a narrow laser linewidth of 2.79 kHz and low relative intensity noise (RIN) of -135 dB/Hz are achieved. Furthermore, optical data transmission at 12.5 Gb/s is demonstrated where a signal-to-noise ratio (SNR) of 10 dB is measured.

physics.optics↗

arXiv2024

Detect-Order-Construct: A Tree Construction based Approach for Hierarchical Document Structure Analysis

Document structure analysis (aka document layout analysis) is crucial for understanding the physical layout and logical structure of documents, with applications in information retrieval, document summarization, knowledge extraction, etc. In this paper, we concentrate on Hierarchical Document Structure Analysis (HDSA) to explore hierarchical relationships within structured documents created using authoring software employing hierarchical schemas, such as LaTeX, Microsoft Word, and HTML. To comprehensively analyze hierarchical document structures, we propose a tree construction based approach that addresses multiple subtasks concurrently, including page object detection (Detect), reading order prediction of identified objects (Order), and the construction of intended hierarchical structure (Construct). We present an effective end-to-end solution based on this framework to demonstrate its performance. To assess our approach, we develop a comprehensive benchmark called Comp-HRDoc, which evaluates the above subtasks simultaneously. Our end-to-end system achieves state-of-the-art performance on two large-scale document layout analysis datasets (PubLayNet and DocLayNet), a high-quality hierarchical document structure reconstruction dataset (HRDoc), and our Comp-HRDoc benchmark. The Comp-HRDoc benchmark will be released to facilitate further research in this field.

cs.CV↗

arXiv2024

Dynamic Phase Enabled Topological Mode Steering in Composite Su-Schrieffer-Heeger Waveguide Arrays

Topological boundary states localize at interfaces whenever the interface implies a change of the associated topological invariant encoded in the geometric phase. The generically present dynamic phase, however, which is energy and time dependent, has been known to be non-universal, and hence not to intertwine with any topological geometric phase. Using the example of topological zero modes in composite Su-Schrieffer-Heeger (c-SSH) waveguide arrays with a central defect, we report on the selective excitation and transition of topological boundary mode based on dynamic phase-steered interferences. Our work thus provides a new knob for the control and manipulation of topological states in composite photonic devices, indicating promising applications where topological modes and their bandwidth can be jointly controlled by the dynamic phase, geometric phase, and wavelength in on-chip topological devices.

physics.optics↗

arXiv2024

ContextSeg: Sketch Semantic Segmentation by Querying the Context with Attention

Sketch semantic segmentation is a well-explored and pivotal problem in computer vision involving the assignment of pre-defined part labels to individual strokes. This paper presents ContextSeg - a simple yet highly effective approach to tackling this problem with two stages. In the first stage, to better encode the shape and positional information of strokes, we propose to predict an extra dense distance field in an autoencoder network to reinforce structural information learning. In the second stage, we treat an entire stroke as a single entity and label a group of strokes within the same semantic part using an auto-regressive Transformer with the default attention mechanism. By group-based labeling, our method can fully leverage the context information when making decisions for the remaining groups of strokes. Our method achieves the best segmentation accuracy compared with state-of-the-art approaches on two representative datasets and has been extensively evaluated demonstrating its superior performance. Additionally, we offer insights into solving part imbalance in training data and the preliminary experiment on cross-category training, which can inspire future research in this field.

cs.CV↗

arXiv2024

EM-TTS: Efficiently Trained Low-Resource Mongolian Lightweight Text-to-Speech

Recently, deep learning-based Text-to-Speech (TTS) systems have achieved high-quality speech synthesis results. Recurrent neural networks have become a standard modeling technique for sequential data in TTS systems and are widely used. However, training a TTS model which includes RNN components requires powerful GPU performance and takes a long time. In contrast, CNN-based sequence synthesis techniques can significantly reduce the parameters and training time of a TTS model while guaranteeing a certain performance due to their high parallelism, which alleviate these economic costs of training. In this paper, we propose a lightweight TTS system based on deep convolutional neural networks, which is a two-stage training end-to-end TTS model and does not employ any recurrent units. Our model consists of two stages: Text2Spectrum and SSRN. The former is used to encode phonemes into a coarse mel spectrogram and the latter is used to synthesize the complete spectrum from the coarse mel spectrogram. Meanwhile, we improve the robustness of our model by a series of data augmentations, such as noise suppression, time warping, frequency masking and time masking, for solving the low resource mongolian problem. Experiments show that our model can reduce the training time and parameters while ensuring the quality and naturalness of the synthesized speech compared to using mainstream TTS models. Our method uses NCMMSC2022-MTTSC Challenge dataset for validation, which significantly reduces training time while maintaining a certain accuracy.

cs.SD↗