arXiv subjects

Christian Benz

Publications and source records attributed to Christian Benz.

5 recordsLinked to original sources

arXiv2026

P-JEPA: Procedural Video Representation Learning via Joint Embedding Predictive Architecture

The increasing maturity of embodied AI platforms has driven a growing interest in procedural video representation learning to support intelligent assistance systems for complex, multi-step tasks. Leveraging large-scale latent predictive training, video foundation models capture video dynamics, enabling downstream tasks such as activity understanding, spatiotemporal localization, and predictive control. However, procedural videos include actions with long-range dependencies that these models do not support, due to the quadratic complexity of self-attention. Distinct actions, for example, may be visually similar despite appearing at different points in the procedure, such as turning the stove on versus off. Here, we propose a backbone-agnostic approach that learns long-duration video representations by reducing the problem to a dense, frame-aligned action space and predicting pooled masked latent vectors. This approach allows our Procedural Joint Embedding Predictive Architecture (P-JEPA) to ingest videos over 30 minutes long, enabling effective long-form understanding of procedural steps. We evaluate P-JEPA using features extracted with VJEPA2.1, TSM, and I3D over the EgoExo4D, EgoProceL, and Assembly101 datasets, finding that it consistently improves linear separability, streaming inference, and temporal action segmentation performance, achieving state-of-the-art results on EgoExo4D fine-grained action classification while using an order of magnitude fewer parameters than LLM-based methods and running in real time.

cs.CV↗

arXiv2026

OR-Action: Multi-Role Video Understanding with Fine-Grained Actions

Fine-grained understanding of operating room (OR) activity could enable workflow-aware assistance, yet remains difficult due to clutter, occlusions, and limited sensing. The prevailing approach to model this environment is scene graphs as an interpretable representation of OR interactions. Converting their frame-wise relational predictions into temporally extended, fine-grained actions however, is challenging without explicit temporal modeling. To enable a principled temporal evaluation of current OR understanding methods, we introduce the first action-centric benchmark built on a publicly available ego-exocentric OR dataset by defining a fine-grained, multi-role action taxonomy and generating dense action segments via distillation from ground-truth scene graph state changes. Experiments on this benchmark show that current scene graph prediction methods struggle to model temporal structure, even when adding explicit modeling through Graph Neural Networks. We therefore introduce a vision-only temporal model that outperforms graph-based methods significantly when using all available egocentric video as input. Building on this model we also introduce a novel multi- to single-view feature alignment strategy that improves single-view performance on multi-role action recognition, mitigating the need for extensive egocentric video capture. Benchmark and code will be released upon acceptance.

cs.CV↗

arXiv2024

ENSTRECT: A Stage-based Approach to 2.5D Structural Damage Detection

To effectively assess structural damage, it is essential to localize the instances of damage in the physical world of a civil structure. ENSTRECT is a stage-based approach designed to accomplish 2.5D structural damage detection. The method requires an image collection, the relative orientation, and a point cloud. Using these inputs, surface damages are segmented at the image level and then mapped into the point cloud space, resulting in a segmented point cloud. To enable further quantitative analyses, the segmented point cloud is transformed into measurable damage instances: cracks are extracted by contracting the clustered point cloud into a corresponding medial axis. For areal damages, such as spalling and corrosion, a procedure is proposed to compute the bounding polygon based on PCA and alpha shapes. With a localization tolerance of 4cm, ENSTRECT can achieve IoUs of over 90% for cracks, 82% for corrosion, and 41% for spalling. Detection at the instance level yields an AP50 of about 45% (cracks, spalling) and 56% (corrosion).

cs.CV↗

arXiv2024

Freeform terahertz structures fabricated by multi-photon lithography and metal coating

Direct-write multi-photon laser lithography (MPL) combines highest resolution on the nanoscale with essentially unlimited 3D design freedom. Over the previous years, the groundbreaking potential of this technique has been demonstrated in various application fields, including micromechanics, material sciences, microfluidics, life sciences as well as photonics, where in-situ printed optical coupling elements offer new perspectives for package-level system integration. However, millimeter-wave (mmW) and terahertz (THz) devices could not yet leverage the unique strengths of MPL, even though the underlying devices and structures could also greatly benefit from 3D freeform microfabrication. One of the key challenges in this context is the fact that functional mmW and THz structures require materials with high electrical conductivity and low dielectric losses, which are not amenable to structuring by multi-photon polymerization. In this work, we introduce and experimentally demonstrate a novel approach that allows to leverage MPL for fabricating high-performance mmW and THz structures with hitherto unachieved functionalities. Our concept exploits in-situ printed polymer templates that are selectively coated through highly directive metal deposition techniques in combination with precisely aligned 3D-printed shadowing structures. The resulting metal-coated freeform structures offer high surface quality in combination with low dielectric losses and conductivities comparable to bulk material values, while lending themselves to fabrication on planar mmW/THz circuits. We experimentally show the viability of our concept by demonstrating a series of functional THz structures such as THz interconnects, probe tips, and suspended antennas. We believe that our approach offers disruptive potential in the field of mmW and THz technology and may unlock an entirely new realm of laser-based 3D manufacturing.

physics.optics↗

arXiv2023

Drawing the Same Bounding Box Twice? Coping Noisy Annotations in Object Detection with Repeated Labels

The reliability of supervised machine learning systems depends on the accuracy and availability of ground truth labels. However, the process of human annotation, being prone to error, introduces the potential for noisy labels, which can impede the practicality of these systems. While training with noisy labels is a significant consideration, the reliability of test data is also crucial to ascertain the dependability of the results. A common approach to addressing this issue is repeated labeling, where multiple annotators label the same example, and their labels are combined to provide a better estimate of the true label. In this paper, we propose a novel localization algorithm that adapts well-established ground truth estimation methods for object detection and instance segmentation tasks. The key innovation of our method lies in its ability to transform combined localization and classification tasks into classification-only problems, thus enabling the application of techniques such as Expectation-Maximization (EM) or Majority Voting (MJV). Although our main focus is the aggregation of unique ground truth for test data, our algorithm also shows superior performance during training on the TexBiG dataset, surpassing both noisy label training and label aggregation using Weighted Boxes Fusion (WBF). Our experiments indicate that the benefits of repeated labels emerge under specific dataset and annotation configurations. The key factors appear to be (1) dataset complexity, the (2) annotator consistency, and (3) the given annotation budget constraints.

cs.CV↗