arXiv subjects

Guenter Hesse

Publications and source records attributed to Guenter Hesse.

9 recordsLinked to original sources

arXiv2021

ESPBench: The Enterprise Stream Processing Benchmark

Growing data volumes and velocities in fields such as Industry 4.0 or the Internet of Things have led to the increased popularity of data stream processing systems. Enterprises can leverage these developments by enriching their core business data and analyses with up-to-date streaming data. Comparing streaming architectures for these complex use cases is challenging, as existing benchmarks do not cover them. ESPBench is a new enterprise stream processing benchmark that fills this gap. We present its architecture, the benchmarking process, and the query workload. We employ ESPBench on three state-of-the-art stream processing systems, Apache Spark, Apache Flink, and Hazelcast Jet, using provided query implementations developed with Apache Beam. Our results highlight the need for the provided ESPBench toolkit that supports benchmark execution, as it enables query result validation and objective latency measures.

cs.PF↗

arXiv2020

Mining for Process Improvements: Analyzing Software Repositories in Agile Retrospectives

Software Repositories contain knowledge on how software engineering teams work, communicate, and collaborate. It can be used to develop a data-informed view of a team's development process, which in turn can be employed for process improvement initiatives. In modern, Agile development methods, process improvement takes place in Retrospective meetings, in which the last development iteration is discussed. However, previously proposed activities that take place in these meetings often do not rely on project data, instead depending solely on the perceptions of team members. We propose new Retrospective activities, based on mining the software repositories of individual teams, to complement existing approaches with more objective, data-informed process views.

cs.SE↗

arXiv2020

How Fast Can We Insert? An Empirical Performance Evaluation of Apache Kafka

Message brokers see widespread adoption in modern IT landscapes, with Apache Kafka being one of the most employed platforms. These systems feature well-defined APIs for use and configuration and present flexible solutions for various data storage scenarios. Their ability to scale horizontally enables users to adapt to growing data volumes and changing environments. However, one of the main challenges concerning message brokers is the danger of them becoming a bottleneck within an IT architecture. To prevent this, knowledge about the amount of data a message broker using a specific configuration can handle needs to be available. In this paper, we propose a monitoring architecture for message brokers and similar Java Virtual Machine-based systems. We present a comprehensive performance analysis of the popular Apache Kafka platform using our approach. As part of the benchmark, we study selected data ingestion scenarios with respect to their maximum data ingestion rates. The results show that we can achieve an ingestion rate of about 420,000 messages/second on the used commodity hardware and with the developed data sender tool.

cs.PF↗

arXiv2019

Towards Using Data to Inform Decisions in Agile Software Development: Views of Available Data

Software development comprises complex tasks which are performed by humans. It involves problem solving, domain understanding and communication skills as well as knowledge of a broad variety of technologies, architectures, and solution approaches. As such, software development projects include many situations where crucial decisions must be made. Making the appropriate organizational or technical choices for a given software team building a product can make the difference between project success or failure. Software development methods have introduced frameworks and sets of best practices for certain contexts, providing practitioners with established guidelines for these important choices. Current Agile methods employed in modern software development have highlighted the importance of the human factors in software development. These methods rely on short feedback loops and the self-organization of teams to enable collaborative decision making. While Agile methods stress the importance of empirical process control, i.e. relying on data to make decisions, they do not prescribe in detail how this goal should be achieved. In this paper, we describe the types and abstraction levels of data and decisions within modern software development teams and identify the benefits that usage of this data enables. We argue that the principles of data-driven decision making are highly applicable, yet underused, in modern Agile software development.

cs.SE↗

arXiv2019

Application of Data Stream Processing Technologies in Industry 4.0 -- What is Missing?

Industry 4.0 is becoming more and more important for manufacturers as the developments in the area of Internet of Things advance. Another technology gaining more attention is data stream processing systems. Although such streaming frameworks seem to be a natural fit for Industry 4.0 scenarios, their application in this context is still low. The contributions in this paper are threefold. Firstly, we present industry findings that we derived from site inspections with a focus on Industry 4.0. Moreover, our view on Industry 4.0 and important related aspects is elaborated. As a third contribution, we illustrate our opinion on why data stream processing technologies could act as an enabler for Industry 4.0 and point out possible obstacles on this way.

cs.DB↗

arXiv2019

Quantitative Impact Evaluation of an Abstraction Layer for Data Stream Processing Systems

With the demand to process ever-growing data volumes, a variety of new data stream processing frameworks have been developed. Moving an implementation from one such system to another, e.g., for performance reasons, requires adapting existing applications to new interfaces. Apache Beam addresses these high substitution costs by providing an abstraction layer that enables executing programs on any of the supported streaming frameworks. In this paper, we present a novel benchmark architecture for comparing the performance impact of using Apache Beam on three streaming frameworks: Apache Spark Streaming, Apache Flink, and Apache Apex. We find significant performance penalties when using Apache Beam for application development in the surveyed systems. Overall, usage of Apache Beam for the examined streaming applications caused a high variance of query execution times with a slowdown of up to a factor of 58 compared to queries developed without the abstraction layer. All developed benchmark artifacts are publicly available to ensure reproducible results.

cs.PF↗

arXiv2019

Adding Value by Combining Business and Sensor Data: An Industry 4.0 Use Case

Industry 4.0 and the Internet of Things are recent developments that have lead to the creation of new kinds of manufacturing data. Linking this new kind of sensor data to traditional business information is crucial for enterprises to take advantage of the data's full potential. In this paper, we present a demo which allows experiencing this data integration, both vertically between technical and business contexts and horizontally along the value chain. The tool simulates a manufacturing company, continuously producing both business and sensor data, and supports issuing ad-hoc queries that answer specific questions related to the business. In order to adapt to different environments, users can configure sensor characteristics to their needs.

cs.DB↗

arXiv2019

An Additional Set of (Automated) Eyes: Chatbots for Agile Retrospectives

Recent advances in natural-language processing and data analysis allow software bots to become virtual team members, providing an additional set of automated eyes and additional perspectives for informing and supporting teamwork. In this paper, we propose employing chatbots in the domain of software development with a focus on supporting analyses and measurements of teams' project data. The software project artifacts produced by agile teams during regular development activities, e.g. commits in a version control system, represent detailed information on how a team works and collaborates. Analyses of this data are especially relevant for agile retrospective meetings, where adaptations and improvements to the executed development process are discussed. Development teams can use these measurements to track the progress of identified improvement actions over development iterations. Chatbots provide a convenient user interface for interacting with the outcomes of retrospectives and the associated measurements in a chat-based channel that is already being employed by team members.

cs.SE↗

arXiv2018

Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching Efforts

This Innovative Practice Full Paper presents an approach of using software development artifacts to gauge student behavior and the effectiveness of changes to curriculum design. There is an ongoing need to adapt university courses to changing requirements and shifts in industry. As an educator it is therefore vital to have access to methods, with which to ascertain the effects of curriculum design changes. In this paper, we present our approach of analyzing software repositories in order to gauge student behavior during project work. We evaluate this approach in a case study of a university undergraduate software development course teaching agile development methodologies. Surveys revealed positive attitudes towards the course and the change of employed development methodology from Scrum to Kanban. However, surveys were not usable to ascertain the degree to which students had adapted their workflows and whether they had done so in accordance with course goals. Therefore, we analyzed students' software repository data, which represents information that can be collected by educators to reveal insights into learning successes and detailed student behavior. We analyze the software repositories created during the last five courses, and evaluate differences in workflows between Kanban and Scrum usage.

cs.SE↗