arXiv Science⌕ Search

arXiv subjects

Imranur Rahman

Publications and source records attributed to Imranur Rahman.

8 recordsLinked to original sources

How Quickly Do Development Teams Update Their Vulnerable Dependencies?

Industry practitioners are increasingly concerned with software that contains vulnerable versions of third-party dependencies that are included both directly and transitively. To address this problem, projects are encouraged to both (a)~quickly update to non-vulnerable versions of dependencies and (b)~be mindful of the update practices of the dependencies they choose to use. To this end, researchers have proposed metrics to measure the responsiveness of the development teams of the packages in keeping their dependencies updated: Mean-Time-To-Update (MTTU) and Mean-Time-To-Remediate (MTTR). While MTTU covers all dependencies, MTTR quantifies the time needed for a package to update its vulnerable dependencies. However, existing metrics fail to capture important nuances, such as considering floating versions and prioritizing recent updates, leading to inaccurate reflections of a development team's update practices. \textit{The goal of this study is to aid practitioners in understanding how quickly packages update their dependencies.} We propose two novel metrics, Mean-Time-To-Update for dependencies (MTTU) and Mean-Time-To-Remediate for vulnerable dependencies (MTTR), that overcome the limitations of existing metrics. We conduct an empirical study using $163,207$ packages in npm ($117,129$), PyPI ($42,777$), and Cargo ($3,301$) and characterize how the ecosystems differ in MTTU and MTTR, as well as what package characteristics influence MTTU and MTTR. We found that most packages have a relatively fast dependency update practice. We further study whether MTTU can be used as a proxy for MTTR when sufficient vulnerability data is not available. As we did not find enough statistical evidence for a strong proxy, our findings suggest that MTTU could only be partially used (may be used but with caution) as a proxy for MTTR when vulnerability data is not available.

cs.SE↗

What's in a Package? Getting Visibility Into Dependencies Using Security-Sensitive API Calls

The Log4Shell vulnerability occurred because the JNDI (Java Naming and Directory Interface) lookup feature was used, even though it was not needed for log4j's core functionality. This case illustrates why developers may want visibility into the sensitive capabilities available through a dependency. The presence of a call alone does not establish that the call is unnecessary or risky. Security-sensitive APIs provide access to security-sensitive resources, such as filesystem, network, or process-related resources. If developers know which types of sensitive resources a dependency can access, they can control their dependencies at a finer grain, and they can consider an alternative dependency when a dependency makes unnecessary or suspicious security-sensitive API calls. However, prior work has not provided a documented, repeatable procedure for deriving an ecosystem-specific security-sensitive API list, so developers lack systematic visibility into the sensitive capabilities of their dependencies. The goal of this study is to aid developers in understanding sensitive resources accessible to their dependencies by identifying security-sensitive APIs in packages through an empirical study of Java packages. In this study, we present a novel and repeatable methodology to construct a security-sensitive API list for an ecosystem, and we apply it to Java. We then compare the security-sensitive API calls of functionally similar Java packages. The number of security-sensitive API calls (call level) ranges from 0 to 368 in one capability and from 0 to 429 in total among packages with similar functionality. These packages also differ in capability breadth, the number of the 15 capabilities that a package reaches. For example, capability breadth ranges from 0 to 10 in the "Dependency Injection" category and from 0 to 13 in the "Web Frameworks" category.

cs.CR↗

Beyond Stars: MTTX Brings Developer Responsiveness to Dependency Decisions, from Dashboard to IDE to GitHub Actions

Modern software reuses open-source packages as dependencies, and developers often choose among candidate packages with cheap signals, such as GitHub stars and download counts. However, such signals are easy to fake, so developers need signals that represent "observed behavior," such as how quickly the developers of a package respond to new dependency releases and to security fixes. Prior work computed these responsiveness metrics, MTTU (Mean Time To Update) dependencies and MTTR (Mean Time To Remediate) vulnerable dependencies, but only with a research prototype for a fixed set of packages. Therefore, developers cannot easily benefit from them or integrate them into their workflow. We present MTTX, an open-source tool that computes MTTU and MTTR with accurate historical dependency resolution for any package and given time window, and brings them into different phases of developer workflow. MTTX resolves each version constraint of npm, PyPI, and Cargo packages as of a past date, with data from the native registries or deps.dev and vulnerability data from OSV. We validated MTTX with 328 automated tests, and for 90 packages from prior work, we manually verified its dependency resolution at each historical release point and its MTTU and MTTR computation. Developers can use MTTX in an interactive web dashboard, an IDE extension, and a GitHub Action, which are all available from the project homepage at https://imranur-rahman.github.io/dependency-update-metrics/. We envision that package registries (e.g., npm) and metadata aggregators and intelligence platforms (e.g., deps.dev) will display these values on the package page, alongside download counts and stars, to make them easier for developers to access. A demo video is at https://youtu.be/MZqjDnJ8bEQ.

cs.SE↗

CHRONO-RESOLUTION: A Dependency Resolution Dataset at Release Points for npm, PyPI, and crates.io Packages

Dependency resolution at a specified point in time in the past can provide insight into software evolution in software ecosystems and facilitate the design of dynamic metrics (e.g., dependency freshness, dependency update rhythm). However, dependency resolution at specified points in time is not possible in major software ecosystems due to a lack of support from package management tools. The goal of this paper is to aid practitioners and researchers in analyzing the state of the ecosystem dependency graph at release points using an enriched dataset with dependency resolution at release points for npm, PyPI, and crates.io packages. In this paper, we present a methodology to construct dependency resolution at release points of packages in software ecosystems, which we enrich with vulnerability data from OSV. We apply our methodology to construct CHRONO-RESOLUTION, a dataset of dependency resolution at release points for npm, PyPI, and crates.io packages, and release it for future research.

cs.SE↗

Which Is Better For Reducing Outdated and Vulnerable Dependencies: Pinning or Floating?

Developers consistently use version constraints to specify acceptable versions of the dependencies for their project. Pinning dependencies can reduce the likelihood of breaking changes, but comes with a cost of manually managing the replacement of outdated and vulnerable dependencies. On the other hand, floating can be used to automatically get bug fixes and security fixes, but comes with the risk of breaking changes. Security practitioners advocate pinning dependencies to prevent against software supply chain attacks, e.g., malicious package updates. However, since pinning is the tightest version constraint, pinning is the most likely to result in outdated dependencies. Nevertheless, how the likelihood of becoming outdated or vulnerable dependencies changes across version constraint types is unknown. The goal of this study is to aid developers in making an informed dependency version constraint choice by empirically evaluating the likelihood of dependencies becoming outdated or vulnerable across version constraint types at scale. In this study, we first identify the trends in dependency version constraint usage and the patterns of version constraint type changes made by developers in the npm, PyPI, and Cargo ecosystems. We then modeled the dependency state transitions using survival analysis and estimated how the likelihood of becoming outdated or vulnerable changes when using pinning as opposed to the rest of the version constraint types. We observe that among outdated and vulnerable dependencies, the most commonly used version constraint type is floating-minor, with pinning being the next most common. We also find that floating-major is the least likely to result in outdated and floating-minor is the least likely to result in vulnerable dependencies.

cs.SE↗

Relative Positioning Based Code Chunking Method For Rich Context Retrieval In Repository Level Code Completion Task With Code Language Model

Code completion can help developers improve efficiency and ease the development lifecycle. Although code completion is available in modern integrated development environments (IDEs), research lacks in determining what makes a good context for code completion based on the information available to the IDEs for the large language models (LLMs) to perform better. In this paper, we describe an effective context collection strategy to assist the LLMs in performing better at code completion tasks. The key idea of our strategy is to preprocess the repository into smaller code chunks and later use syntactic and semantic similarity-based code chunk retrieval with relative positioning. We found that code chunking and relative positioning of the chunks in the final context improve the performance of code completion tasks.

cs.SE↗

Assumptions to Evidence: Evaluating Security Practices Adoption and Their Impact on Outcomes in the npm Ecosystem

Practitioners often struggle with the overwhelming number of security practices outlined in cybersecurity frameworks for risk mitigation. Given the limited budget, time, and resources, practitioners want to prioritize the adoption of security practices based on empirical evidence. The goal of this study is to assist practitioners and policymakers in making informed decisions on which security practices to adopt by evaluating the relationship between software security practices adoption and security outcome metrics. To do this, we analyzed the adoption of security practices and their impact on security outcome metrics across 145K npm packages. We selected the OpenSSF Scorecard metrics to automatically measure the adoption of security practices in npm GitHub repositories. We also investigated project-level security outcome metrics: the number of open vulnerabilities (Vul_Count)), mean time to remediate (MTTR) vulnerabilities in dependencies, and mean time to update (MTTU) dependencies. We conducted regression and causal analysis using 11 Scorecard metrics and the aggregated Scorecard score (computed by aggregating individual security practice scores) as predictors and Vul_Count), MTTR, and MTTU as target variables. Our findings reveal that aggregated adoption of security practices is associated with 5.2 fewer vulnerabilities, 216.8 days faster MTTR, and 52.3 days faster MTTU. Repository characteristics have an impact on security practice effectiveness: repositories with high security practice adoptions, especially those that are mature, actively maintained, large in size, have many contributors, few dependencies, and high download volumes, tend to exhibit better outcomes compared to smaller or inactive repositories.

cs.SE↗

S3C2 Summit 2024-09: Industry Secure Software Supply Chain Summit

While providing economic and software development value, software supply chains are only as strong as their weakest link. Over the past several years, there has been an exponential increase in cyberattacks, specifically targeting vulnerable links in critical software supply chains. These attacks disrupt the day-to-day functioning and threaten the security of nearly everyone on the internet, from billion-dollar companies and government agencies to hobbyist open-source developers. The ever-evolving threat of software supply chain attacks has garnered interest from the software industry and the US government in improving software supply chain security. On September 20, 2024, three researchers from the NSF-backed Secure Software Supply Chain Center (S3C2) conducted a Secure Software Supply Chain Summit with a diverse set of 12 practitioners from 9 companies. The goals of the Summit were to: (1) to enable sharing between individuals from different companies regarding practical experiences and challenges with software supply chain security, (2) to help form new collaborations, (3) to share our observations from our previous summits with industry, and (4) to learn about practitioners' challenges to inform our future research direction. The summit consisted of discussions of six topics relevant to the companies represented, including updating vulnerable dependencies, component and container choice, malicious commits, building infrastructure, large language models, and reducing entire classes of vulnerabilities.

cs.CR↗