Latest Articles for Benchmark

Researchers analyze the predictability of language model performance as training compute scales.

A look at backdoor attacks and defenses in deep learning models.

This paper assesses the efficiency of generated code from various models.

This article presents a benchmark to assess large language models with complex tasks.

This study assesses large language models' capabilities in complex planning scenarios.

Research examines the use of VLMs to assess robot actions.

Exploring the role of large language models in molecular science.

Exploring methods to improve robot performance in unpredictable environments.

AV-SUPERB evaluates audio and visual models across various tasks for better performance.

New tools improve how systems retrieve information from long documents.

This benchmark assesses the performance of medical language models in healthcare.

A method to keep AI models updated based on real-world events.

New benchmark tests MLLMs on social media tasks like misinformation and hate speech.

RobotScript enhances how robots execute tasks from natural language.

A fresh perspective on finding hidden threats in hardware design.

New methods aim to better evaluate reasoning skills in AI language models.

DyPyBench offers a diverse set of projects for dynamic analysis in Python.

AI's capability to turn designs into code is reshaping web development.

Study reveals significant data overlap affecting language model evaluations in code generation.

Assessing LLM performance through a dedicated benchmark for bio-image analysis.

A new method for assessing language processing tools shows promise for improvement.

A method for assessing the transferability of pre-trained models for object detection.

A resource designed to help robots learn everyday tasks effectively.

A look at assessing the decision-making capabilities of large language models.

A framework to improve NLP performance across various language dialects.

A fresh benchmark uncovers strengths and weaknesses of VLLMs in multimodal tasks.

Experts gather to discuss Monte Carlo simulations and GPU enhancements.

New benchmarks reveal strengths and weaknesses of coding language models.

Meerkat-7B sets a new standard for open-source medical language models.

New methods improve video summarization using large datasets and advanced models.

Research reveals challenges LLMs face in understanding long texts and proposes new benchmarks.

Exploring the design and benefits of a PMU for RISC-V processors used in space.

This study examines quality problems in prompts for code generation models.

A new benchmark reveals gaps in visual understanding of large language models.

A new benchmark improves how we assess LVLMs and their accuracy.

The CHC competition showcased advances in solvers and their applications in program verification.

This article explores how to improve the understanding of indirect answers.

A study evaluating few-shot learning methods for Polish language classification.

PatentGPT models are designed to address unique challenges in Intellectual Property.

A study on the effectiveness of SAST tools for smart contracts.