Skip to content

Commit bea4623

Browse files
authored
Updating about section
1 parent 4186c87 commit bea4623

1 file changed

Lines changed: 40 additions & 4 deletions

File tree

README.md

Lines changed: 40 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,41 @@
1-
# PLSemanticsBench
2-
3-
[![Hugging Face](https://img.shields.io/badge/Hugging%20Face-Dataset-blue)](https://huggingface.co/datasets/EngineeringSoftware/PLSemanticsBench)
1+
<div align="center">
2+
<p>The 43rd International Conference on Machine Learning (ICML 2026), Seoul, South Korea</p>
3+
<h1>
4+
LLMs Lean on Priors, Not Programming Language Semantics
5+
<img
6+
src="https://raw.githubusercontent.com/EngineeringSoftware/PLSemanticsBench/main/docs/logo.png"
7+
alt="PLSemanticsBench logo"
8+
width="50"
9+
style="display: inline-block !important;
10+
vertical-align: 4.0em;
11+
margin-left: 10px;
12+
margin-top: 10px;">
13+
</span>
14+
</h1>
15+
16+
<p style="font-size: 20px;">
17+
by
18+
<a href="https://www.adityathimmaiah.com">Aditya Thimmaiah</a><sup>1</sup>,
19+
<a href="https://jiyangzhang.github.io/">Jiyang Zhang</a><sup>1</sup>,
20+
<a href="https://scholar.google.com/citations?user=HtNfeKYAAAAJ&hl=en">Jayanth Srinivasa</a><sup>2</sup>,
21+
<a href="https://www.jessyli.com">Junyi Jessy Li</a><sup>1</sup>,
22+
<a href="https://users.ece.utexas.edu/~gligoric/">Milos Gligoric</a><sup>1</sup>
23+
</p>
24+
25+
<p>
26+
<sup>1</sup>The University of Texas at Austin &nbsp;&nbsp;&nbsp;
27+
<sup>2</sup>Cisco Research
28+
</p>
29+
</div>
30+
31+
<div align="center">
32+
33+
[![Website](https://img.shields.io/badge/Project_Page-PLSemanticsBench-blueviolet)](https://engineeringsoftware.github.io/PLSemanticsBench/)
34+
[![arXiv](https://img.shields.io/badge/arXiv-2510.03415v3-b31b1b.svg)](https://arxiv.org/pdf/2510.03415v3)
35+
[![Code](https://img.shields.io/badge/Code-GitHub-black)](https://github.com/EngineeringSoftware/PLSemanticsBench)
36+
[![Dataset](https://img.shields.io/badge/🤗-Dataset-yellow)](https://huggingface.co/datasets/EngineeringSoftware/PLSemanticsBench)
37+
38+
</div>
439

540

641
## Table of Contents
@@ -12,7 +47,8 @@
1247
- [Citation](#citation)
1348

1449
## About
15-
PLSemanticsBench is the first benchmark for evaluating LLMs as programming language interpreters. We introduce three tasks to evaluate this:
50+
PLSemanticsBench is the first counterfactual PL semantics dataset for evaluating rule-conditioned reasoning in LLMs.
51+
Program execution is used as a lens for evaluating it via three tasks:
1652

1753
| Task | Description |
1854
|------|-------------|

0 commit comments

Comments
 (0)