What is DocParsingBench?
DocParsingBench is an open-source document parsing benchmark from SoMark. It includes a public dataset, a scoring CLI, and visual analysis tools. It compares a model’s Markdown output against a human annotation of the same page and reports text, formula, table, and overall DPB scores. Any model that produces Markdown can be compared on the same dataset and metrics.GitHub repository
View the source code, scoring method, and public leaderboard.
Hugging Face dataset
Download the original page images and reference Markdown.
ModelScope dataset
Access the dataset through ModelScope.
Dataset coverage
The dataset contains 1,400 manually annotated pages from business workflows. It covers Chinese, English, and mixed-language pages, with single-column, two-column, three-column, and mixed layouts. The pages preserve conditions such as scan noise, overlapping stamps, and blurred characters. Dataset cardScoring metrics
The toolkit segments the Markdown into text, display formulas, and tables, then pairs segments one to one within each category. Corresponding content can still match when a model places segments in a different order. Unmatched segments receive zero similarity.
The general preset assigns 50% to text, 30% to display formulas, and 20% to tables. It redistributes weights across the categories present when a category is absent from both reference and prediction. The default configuration filters images and skips pages whose reference Markdown contains
\smiles{...}, so the scored page count can differ from the dataset size. Example configuration

