Install and configure
This example uses Python 3.12 to installdpb and its FastCDM dependency in a virtual environment.
Score one page first
Create two text-only Markdown files in the repository root. The reference says120 USD, while the prediction reads 100 USD. Use the configuration file you just copied.
Files=1, DPB=0.9286, and Text=0.9286. The overall and text scores are equal because this page contains only text. In single-page.result.json, see summary for the scores and matches for match details.
Prepare references and model output
Download the dataset using the Hugging Face CLI installed with the toolkit.images/ contains the source pages, and markdowns/ contains the corresponding reference Markdown.
images/ and save each page’s Markdown directly under predictions/model-a/. For example, save the prediction for education_chemistry_00001.png as predictions/model-a/education_chemistry_00001.md. The evaluator pairs only .md files with identical names at the top level of both directories.
Run the evaluation and check results
To use CDM, the default formula metric, install Node.js, Chrome, and a compatible ChromeDriver. Setchromedriver_path in config.yaml to your local driver path. ChromeDriver installation guide
Files, DPB, Text, Formula, Table, and FormulaRenderFailures. The JSON output contains aggregate scores in summary and page scores and segment matches in reports.
Use the same page list and scoring configuration when comparing models. Check Files, the per-page reports, and the formula rendering failure count to confirm that the runs cover the same pages. The default configuration skips pages whose reference annotation contains \smiles{...}.
Score individual formulas with FastCDM
You can run the Python example below in a DocParsingBench environment. For formula scoring alone, runpython -m pip install fastcdm in a virtual environment and use the browser and driver setup above. Replace the example path with your local ChromeDriver path before running it.
visualize=True and read the fourth return value. FastCDM usage
Inspect differences page by page
When a page scores poorly, usedpb segment-report to export its reference, prediction, normalized Markdown, and segment breakdown. Replace the filename on the last line with the page you want to inspect.
In repository revision
90bf1fb, dpb visualize passes a css launch argument that Gradio 4.x rejects. If you encounter this error, inspect the per-page JSON or the segment report. Visual analysis launch codeGenerate an interactive leaderboard
Save each model’s batch results as*.result.json files in results/ and run the following command. Open results/leaderboard.html to filter by industry and sort by DPB or component score.

