# How do we ensure and measure accuracy?

**Model accuracy**

Since the models we use to extract and link information together include LLMs, there is a risk that some of the information we include on our platform will be inaccurate. To reduce the risk of sharing inaccurate content, [we have set up multiple guardrails](https://about.system.com/blog/release-risks-october-2022), starting at the source of extraction, with processes in place for monitoring, evaluating, and gathering feedback on inaccuracies.

Since our models are looking for statistical pieces of information in a specific text, we incorporate validation methods both pre- and post-extraction to ensure the content extracted is actually reported in the abstract, design our models to extract information that meets a specific format, and conduct additional post-processing to filter our potentially erroneous results. Together, these guardrails help us significantly reduce the risk of hallucination.

Additionally, by incorporating clear instructions and rules, we have achieved between [84%-90% accuracy in our extraction models](https://systeminc.notion.site/systeminc/911ba241fe2a4f25bbe83c0d23d08022?v=70cf0265938e4779ab670cdb343f0a0f), meaning that each component of the statistical relationship extracted is accurate to the source. Inaccuracies can include misleading variable names, incomplete statistic types (e.g., “odds ratio” instead of “adjusted odds ratio”) or p-values with the wrong sign (p=0.05, instead of p<0.05).

**Tracking and fixing inaccuracies**

In order to continue to improve on these models, we routinely conduct external evaluations on large samples of statistical relationships, and use this information to improve our models. Additionally, we have created processes on the front end for users of System Pro to flag any inaccuracies and suggest revisions to extracted information, which then removes those pieces of evidence from the site until the content can be reviewed and updated. <br>

<figure><img src="/files/QGThslXNN8MmJbKlaWLK" alt="" width="375"><figcaption></figcaption></figure>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.system.com/system/how-system-works/system-platform/how-do-we-ensure-and-measure-accuracy.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.