Climate change is rapidly altering ecosystems, and researchers and policymakers struggle to keep up despite an expanding but complex corpus of scientific literature. Large language models could help understand this data, but their propensity for unsupported or incorrect outputs limits their utility for environmental decision-making.
Knowledge graphs (KGs) enable symbolic fact-checking of LLMs, but current approaches assume that the underlying graph is reliable. This is problematic when LLMs are increasingly used to automate KG construction: converting probabilistic LLM outputs into discrete symbolic triples strips away the reliability signal regarding source support. Consequently, downstream systems may treat unreliable graph content as absolute truth. This undermines the core purpose of KG-based fact-checking: if the KG itself contains unsupported or hallucinated facts, it may incorrectly validate false claims rather than prevent them.
GUARDRAIL investigates the end-to-end propagation of reliability signals in KG-grounded LLM reasoning for environmental data. In order to improve reliability, the project encodes metrics of source-groundedness and citation network density during KG construction. These signals are propagated through retrieval and reasoning, allowing for claim verification and abstention to prevent false information from reaching the end user. We develop and apply GUARDRAIL to climate-related claim verification, with a secondary transfer to plant- and pest-related QA.
We address these research questions:
● Can KG triples’ reliability be quantified using extraction uncertainty and source-quality signals?
● Can these signals be used to improve retrieval and claim-verification performance?
● Does propagating unreliability measures enable safer, better calibrated deferral?
● Can these methods improve LLM factuality across multiple environmental domains?