The Department for Business and Trade (DBT) has implemented self-hosted Large Language Models (LLMs) within a secure, internal environment to address security and privacy concerns associated with public AI tools.
This initiative, led by the Digital, Data and Technology (DDaT) team, aims to leverage AI for efficient data analysis while maintaining strict control over sensitive information like trade policy and business data.
Using Amazon SageMaker within a private Virtual Private Cloud (VPC), DBT deployed open-source LLMs, enabling GPU usage for faster processing. A successful pilot with 15 users showed no critical issues and received positive feedback, marking DBT as one of the first government departments to adopt this approach.
User research highlighted a demand for secure, powerful LLMs, and an internal Python package was developed to facilitate interaction with SageMaker, optimizing costs and resource sharing across projects.
Measuring the Impact of AI
Here they discuss the role of evaluation in assessing the impact of AI interventions across UK government departments, led by the Department for Business and Trade (DBT).
It emphasizes the need for robust evaluation to ensure AI delivers value, improves outcomes, and maintains public trust. The DBT’s Evaluation Community of Practice highlights challenges like defining AI intervention success and measuring intangible benefits.
Key evaluation methods include Randomized Control Trials, quasi-experimental approaches, and qualitative methods like interviews to capture user experiences.
The article stresses aligning evaluations with policy goals, addressing ethical concerns, and fostering cross-government collaboration to share best practices. DBT’s efforts aim to build evidence-based AI adoption, ensuring transparency and accountability in public sector AI use.
