We want to build a tool that improves the quality of data on Looked After Children so that Children’s Services Departments have all the information needed to enhance their services.
We believe that a tool that highlights and helps fixing data errors would be valuable for:
- Reducing the time analysts, business support and social workers spend cleaning data.
- Enabling leadership to better use evidence in supporting Looked After Children.
The aim of this project is to deliver a tool to relieve some of the pain-points of reporting and quality in children's services data. This project focuses, in particular, on data on looked after children (LAC) and the SSDA903 return.
The project consists of a number of related pieces of work:
- Hosted Tool
- React & Pyodide Front-End [this repo]
- Python Validator Engine & Rules
- Local Authority Reference Data
- Postcode Reference Data
The core parts consist of a Python validator engine and rules using Pandas with Poetry for dependency management. The tool is targeted to run either standalone, or in pyodide in the browser for a zero-install deployment with offline capabilities.
It provides methods of finding the validation errors defined by the DfE in 903 data. The validator needs to be provided with a set of input files for the current year and, optionally, the previous year. These files are coerced into a common format and sent to each of the validator rules in turn. The validators report on rows not meeting the rules and a report is provided highlight errors for each row and which fields were included in the checks.
To get started, you need yarn and a recent version of node.
Install dependencies and deploy a test version:
yarn install
yarn start
The front-end should automatically re-load when changes are made, but if not,
simply reload. You don't need to run yarn install again unless you have changed
dependencies.
The version of the validator is set through the [.env][./.env] file:
REACT_APP_VALIDATOR_RELEASE=quality-lac-data-validator==x.x.x
(where x.x.x is the target release)
To use a development version of the validator, copy the python wheel to the public folder of this project, then create a file called .env.local and add the following line:
REACT_APP_VALIDATOR_RELEASE=http://<hostname>:<port>/quality_lac_data_validator-<x.x.x>-py3-none-any.whlWhere hostname and port are the hostname and port as appearing in your development server, e.g.
Compiled successfully!
You can now view quality-lac-data-beta in the browser.
Local: http://localhost:3000
On Your Network: http://192.168.86.31:3000
Note that the development build is not optimized.
To create a production build, use yarn build.