Offline reinforcement learning for safer blood glucose control in people with type 1 diabetes

The widespread adoption of effective hybrid closed loop systems would represent an important milestone of care for people living with type 1 diabetes (T1D). These devices typically utilise simple control algorithms to select the optimal insulin dose for maintaining blood glucose levels within a heal...

Full description

Saved in:
Bibliographic Details
Published in:Journal of biomedical informatics 2023-06, Vol.142, p.104376-104376, Article 104376
Main Authors: Emerson, Harry, Guy, Matthew, McConville, Ryan
Format: Article
Language:English
Subjects:
Artificial pancreas
Glucose control
Reinforcement learning
Type 1 diabetes
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The widespread adoption of effective hybrid closed loop systems would represent an important milestone of care for people living with type 1 diabetes (T1D). These devices typically utilise simple control algorithms to select the optimal insulin dose for maintaining blood glucose levels within a healthy range. Online reinforcement learning (RL) has been utilised as a method for further enhancing glucose control in these devices. Previous approaches have been shown to reduce patient risk and improve time spent in the target range when compared to classical control algorithms, but are prone to instability in the learning process, often resulting in the selection of unsafe actions. This work presents an evaluation of offline RL for developing effective dosing policies without the need for potentially dangerous patient interaction during training. This paper examines the utility of BCQ, CQL and TD3-BC in managing the blood glucose of the 30 virtual patients available within the FDA-approved UVA/Padova glucose dynamics simulator. When trained on less than a tenth of the total training samples required by online RL to achieve stable performance, this work shows that offline RL can significantly increase time in the healthy blood glucose range from 61.6±0.3% to 65.3±0.5% when compared to the strongest state-of-art baseline (p
ISSN:1532-0464
1532-0480
DOI:10.1016/j.jbi.2023.104376