A deep learning-based medication behavior monitoring system.

Roh, Hyeji; Shin, Seulgi; Han, Jinseo; Lim, Sangsoon

The internet of things (IoT) and deep learning are emerging technologies in diverse research fields, including the provision of IT services in medical domains. In the COVID-19 era, intelligent medication behavior monitoring systems for stable patient monitoring are further required, because many patients cannot easily visit hospitals. Several previous studies made use of wearable devices to detect medication behaviors of patients. However, the wearable devices cause inconvenience while equipping the devices. In addition, they suffer from inconsistency problems due to errors of measured values. We devise a medication behavior monitoring system that uses the IoT and deep learning to avoid sensing errors and improve user experiences by effectively detecting various activities of patients. Based on the real-time operation of our proposed IoT device, the proposed solution processes captured images of patents via OpenPose to check medication situations. The proposed system identifies medication status on time by using a human activity recognition scheme and provides various notifications to patients' mobile devices. To support reliable communication between our system and doctors, we employ MQTT protocol with periodic data transmissions. Thus, the measured information of patient's medication status is transmitted to the doctors so that they can periodically perform remote treatments. Experimental results show that all medication behaviors are accurately detected and notified to the doctor efficiently, improving the accuracy of monitoring the patient's medication behavior.