ICU SpyEyes

Problem: The ICU requires a system to automatically record video loops capturing key activities like patient and personnel movements. This system will allow retrospective annotations by humans, pre-annotate and identify persons, and optionally blur patient faces. It aims to collect data for developing future patient surveillance algorithms in hospitals.

Solution: We've developed a system for the ICU that utilizes motion detection, LLM, and standard CV techniques for video analysis. It builds an annotated video database essential for training algorithms focused on patient monitoring. Our solution generates structured reports and videos shared via FHIR, ensuring efficient data handling. Quality is enhanced by manual review through the open-source platform CVAT, optimizing our real-time alarm system for monitoring patient recovery.

Impact: Thanks to our cutting-edge solution, it is possible to train an algorithm that can function as a real-time alarm in the ICU for monitoring patient recovery.

Feasibility & financials: There is nothing stopping our project going live. All the technologies we've used are standard and we can connecting the cameras immediately.

What is new about your solution?: We've created a system that generates a high volume of annotated videos for advanced algorithmic learning. This innovation significantly boosts data availability for algorithm development. With minor modifications, our system also promises to deliver real-time alerts, leveraging its rapid analysis capability—completing in seconds, not minutes. This advancement is a major step towards more immediate and effective patient monitoring in ICUs.

What you have built at the hackathon - text explanation + code (e.g. GitHub link): We have developed a system that processes video feeds from ICU rooms, monitoring for incidents. When an incident is detected, it generates a video clip. This clip is analyzed to identify the people present, the patient's position, and to create a summary. The data and video are then integrated into FHIR databases and CVAT. https://github.com/martingeorgiu/icu-spyeyes https://app.cvat.ai/tasks?page=1 done.sulc@gmail.com Ehh2023*

What you had before the hackathon, please mention open source as well: The entirety of our research and development for this project was conducted exclusively during the hackathon. Our work began from scratch on Friday afternoon. Naturally, as we progressed with our development, we incorporated some existing open-source projects into our solution.

What comes next and what you wish to achieve: Discuss next steps with Dr. Šramko and aim to initiate a pilot program in IKEM's ICU as soon as possible.

Video: https://www.youtube.com/embed/-9tJfnMx64E