Neural networks in the field of intravascular ultrasound studies (literature review)
AbstractThis article is a review discussing neural networks used in the field of intravascular ultrasound examinations and performing functions of automatic identification of unstable plaques, isolation of vascular walls, prediction of the fractional flow reserve. Based on the analysed material, it was determined that neural networks in the field of ultrasound studies are currently an emerging and actively developing trend of intravascular imaging. In case if it is possible to achieve the human-comparable accuracy of the obtained results and to prove them in direct comparison in a clinical trial it could lead to decreasing the cost and increasing the rapidity of performing a percutaneous coronary intervention. We also believe that in future it will be possible to combine the algorithms of automated detection of unstable plaques and prediction of fractional flow reserve, thus unifying intravascular ultrasound examinations for many clinical situations.
Keywords: neural networks; machine learning; intravascular ultrasound examination; intravascular imaging
Funding. The study had no financial support.
Conflict of interest. The authors declare no conflicts of interest.
Authors’ contribution. Study conception and design – Khidirova L.D., Kovalev E.A.; data collection and handling – Kovalev E.A.; draft manuscript preparation – Kovalev E.A.; manuscript revision – Khidirova L.D.
For citation: Kovalev E.A., Khidirova L.D. Neural networks in the field of intravascular ultrasound studies (literature review). Angiology and Vascular Surgery. Journal named Academician A.V. Pokrovsky. 2022; 28 (3): 32–6. DOI: https://doi.org/10.33029/1027-6661-2022-28-3-32-36 (in Russian)
References
1. Benjamens S., Dhunnoo P., Meskó B. The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. NPJ Digital Medicine. 2020; 3 (1): 1–8. DOI: https://doi.org/10.1038/s41746-020-00324-0
2. Matheny M.E., Whicher D., Thadaney I.S. Artificial intelligence in health care: a report from the National Academy of Medicine. JAMA. 2020; 323 (6): 509–510. DOI: https://doi.org/10.1001/jama.2019.21579
3. Mincholé A., Rodriguez B. Artificial intelligence for the electrocardiogram. Nature Medicine. 2019; 25 (1): 22–23. DOI: https://doi.org/10.1038/s41591-018-0306-1
4. Hata T., Koyanagi A., Yamanishi T., et al. Superb microvascular imaging with Doppler luminance using an 18-MHz probe to visualize fetal intra-abdominal blood vessels and organ microvasculature. Journal of Perinatal Medicine. 2020; 48 (2): 184–188. DOI: https://doi.org/10.1515/jpm-2019-0411
5. Bae Y., Kang S.J., Kim G., et al. Prediction of coronary thin-cap fibroatheroma by intravascular ultrasound-based machine learning. Atherosclerosis. 2019; 288 (5): 168–174. DOI: https://doi.org/10.1016/j.atherosclerosis.2019.04.228
6. Jun T.J., Kang S.J., Lee J.G., et al. Automated detection of vulnerable plaque in intravascular ultrasound images. Medical and Biological Engineering and Computing. 2019; 57 (4): 863–876. DOI: https://doi.org/10.1007/s11517-018-1925-x
7. Allen W.E., Chen M.Z., Pichamoorthy N., et al. Thirst regulates motivated behavior through modulation of brainwide neural population dynamics. Science. 2019; 364 (6437): 253. DOI: https://doi.org/10.1126/science.aav3932
8. Cao Y., Xiao X., Liu Z., et al. Detecting vulnerable plaque with vulnerability index based on convolutional neural networks. Computerized Medical Imaging and Graphics. 2020; 81: 101711. DOI: https://doi.org/10.1016/j.compmedimag.2020.101711
9. Lo V.L., Fresno M., Larrabide I. Lumen-intima and media-adventitia segmentation in IVUS images using supervised classifications of arterial layers and morphological structures. Computer Methods and Programs in Biomedicine. 2019; 177: 113–121. DOI: https://doi.org/10.1016/j.cmpb.2019.05.021
10. Yang J., Faraji M., Basu A. Robust segmentation of arterial walls in intravascular ultrasound images using Dual Path U-Net. Ultrasonics. 2019; 96 (3): 24–33. DOI: https://doi.org/10.1016/j.ultras.2019.03.014
11. Wang Y.Y., Qiu C.H., Jiang J., et al. Detecting the media-adventitia border in intravascular ultrasound images through a classification-based approach. Ultrasonic Imaging. 2019; 41 (2): 78–93. DOI: https://doi.org/10.1177/0161734618820112
12. Gao Z., Hau W.K., Lu M., et al. Automated framework for detecting lumen and media-adventitia borders in intravascular ultrasound images. Ultrasound in Medicine and Biology. 2019; 41 (7): 2001–2021. DOI: https://doi.org/10.1016/j.ultrasmedbio.2015.03.022
13. Wu H., Gole R., Ghosh S., et al. Alternative techniques for breast tumour detection using ultrasound. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2020; 7: 2047–2050. DOI: https://doi.org/10.1109/EMBC44109.2020.9175867
14. Papaioannou T.G., Schizas D., Vavuranakis M., et al. Quantification of new structural features of coronary plaques by computational post-hoc analysis of virtual histology-intravascular ultrasound images. Computer Methods in Biomechanics and Biomedical Engineering. 2014; 17 (6): 643–651. DOI: https://doi.org/10.1080/10255842.2012.713940
15. Athanasiou L., Sakellarios A.I., Bourantas C.V., et al. Currently avail-able methodologies for the processing of intravascular ultrasound and optical coherence tomography images. Expert Review of Cardiovascular Therapy. 2014; 12 (7): 885–900. DOI: https://doi.org/10.1586/14779072.2014.922413
16. Xia M., Yan W., Huang Y., et al. Extracting membrane borders in IVUS images using a multi-scale feature aggregated U-Net. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2020; 7: 1650–1653. DOI: https://doi.org/10.1109/EMBC44109.2020.9175970
17. Azad M., Asadi–Aghbolaghi M., Fathy M., Escalera S. Bi-directional ConvLSTM U-Net with densely connected convolutions. International Conference on Computer Vision Workshops. 2019; 1–10.
18. Katouzian A., Angelini E.D., Carlier S.G., еt al. A state-of-the-art review on segmentation algorithms in intravascular ultrasound (IVUS) images. IEEE Transactions on Information Technology in Biomedicine. 2012; 16 (5): 823–834. DOI: https://doi.org/10.1109/TITB.2012.2189408
19. Inaba S., Mintz G.S., Burke A.P., et al. Intravascular ultrasound and near-infrared spectroscopic characterization of thin-cap fibroatheroma. American Journal of Cardiology. 2017; 119 (3): 372–378. DOI: https://doi.org/10.1016/j.amjcard.2016.10.031
20. Lee J.G., Ko J., Hae H., et al. Intravascular ultrasound-based machine learning for predicting fractional flow reserve in intermediate coronary artery lesions. Atherosclerosis. 2020; 292: 171–177. DOI: https://doi.org/10.1016/j.atherosclerosis.2019.10.022
Посредством предлагаемого вида нейросетей возможно прогнозировать такой показатель, как ФРК. Разрабатывался алгоритм следующим образом: были отобраны сведения из проведенных исследований в период с ноября 2009 по июль 2015 г. 1657 пациентов со стабильной и нестабильной стенокардией, из которых в последующем были исключены 329. Критериями исключения являлись многососудистые поражения, наличие стента внутри целевого сосуда, оценка боковых ветвей при проведении коронарографии, стеноз ствола левой коронарной артерии более 30% [20]. Пациентам выполнялись инвазивная коронарография, предоперационная ВСУЗИ и ФРК. Далее была проведены рандомизация и распределение исследований на 2 выборки - обучающую и тестовую в соотношении 4:1 (1063:265).
С целью обучения сети и формирования прогноза ФРК в тестовой группе в алгоритм также были заложены следующие сведения о пациентах: характеристики атеромы, полученные при проведении ВСУЗИ, возраст, пол, наличие сахарного диабета, курил ли на момент выполнения коронарографии больной, наличие артериальной гипертензии, стабильной или нестабильной стенокардии, площадь поверхности тела, ФРК, сегмент какой артерии был поражен (проксимальный/средний/дистальный передней нисходящей артерии; проксимальный/дистальный правой коронарной артерии; проксимальный/дистальный левой огибающей артерии).
При тестировании нейросети предоставлялись все вышеперечисленные данные, за исключением ФРК соответственно. Результатом наилучшей модификации алгоритма оказалась точность 87% (AUC). В будущем группа исследователей планирует продолжить свою работу в этой области и подтвердить полученные результаты на большей когорте.
Заключение
В настоящее время использование нейросетей в области ВСУЗИ представляет собой зарождающееся и активно развивающееся направление исследований. Использование внутрисосудистой ультразвуковой диагностики, особенно при лечении сложных коронарных поражений, помогает оптимизировать результаты эндоваскулярных вмешательств и способно повлиять на клинический исход в отдаленные сроки после процедуры. Применение нейросети, возможно, позволит добиться более качественного результата и лучшего прогноза.
Можно предположить, что алгоритмы автоматического определения нестабильных бляшек и прогнозирования ФРК в будущем можно будет объединить, что приведет к унификации ВСУЗИ при многих клинических ситуациях.
Литература/References
1. Benjamens S., Dhunnoo P., Meskó B. The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. NPJ Digital Medicine. 2020; 3 (1): 1-8. DOI: https://doi.org/10.1038/s41746-020-00324-0
2. Matheny M.E., Whicher D., Thadaney I.S. Artificial intelligence in health care: a report from the National Academy of Medicine. JAMA. 2020; 323 (6): 509-510. DOI: https://doi.org/10.1001/jama.2019.21579
3. Mincholé A., Rodriguez B. Artificial intelligence for the electrocardiogram. Nature Medicine. 2019; 25 (1): 22-23. DOI: https://doi.org/10.1038/s41591-018-0306-1
4. Hata T., Koyanagi A., Yamanishi T., et al. Superb microvascular imaging with Doppler luminance using an 18-MHz probe to visualize fetal intra-abdominal blood vessels and organ microvasculature. Journal of Perinatal Medicine. 2020; 48 (2): 184-188. DOI: https://doi.org/10.1515/jpm-2019-0411
5. Bae Y., Kang S.J., Kim G., et al. Prediction of coronary thin-cap fibroatheroma by intravascular ultrasound-based machine learning. Atherosclerosis. 2019; 288 (5): 168-174. DOI: https://doi.org/10.1016/j.atherosclerosis.2019.04.228
6. Jun T.J., Kang S.J., Lee J.G., et al. Automated detection of vulnerable plaque in intravascular ultrasound images. Medical and Biological Engineering and Computing. 2019; 57 (4): 863-876. DOI: https://doi.org/10.1007/s11517-018-1925-x
7. Allen W.E., Chen M.Z., Pichamoorthy N., et al. Thirst regulates motivated behavior through modulation of brainwide neural population dynamics. Science. 2019; 364 (6437): 253. DOI: https://doi.org/10.1126/science.aav3932
8. Cao Y., Xiao X., Liu Z., et al. Detecting vulnerable plaque with vulnerability index based on convolutional neural networks. Computerized Medical Imaging and Graphics. 2020; 81: 101711. DOI: https://doi.org/10.1016/j.compmedimag.2020.101711
9. Lo V.L., Fresno M., Larrabide I. Lumen-intima and media-adventitia segmentation in IVUS images using supervised classifications of arterial layers and morphological structures. Computer Methods and Programs in Biomedicine. 2019; 177: 113-121. DOI: https://doi.org/10.1016/j.cmpb.2019.05.021
10. Yang J., Faraji M., Basu A. Robust segmentation of arterial walls in intravascular ultrasound images using Dual Path U-Net. Ultrasonics. 2019; 96 (3): 24-33. DOI: https://doi.org/10.1016/j.ultras.2019.03.014
11. Wang Y.Y., Qiu C.H., Jiang J., et al. Detecting the media-adventitia border in intravascular ultrasound images through a classification-based approach. Ultrasonic Imaging. 2019; 41 (2): 78-93. DOI: https://doi.org/10.1177/0161734618820112
12. Gao Z., Hau W.K., Lu M., et al. Automated framework for detecting lumen and media-adventitia borders in intravascular ultrasound images. Ultrasound in Medicine and Biology. 2019; 41 (7): 2001-2021. DOI: https://doi.org/10.1016/j.ultrasmedbio.2015.03.022
13. Wu H., Gole R., Ghosh S., et al. Alternative techniques for breast tumour detection using ultrasound. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2020; 7: 2047-2050. DOI: https://doi.org/10.1109/EMBC44109.2020.9175867
14. Papaioannou T.G., Schizas D., Vavuranakis M., et al. Quantification of new structural features of coronary plaques by computational post-hoc analysis of virtual histology-intravascular ultrasound images. Computer Methods in Biomechanics and Biomedical Engineering. 2014; 17 (6): 643-651. DOI: https://doi.org/10.1080/10255842.2012.713940
15. Athanasiou L., Sakellarios A.I., Bourantas C.V., et al. Currently avail-able methodologies for the processing of intravascular ultrasound and optical coherence tomography images. Expert Review of Cardiovascular Therapy. 2014; 12 (7): 885-900. DOI: https://doi.org/10.1586/14779072.2014.922413
16. Xia M., Yan W., Huang Y., et al. Extracting membrane borders in IVUS images using a multi-scale feature aggregated U-Net. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2020; 7: 1650-1653. DOI: https://doi.org/10.1109/EMBC44109.2020.9175970
17. Azad M., Asadi-Aghbolaghi M., Fathy M., Escalera S. Bi-directional ConvLSTM U-Net with densely connected convolutions. International Conference on Computer Vision Workshops. 2019; 1-10.
18. Katouzian A., Angelini E.D., Carlier S.G., еt al. A state-of-the-art review on segmentation algorithms in intravascular ultrasound (IVUS) images. IEEE Transactions on Information Technology in Biomedicine. 2012; 16 (5): 823-834. DOI: https://doi.org/10.1109/TITB.2012.2189408
19. Inaba S., Mintz G.S., Burke A.P., et al. Intravascular ultrasound and near-infrared spectroscopic characterization of thin-cap fibroatheroma. American Journal of Cardiology. 2017; 119 (3): 372-378. DOI: https://doi.org/10.1016/j.amjcard.2016.10.031
20. Lee J.G., Ko J., Hae H., et al. Intravascular ultrasound-based machine learning for predicting fractional flow reserve in intermediate coronary artery lesions. Atherosclerosis. 2020; 292: 171-177. DOI: https://doi.org/10.1016/j.atherosclerosis.2019.10.022
All articles in our journal are distributed under the CC BY-NC-ND 4.0 (Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International)
п?? GEOTAR-Media Publishing Group. The use of textual and illustrative content from this publication for the training of any artificial intelligence systems п?? including machine learning models and neural networks п?? is strictly prohibited without the prior written consent of the copyright holder.