{"id":1180878,"date":"2026-06-12T00:00:00","date_gmt":"2026-06-12T07:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-research-item&#038;p=1180878"},"modified":"2026-08-21T10:12:59","modified_gmt":"2026-08-21T17:12:59","slug":"comprehensive-framework-for-evaluation-of-deep-neural-networks-in-detection-and-quantification-of-lymphoma-from-pet-ct-images-clinical-insights-pitfalls-and-observer-agreement-analyses","status":"publish","type":"msr-research-item","link":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/comprehensive-framework-for-evaluation-of-deep-neural-networks-in-detection-and-quantification-of-lymphoma-from-pet-ct-images-clinical-insights-pitfalls-and-observer-agreement-analyses\/","title":{"rendered":"Comprehensive framework for evaluation of deep neural networks in detection and quantification of lymphoma from PET\/CT images: Clinical insights, pitfalls, and observer agreement analyses."},"content":{"rendered":"\n\n\n<h4 id=\"purpose\" class=\"wp-block-heading\">Purpose:<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">This study addresses critical gaps in automated lymphoma segmentation from PET\/CT imaging, often overlooked in prior work. While deep learning has been applied to this task, few studies evaluate generalizability on external or out-of-distribution data. Similarly, intra- and inter-observer variability analyses remain rare, limiting understanding of task difficulty. Moreover, most methods emphasize global segmentation metrics, neglecting lesion-level characteristics that are crucial for clinical decision-making.<\/p>\n\n\n\n<h4 id=\"methods\" class=\"wp-block-heading\">Methods:<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">We propose a clinically-relevant evaluation framework to assess four commonly used deep segmentation networks (ResUNet, SegResNet, DynUNet, SwinUNETR) on 611 PET\/CT cases from multi-institutional datasets spanning varied lymphoma subtypes and lesion characteristics. In addition to the Dice similarity coefficient (DSC), we compute prediction errors on clinical lesion measures and analyze DSC performance as a function of these measures. Additionally, we use traditional lesion-specific detection criteria (1 and 2), providing insights into network\u2019s performance in identifying and localizing lesions respectively, and propose an additional Criterion 3 for segmenting lesions based on metabolic characteristics. Finally, we contextualize network performance by comparing it to expert human observers through intra- and inter-observer variability analyses.<\/p>\n\n\n\n<h4 id=\"results\" class=\"wp-block-heading\">Results:<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">Networks perform best on large, metabolically active lesions. Their error patterns closely resemble those of expert annotators, while small and faint lesions remain challenging for both networks and physicians.<\/p>\n\n\n\n<h4 id=\"conclusion\" class=\"wp-block-heading\">Conclusion:<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">Our clinically-relevant benchmarking framework enables more consistent and meaningful evaluation of lymphoma segmentation models, supporting robust decision-making in patient care. The approach is extensible to other architectures and disease types. Code is available at:&nbsp;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" href=\"https:\/\/github.com\/microsoft\/lymphoma-segmentation-dnn\" target=\"_blank\" rel=\"noopener noreferrer\">https:\/\/github.com\/microsoft\/lymphoma-segmentation-dnn<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>This study addresses critical gaps in automated lymphoma segmentation from PET\/CT imaging, often overlooked in prior work. While deep learning has been applied to this task, few studies evaluate generalizability on external or out-of-distribution data. Similarly, intra- and inter-observer variability analyses remain rare, limiting understanding of task difficulty. Moreover, most methods emphasize global segmentation metrics, [&hellip;]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[{"type":"text","value":"Shadab Ahamed","user_id":0},{"type":"user_nicename","value":"Yixi Xu","user_id":"39775"},{"type":"text","value":"Sara Kurkowska","user_id":0},{"type":"text","value":"Claire Gowdy","user_id":0},{"type":"text","value":"Joo H.O. ","user_id":0},{"type":"text","value":"Ingrid Bloise","user_id":0},{"type":"text","value":"Don Wilson","user_id":0},{"type":"text","value":"Patrick Martineau","user_id":0},{"type":"text","value":"Fran&ccedil;ois B&eacute;nard","user_id":0},{"type":"text","value":"Fereshteh Yousefirizi ","user_id":0},{"type":"user_nicename","value":"Rahul Dodhia","user_id":"41401"},{"type":"user_nicename","value":"Juan M. Lavista Ferres","user_id":"39552"},{"type":"text","value":"William B Weeks","user_id":0},{"type":"text","value":"Carlos F. Uribe","user_id":0},{"type":"text","value":"Arman Rahmim","user_id":0}],"msr_publishername":"","msr_publisher_other":"","msr_booktitle":"","msr_chapter":"","msr_edition":"","msr_editors":"","msr_how_published":"","msr_isbn":"","msr_issue":"","msr_journal":"","msr_number":"","msr_organization":"","msr_pages_string":"","msr_page_range_start":"","msr_page_range_end":"","msr_series":"","msr_volume":"","msr_copyright":"","msr_conference_name":"","msr_doi":"10.1016\/j.ejmp.2026.105832","msr_arxiv_id":"","msr_mag_id":"","msr_other_authors":"","msr_other_contributors":"","msr_speaker":"","msr_award":"","msr_affiliation":"","msr_institution":"","msr_host":"","msr_version":"","msr_duration":"","msr_release_tracker_id":"","msr_highlight_type":"","msr_date_display_format":"","msr_main_download_label":"","msr_external_link_label":"","msr_doi_label":"","msr_published_date":"2026-08","msr_startdate":"","msr_presentation_date":"","msr_highlight_text":"","msr_notes":"","msr_longbiography":"","msr_publicationurl":"","msr_external_url":"","msr_secondary_video_url":"","msr_conference_url":"","msr_journal_url":"","msr_year":2026,"msr_month":8,"msr_day":0,"msr_microsoftintellectualproperty":false,"msr_pub_id":"","msr_publication_uploader":[{"type":"url","title":"https:\/\/www.sciencedirect.com\/science\/article\/abs\/pii\/S1120179726001195","label_id":243109,"id":false,"viewUrl":false},{"type":"url","title":"https:\/\/doi.org\/10.1016\/j.ejmp.2026.105832","label_id":243106,"id":false,"viewUrl":false},{"type":"url","title":"https:\/\/arxiv.org\/pdf\/2311.09614","label_id":243132,"id":false,"viewUrl":false}],"msr_related_uploader":[],"msr_original_fields_of_study":[],"msr_s2_paper_id":"","msr_s2_pdf_url":"","msr_citation_count_updated":"","msr_citation_count":0,"msr_influential_citations":0,"msr_reference_count":0,"msr_s2_open_access":false,"msr_s2_author_ids":[],"msr_pub_ids":[],"msr_hide_image_in_river":0,"footnotes":""},"msr-research-highlight":[],"research-area":[13556,13553],"msr-publication-type":[193715],"msr-publisher":[],"msr-publication-cta":[],"msr-focus-area":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[246985],"msr-conference":[],"msr-journal":[270515],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-1180878","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-artificial-intelligence","msr-research-area-medical-health-genomics","msr-locale-en_us","msr-field-of-study-medicine"],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2026-08","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":0,"msr_main_download":"","msr_publicationurl":"","msr_doi":"10.1016\/j.ejmp.2026.105832","msr_publication_uploader":[{"type":"url","title":"https:\/\/www.sciencedirect.com\/science\/article\/abs\/pii\/S1120179726001195","label_id":243109,"id":false,"viewUrl":false},{"type":"url","title":"https:\/\/doi.org\/10.1016\/j.ejmp.2026.105832","label_id":243106,"id":false,"viewUrl":false},{"type":"url","title":"https:\/\/arxiv.org\/pdf\/2311.09614","label_id":243132,"id":false,"viewUrl":false}],"msr_related_uploader":[],"msr_citation_count":0,"msr_citation_count_updated":"","msr_s2_paper_id":"","msr_influential_citations":0,"msr_reference_count":0,"msr_arxiv_id":"","msr_s2_author_ids":[],"msr_s2_open_access":false,"msr_s2_pdf_url":null,"msr_attachments":[],"msr-author-ordering":[{"type":"text","value":"Shadab Ahamed","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Yixi Xu","user_id":39775,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Yixi Xu"},{"type":"text","value":"Sara Kurkowska","user_id":0,"rest_url":false},{"type":"text","value":"Claire Gowdy","user_id":0,"rest_url":false},{"type":"text","value":"Joo H.O.","user_id":0,"rest_url":false},{"type":"text","value":"Ingrid Bloise","user_id":0,"rest_url":false},{"type":"text","value":"Don Wilson","user_id":0,"rest_url":false},{"type":"text","value":"Patrick Martineau","user_id":0,"rest_url":false},{"type":"text","value":"Fran&ccedil;ois B&eacute;nard","user_id":0,"rest_url":false},{"type":"text","value":"Fereshteh Yousefirizi","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Rahul Dodhia","user_id":41401,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Rahul Dodhia"},{"type":"user_nicename","value":"Juan M. Lavista Ferres","user_id":39552,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Juan M. Lavista Ferres"},{"type":"text","value":"William B Weeks","user_id":0,"rest_url":false},{"type":"text","value":"Carlos F. Uribe","user_id":0,"rest_url":false},{"type":"text","value":"Arman Rahmim","user_id":0,"rest_url":false}],"msr_impact_theme":[],"msr_research_lab":[],"msr_event":[],"msr_group":[696544],"msr_project":[778522],"publication":[],"video":[],"msr-tool":[],"msr_publication_type":"article","related_content":{"projects":[{"ID":778522,"post_title":"AI for Health","post_name":"ai-for-health","post_type":"msr-project","post_date":"2023-05-16 14:26:13","post_modified":"2024-10-14 15:42:21","post_status":"publish","permalink":"https:\/\/www.microsoft.com\/en-us\/research\/project\/ai-for-health\/","post_excerpt":"AI for Health is a philanthropic program launched by Microsoft, which aims to support nonprofits, researchers, and organizations working on global health challenges. The program provides access to artificial intelligence (AI) technology and expertise in three main areas: population health, imaging analytics, genomics &amp; proteomics.","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/778522"}]}}]},"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1180878","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":3,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1180878\/revisions"}],"predecessor-version":[{"id":1181553,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1180878\/revisions\/1181553"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1180878"}],"wp:term":[{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=1180878"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1180878"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=1180878"},{"taxonomy":"msr-publisher","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publisher?post=1180878"},{"taxonomy":"msr-publication-cta","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-cta?post=1180878"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=1180878"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1180878"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1180878"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=1180878"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=1180878"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=1180878"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1180878"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=1180878"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}