As agentic AI systems move from research prototypes into large-scale production deployments, a critical evaluation gap has emerged: existing methodologies focus primarily on pre-deployment capability assessment, while offering limited support for post-deployment monitoring, model evolution risk, and production governance. Recent surveys show that agent evaluations remain dominated by technical capability metrics, with substantially less attention to human-centered, safety, economic, and lifecycle-oriented dimensions. This workshop addresses this gap by advancing evaluation and trustworthiness methodologies across the full agentic AI lifecycle, with particular emphasis on real-time monitoring, API-driven model drift, stochastic behavior assessment, and regulatory compliance. Building on three consecutive KDD workshops on AI evaluation from 2023 to 2025, the workshop will bring together researchers, practitioners, and policymakers to develop scalable and regulation-aware frameworks that help ensure autonomous AI systems remain reliable, accountable, and safe throughout their operational lifetime.<\/p>\n","protected":false},"excerpt":{"rendered":"
As agentic AI systems move from research prototypes into large-scale production deployments, a critical evaluation gap has emerged: existing methodologies focus primarily on pre-deployment capability assessment, while offering limited support for post-deployment monitoring, model evolution risk, and production governance. Recent surveys show that agent evaluations remain dominated by technical capability metrics, with substantially less attention […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[{"type":"text","value":"Yuan Ling","user_id":0},{"type":"text","value":"Shujing Dong","user_id":0},{"type":"text","value":"Yarong Feng","user_id":0},{"type":"user_nicename","value":"Sadid Hasan","user_id":"42063"},{"type":"text","value":"George Karypis","user_id":0},{"type":"user_nicename","value":"Chandan K A Reddy","user_id":"39814"}],"msr_publishername":"","msr_publisher_other":"","msr_booktitle":"","msr_chapter":"","msr_edition":"","msr_editors":"","msr_how_published":"","msr_isbn":"","msr_issue":"","msr_journal":"","msr_number":"","msr_organization":"ACM","msr_pages_string":"","msr_page_range_start":"","msr_page_range_end":"","msr_series":"","msr_volume":"","msr_copyright":"","msr_conference_name":"Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2","msr_doi":"10.1145\/3770855.3818237","msr_arxiv_id":"","msr_mag_id":"","msr_other_authors":"","msr_other_contributors":"","msr_speaker":"","msr_award":"","msr_affiliation":"","msr_institution":"","msr_host":"","msr_version":"","msr_duration":"","msr_release_tracker_id":"","msr_highlight_type":"","msr_date_display_format":"","msr_main_download_label":"","msr_external_link_label":"","msr_doi_label":"","msr_published_date":"2026-08-08","msr_startdate":"","msr_presentation_date":"","msr_highlight_text":"","msr_notes":"","msr_longbiography":"","msr_publicationurl":"","msr_external_url":"","msr_secondary_video_url":"","msr_conference_url":"","msr_journal_url":"","msr_year":2026,"msr_month":8,"msr_day":8,"msr_microsoftintellectualproperty":false,"msr_pub_id":"f1e8d09c5fe7b163dccf2f0e3f6472620d9b9f5f","msr_publication_uploader":[{"type":"url","title":"https:\/\/dl.acm.org\/doi\/abs\/10.1145\/3770855.3818237","label_id":243109,"id":false,"viewUrl":false}],"msr_related_uploader":[],"msr_original_fields_of_study":[],"msr_s2_paper_id":"f1e8d09c5fe7b163dccf2f0e3f6472620d9b9f5f","msr_s2_pdf_url":"https:\/\/doi.org\/10.1145\/3770855.3818237","msr_citation_count_updated":"","msr_citation_count":0,"msr_influential_citations":0,"msr_reference_count":2,"msr_s2_open_access":true,"msr_s2_author_ids":[],"msr_pub_ids":[{"provider":"s2","id":"f1e8d09c5fe7b163dccf2f0e3f6472620d9b9f5f"},{"provider":"doi","id":"10.1145\/3770855.3818237"},{"provider":"corpusid","id":"290858074"}],"msr_hide_image_in_river":0,"footnotes":""},"msr-research-highlight":[],"research-area":[13556],"msr-publication-type":[193716],"msr-publisher":[],"msr-publication-cta":[],"msr-focus-area":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[],"msr-conference":[270543],"msr-journal":[270542],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-1184496","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-artificial-intelligence","msr-locale-en_us"],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2026-08-08","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"ACM","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":0,"msr_main_download":"","msr_publicationurl":"","msr_doi":"10.1145\/3770855.3818237","msr_publication_uploader":[{"type":"url","title":"https:\/\/dl.acm.org\/doi\/abs\/10.1145\/3770855.3818237","label_id":243109,"id":false,"viewUrl":false}],"msr_related_uploader":[],"msr_citation_count":0,"msr_citation_count_updated":"","msr_s2_paper_id":"f1e8d09c5fe7b163dccf2f0e3f6472620d9b9f5f","msr_influential_citations":0,"msr_reference_count":2,"msr_arxiv_id":"","msr_s2_author_ids":[],"msr_s2_open_access":true,"msr_s2_pdf_url":"https:\/\/doi.org\/10.1145\/3770855.3818237","msr_attachments":[],"msr-author-ordering":[{"type":"text","value":"Yuan Ling","user_id":0,"rest_url":false},{"type":"text","value":"Shujing Dong","user_id":0,"rest_url":false},{"type":"text","value":"Yarong Feng","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Sadid Hasan","user_id":42063,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Sadid Hasan"},{"type":"text","value":"George Karypis","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Chandan K A Reddy","user_id":39814,"rest_url":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Chandan K A Reddy"}],"msr_impact_theme":[],"msr_research_lab":[],"msr_event":[],"msr_group":[],"msr_project":[],"publication":[],"video":[],"msr-tool":[],"msr_publication_type":"inproceedings","related_content":[],"_links":{"self":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1184496","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":2,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1184496\/revisions"}],"predecessor-version":[{"id":1184740,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/1184496\/revisions\/1184740"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1184496"}],"wp:term":[{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=1184496"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1184496"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=1184496"},{"taxonomy":"msr-publisher","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publisher?post=1184496"},{"taxonomy":"msr-publication-cta","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-cta?post=1184496"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=1184496"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1184496"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1184496"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=1184496"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=1184496"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=1184496"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1184496"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/www.microsoft.com\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=1184496"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}