Decision tree ensembles are among the most commonly used machine learning models. These models are used in a wide range of applications and are deployed at scale. Decision tree ensemble inference is usually performed with libraries such as XGBoost, LightGBM, and Sklearn. These libraries incorporate a fixed set of optimizations for the hardware targets they support. However, maintaining these optimizations is prohibitively expensive with the evolution of hardware. Further, they do not specialize the inference code to the model being used, leaving significant performance on the table. This paper presents TREEBEARD, an optimizing compiler that progressively lowers the inference computation to optimized CPU code through multiple intermediate abstractions. By applying model-specific optimizations at the higher levels, tree walk optimizations at the middle level, and machine-specific optimizations lower down, TREEBEARD can specialize inference code for each model on each supported CPU target. TREEBEARD combines several novel optimizations at various abstraction levels to mitigate architectural bottlenecks and enable SIMD vectorization of tree walks. We implement TREEBEARD using the MLIR compiler infrastructure and demonstrate its utility by evaluating it on a diverse set of benchmarks. TREEBEARD is significantly faster than state-of-the-art systems, XGBoost, Treelite and Hummingbird, by 2.6\u00d7, 4.7\u00d7 and 5.4\u00d7 respectively in a single-core execution setting, and by 2.3\u00d7, 2.7\u00d7 and 14\u00d7 respectively in multi-core settings.<\/p>\n","protected":false},"excerpt":{"rendered":"
Decision tree ensembles are among the most commonly used machine learning models. These models are used in a wide range of applications and are deployed at scale. Decision tree ensemble inference is usually performed with libraries such as XGBoost, LightGBM, and Sklearn. These libraries incorporate a fixed set of optimizations for the hardware targets they […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[{"type":"text","value":"Ashwin Prasad","user_id":0},{"type":"user_nicename","value":"Sampath Rajendra","user_id":"43107"},{"type":"user_nicename","value":"Kaushik Rajan","user_id":"32574"},{"type":"text","value":"R Govindarajan","user_id":0},{"type":"text","value":"Uday Bondhugula","user_id":0}],"msr_publishername":"","msr_publisher_other":"","msr_booktitle":"","msr_chapter":"","msr_edition":"","msr_editors":"","msr_how_published":"","msr_isbn":"","msr_issue":"","msr_journal":"","msr_number":"","msr_organization":"IEEE","msr_pages_string":"","msr_page_range_start":"","msr_page_range_end":"","msr_series":"","msr_volume":"","msr_copyright":"","msr_conference_name":"IEEE\/ACM International Symposium on Microarchitecture (MICRO)","msr_doi":"","msr_arxiv_id":"","msr_mag_id":"","msr_other_authors":"","msr_other_contributors":"","msr_speaker":"","msr_award":"","msr_affiliation":"","msr_institution":"","msr_host":"","msr_version":"","msr_duration":"","msr_release_tracker_id":"","msr_highlight_type":"","msr_date_display_format":"","msr_main_download_label":"","msr_external_link_label":"","msr_doi_label":"","msr_published_date":"2022-10-01","msr_startdate":"","msr_presentation_date":"","msr_highlight_text":"","msr_notes":"","msr_longbiography":"","msr_publicationurl":"","msr_external_url":"","msr_secondary_video_url":"","msr_conference_url":"https:\/\/microarch.org\/micro55\/index.php","msr_journal_url":"","msr_year":2022,"msr_month":10,"msr_day":1,"msr_microsoftintellectualproperty":false,"msr_pub_id":"","msr_publication_uploader":[{"type":"url","viewUrl":"false","id":false,"title":"https:\/\/ieeexplore.ieee.org\/document\/9923840","label_id":243109,"label":0}],"msr_related_uploader":[],"msr_original_fields_of_study":[],"msr_s2_paper_id":"","msr_s2_pdf_url":"","msr_citation_count_updated":"","msr_citation_count":0,"msr_influential_citations":0,"msr_reference_count":0,"msr_s2_open_access":false,"msr_s2_author_ids":[],"msr_pub_ids":[],"msr_hide_image_in_river":0,"footnotes":""},"msr-research-highlight":[],"research-area":[13563],"msr-publication-type":[193716],"msr-publisher":[],"msr-publication-cta":[],"msr-focus-area":[],"msr-locale":[268875],"msr-post-option":[],"msr-field-of-study":[],"msr-conference":[],"msr-journal":[],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-980577","msr-research-item","type-msr-research-item","status-publish","hentry","msr-research-area-data-platform-analytics","msr-locale-en_us"],"msr_publishername":"","msr_edition":"","msr_affiliation":"","msr_published_date":"2022-10-01","msr_host":"","msr_duration":"","msr_version":"","msr_speaker":"","msr_other_contributors":"","msr_booktitle":"","msr_pages_string":"","msr_chapter":"","msr_isbn":"","msr_journal":"","msr_volume":"","msr_number":"","msr_editors":"","msr_series":"","msr_issue":"","msr_organization":"IEEE","msr_how_published":"","msr_notes":"","msr_highlight_text":"","msr_release_tracker_id":"","msr_original_fields_of_study":"","msr_download_urls":"","msr_external_url":"","msr_secondary_video_url":"","msr_longbiography":"","msr_microsoftintellectualproperty":0,"msr_main_download":"","msr_publicationurl":"","msr_doi":"","msr_publication_uploader":[{"type":"url","viewUrl":"false","id":"false","title":"https:\/\/ieeexplore.ieee.org\/document\/9923840","label_id":"243109","label":0}],"msr_related_uploader":[],"msr_citation_count":0,"msr_citation_count_updated":"","msr_s2_paper_id":"","msr_influential_citations":0,"msr_reference_count":0,"msr_arxiv_id":"","msr_s2_author_ids":[],"msr_s2_open_access":false,"msr_s2_pdf_url":null,"msr_attachments":[],"msr-author-ordering":[{"type":"text","value":"Ashwin Prasad","user_id":0,"rest_url":false},{"type":"user_nicename","value":"Sampath Rajendra","user_id":43107,"rest_url":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Sampath Rajendra"},{"type":"user_nicename","value":"Kaushik Rajan","user_id":32574,"rest_url":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/microsoft-research\/v1\/researchers?person=Kaushik Rajan"},{"type":"text","value":"R Govindarajan","user_id":0,"rest_url":false},{"type":"text","value":"Uday Bondhugula","user_id":0,"rest_url":false}],"msr_impact_theme":[],"msr_research_lab":[199562,199565],"msr_event":[],"msr_group":[144939],"msr_project":[967329],"publication":[],"video":[],"msr-tool":[],"msr_publication_type":"inproceedings","related_content":{"projects":[{"ID":967329,"post_title":"Domain Specialization","post_name":"domain-specialization","post_type":"msr-project","post_date":"2023-10-16 02:14:29","post_modified":"2026-06-24 12:15:52","post_status":"publish","permalink":"https:\/\/find.codeghost.online\/en-us\/research\/project\/domain-specialization\/","post_excerpt":"Scaling performance beyond Moore's law Note: This research project has reached its conclusion. These pages are maintained for reference and archival purposes. Domain\u00a0specialization\u00a0is\u00a0expected\u00a0to\u00a0play\u00a0a\u00a0big\u00a0role\u00a0in\u00a0how\u00a0computer\u00a0systems\u00a0evolve\u00a0in\u00a0future. With\u00a0the\u00a0end\u00a0of\u00a0Moore's Law, we\u00a0are\u00a0already\u00a0seeing\u00a0CPU,\u00a0GPU\u00a0and domain specific\u00a0hardware\u00a0evolving\u00a0rapidly. The next decade\u00a0is\u00a0therefore expected\u00a0to see\u00a0big\u00a0changes\u00a0in\u00a0how\u00a0we\u00a0develop,\u00a0compile\u00a0and\u00a0run\u00a0software. This project focuses on data systems, a class of systems where, as the data sizes grow, performance scaling is going to be of importance.First,\u00a0we\u00a0believe\u00a0that domain-specific\u00a0compilers\u00a0will\u00a0play\u00a0a\u00a0crucial\u00a0strategic\u00a0role\u00a0in\u00a0helping\u00a0software\u00a0leverage\u00a0the changing hardware landscape. Such compilers will be multi-layered and will progressively lower computation through multiple intermediate abstractions,…","_links":{"self":[{"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/967329"}]}}]},"_links":{"self":[{"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/980577","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item"}],"about":[{"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-research-item"}],"version-history":[{"count":2,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/980577\/revisions"}],"predecessor-version":[{"id":999144,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-research-item\/980577\/revisions\/999144"}],"wp:attachment":[{"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/media?parent=980577"}],"wp:term":[{"taxonomy":"msr-research-highlight","embeddable":true,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-research-highlight?post=980577"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=980577"},{"taxonomy":"msr-publication-type","embeddable":true,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-type?post=980577"},{"taxonomy":"msr-publisher","embeddable":true,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-publisher?post=980577"},{"taxonomy":"msr-publication-cta","embeddable":true,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-publication-cta?post=980577"},{"taxonomy":"msr-focus-area","embeddable":true,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-focus-area?post=980577"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=980577"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=980577"},{"taxonomy":"msr-field-of-study","embeddable":true,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-field-of-study?post=980577"},{"taxonomy":"msr-conference","embeddable":true,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-conference?post=980577"},{"taxonomy":"msr-journal","embeddable":true,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-journal?post=980577"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=980577"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/find.codeghost.online\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=980577"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}