Assessing the Quality of GitHub-Generated SBOMs: A Detailed Bill of Health
Grant Enderson, Nathan Wintersgill, Trevor Stalnaker, Oscar Chaparro, and Denys Poshyvanyk
Proceedings of the 26th IEEE International Conference on Source Code Analysis and Manipulation (SCAM'26)
Research Track, pp. (to appear), 2026
Abstract: Software Bills of Materials (SBOMs) are emerging as vital tools for software supply chain management, providing a means of tracking dependencies with their licenses and promoting transparency. Despite increased regulatory pressure for these documents, accompanying tool support is still nascent. Recently, GitHub introduced SBOM generation for repositories via its built-in dependency tracking system. In this paper, we investigate this tool to ascertain its efficacy in generating accurate SBOMs, identifying the causes of mistakes in generated SBOMs and reasoning about challenges currently facing SBOM generation. We create a dataset of 50 manually-verified SBOMs for Python projects and compare them against those generated by GitHub, identifying areas where the tool struggles. Our results shed light on the GitHub tool’s strengths and weaknesses and suggest directions for future improvements.