Machine programming (MP) is an emerging field at the intersection of deterministic and probabilistic computing, and it aims to assist software and hardware engineers, among other applications. Along with powerful compute resources, MP systems often rely on vast amount of open-source code to learn interesting properties about code and programming and solve problems in the areas of debugging, code recommendation, auto-completion, etc. Unfortunately, several of the existing MP systems either do not consider quality of code repositories or use atypical quality measures than those typically used in software engineering community to select them. As such, impact of quality of code repositories on the performance of these systems needs to be studied. In this preliminary paper, we evaluate impact of different quality repositories on the performance of a candidate MP system. Towards that objective, we develop a framework, named GitRank, to rank open-source repositories on quality, maintainability, and popularity by leveraging existing research on this topic. We then apply GitRank to evaluate correlation between the quality measures used by the candidate MP system and the quality measures used by our framework. Our preliminary results reveal some correlation between the quality measures used in GitRank and ControlFlag's performance, suggesting that some of the measures used in GitRank are applicable to ControlFlag. But it also raises questions around right quality measures for code repositories used in MP systems. We believe that our findings also generate interesting insights towards code quality measures that affect performance of MP systems.

Intel Labs has big plans for a software tool called ControlFlag that uses artificial intelligence to scan through code and pick out errors. One of those goals, perhaps way out in the future, is to bake it into chip packages as a last line of defense against faulty code. This could make the information flow on communications channels safer and efficient. Last week Intel open-sourced the tool – dubbed ControlFlag – to software developers. The software pores over lines of code and points out errors that developers can then fix.

You can view this file as a text file, if you want.

Let the OSS Enterprise newsletter guide your open source journey! Intel today open-sourced ControlFlag, a tool that uses machine learning to detect problems in computer code -- ideally to reduce the time required to debug apps and software. In tests, the company's machine programming research team says that ControlFlag has found hundreds of defects in proprietary, "production-quality" software, demonstrating its usefulness. "Last year, ControlFlag identified a code anomaly in Client URL (cURL), a computer software project transferring data using various network protocols over one billion times a day," Intel principal AI scientist Justin Gottschlich wrote in a blog post on LinkedIn. "Most recently, ControlFlag achieved state-of-the-art results by identifying hundreds of latent defects related to memory and potential system crash bugs in proprietary production-level software. In addition, ControlFlag found dozens of novel anomalies on several high-quality open-source software repositories."

Software debugging has been shown to utilize upwards of 50% of developers' time. Machine programming, the field concerned with the automation of software (and hardware) development, has recently made progress in both research and production-quality automated debugging systems. In this paper, we present ControlFlag, a system that detects possible idiosyncratic violations in software control structures. ControlFlag also suggests possible corrections in the event a true error is detected. A novelty of ControlFlag is that it is entirely self-supervised; that is, it requires no labels to learn about the potential idiosyncratic programming pattern violations. In addition to presenting ControlFlag's design, we also provide an abbreviated experimental evaluation.