Automatic program analysis and evaluation

There is currently considerable interest in the computing community in the evaluation of computer programming. However, in order to objectively evaluate such concepts, it is necessary to undertake a thorough evaluation of the programming process itself. Most previous studies of this type have analyzed, by hand usually, a few instances of programs. This has led to some general conjectures; however, the amount of information that must be processed precludes any large scale analysis. In order to avoid this problem, an automatic data collection facility has been implemented as part of a PL/1 compiler at the University of Maryland. This system automatically collects information on each program that has been compiled - at almost no additional cost to the user of the compiler. This paper will describe the system and will evaluate some of the characteristics of some of the 25,000 programs that have been run since July, 1975.