CCM - A fundamental Process for Improving Quality
Clemens Gasser Joanneum Research, Graz, Austria Edwin Deutschl Joanneum Research, Graz, Austria.
The goals of the Process Improvement Experiment (PIE) were to shorten the software developing cycle, to increase the reuse of source code by introducing a well defined Software Developing Process (SDP) and a CCM. Faster maintenance by better reproducibility and readability of code and by a better documentation were further goals of the project.
Up to now, the SDP and the CCM-plan were defined and introduced to the baseline project DIBIT, a photogrammetic measuring system for the documentation of tunnel advances and underground constructions. This project is in an intensive maintenance phase, with steady new releases and variants for different customers and therefore best suited for an experiment with CCM. In the moment the measurement phase of the PIE is in progress.
First results and key lessons learned are, that the introduction of the CCM-tool was more time consuming than planned and that it is very hard to find expressive metrics.
The next proposed actions are the consolidation of the introduced processes and better documentation of them. But the focus will lie on the metrics, which data gathering process is in full progress. International and internal dissemination are further actions to propose.
Most of the systems are built as prototypes, which are adapted to the customers and their needs. In recent years systems were placed on the market, that were produced in a higher number of units. This required, in addition to normal maintenance for a prototype, further development and adoption to customer specific variants.
Additionally, most of the systems require a software development that depends extremely on hardware restrictions (Intelligent cameras, framegrabber, sensor interfaces, SPS process control, heavy duty environments, etc). Several systems have to fulfill real-time requirements (e.g. optical inspection systems in production lines). This special requirements need a well defined and quality oriented software engineering.
The experiment should help to get closer to a high level software engineering at the department by introducing the key process Configuration Management as an important supporting activity in software development.
The state of software development within the department Industrial Image Analysis at the beginning of the experiment depends strongly on the initiative of the individual project leaders. No common guidelines or defined processes existed for the entire department. The major problems, like exploding budgets or overrun milestones, were partially lead back to inefficient software development.
An overview of the strengths and weaknesses in software development at the starting point of the IECS was received by a SynQuest-Assessment, which is based on the Bootstrap method [13]. These assessments are performed at the beginning and at the end of the experiment. They are used to establish the impacts of the newly introduced processes in the attitude and behaviour of the employees. The guided SynQuest assessment showed the following results concerning strengths and weaknesses of the organisational unit.
Fig. CGAS. 1: Results of the SynQuest assessment – Process fields
Fig. CGAS. 2: Results of the SynQuest assessment – Process attributes
Fig. CGAS. 1 and Fig. CGAS. 2 show the average results of four project groups, each containing three or four employees.
Apart from common organisational aspects the maturity of the software developing process could be judged as ad-hoc and therefore comparable to a CMM level 1.
Split up into the following different points of view the analysis of the strengths and weaknesses brought up:
Shorter software development cycles to reduce personal and resource costs (obviously a very common objective).
This section describes in detail the working plans and their implementation in the ESSI-PIE.
The IECS project manager co-operates intensively with the project manager of the baseline project as well with the baseline project team in the sense of the objectives of the experiment.
It is worth mentioning, that a lean organisational structure has been chosen in respect to the limited personal resources of our department as an SME.
Fig. CGAS. 3: Organisational structure
MKS Source Integrity [11] was selected as CCM – Tool. It was evaluated as best fitting to our criterions for a configuration management tool based on the CCM - plan. The following important requirements were identified to be served by the tool:
For metric purposes the tool Provista from 3Soft was installed. As metric databases MS-Access and MS-Excel are in use. For documentation of both customer and internal requests a MS-Access database was designed and installed.
Table 1: Evaluated CCM - Tools
Most of the training for an efficient use of the CCM Tools was done by training on the job of the concerned staff members. A preceding self-training, by studying the manuals, making pilot-experiments with example-projects and contacting the software supplier in difficult questions, puts one person into ability to train the others.
Self-training can be done by every employee by studying the guidelines and documented processes either by the GroupWare tool LinkWorks or by the Intranet DIBIS. Two people were also trained by an external consultant into an overview in the methods and practises of CCM.
An other external consultant was engaged for supplying the guided SynQuest Assessment. He guided the whole assessment and worked out the documentation of the results. In a separate session he presented the results to the employees of the department.
Contacts to EC, periodic reporting, planning, communication
Holding of a guided SynQuest assessment
Definition of the phases of the Software developing process
Definition of the CCM-Plan and requirements for a CCM-Tool
Work with the baseline project, definition of measurements
Holding of a guided SynQuest assessment and comparison with the initial assessment
Writing of final reports and papers and preparation for dissemination events.
The work, which is actually performed, will be described in the following section.
Project planning: The first phase of the IECS was characterised by project planning and settling into the topics of the experiment. That means, that adequate personal resources were fixed and a detailed work-plan was performed. The installed project manager was not involved in acquiring the PIE, so there was an additional amount to get into the project topics. Special literature had to be obtained and had to be studied.
Basic Assessment: In parallel the initial basic SynQuest Assessment was organised. The assessment was guided by a professional consultant of HM&S, the company which developed the SynQuest tool. The experience and results of keeping a lot of guided assessments in other companies provides a possibility to compare the own maturity-level in software developing with that of other organisations. The consultant showed this comparison at the presentation of the assessment results. The average results lies clearly over the average result of all assessed organisations with the exceptions CCM, metrics and testing.
SDP definition: Also in parallel the SDP was worked out and documented in the PES. For practicability it was attempted to keep it as compact and clear as possible. The formulation of the PES was done with respect to practices and standards already used by the engineers. The resulting subdivision of phases (see Table 2) is related with the classical Waterfall Lifecycle model. It is still used by a wide area of standards and guidelines (see ISO 9000-3 [9], TickIt [5], CMM [8] and the summary of [14]). As part of the SDP Coding conventions were worked out. They define a uniform appearance of the source code and are therefore an essential method to increase the software readability.
Table 2: The phases of the software developing process (SDP)
CCM-Process definition: The phase of defining the CCM-Process was supported by an external consultant. In initial sessions the CCM-requirements depending on the SDP were defined. Individual CCM – activities were discussed and documented in a CCM – plan (see Table 3), to which various standards gave helpful hints. Again the guidelines of ISO 9000-3 [9], TickIt [5] and CMM [8] were taken into account and used as templates.
Table 3: Overview of the CCM-plan
CCM - Tool selection, installation, training: The next phase was the evaluation and selection of an adequate CCM – tool. First a list of potential tools and their suppliers were prepared. The evaluation was based on the inspection of demo versions, documentation material and information received from the tool suppliers. The best rating on this evaluation was accomplished by the tool packet Source Integrity by MKS [11]. Later good references for CCM – tools were found in the Internet, which get a more and more valuable information pool (see [15] as a very extensive storehouse for configuration management). The installation and initial administration of the CCM - tool spent more time than estimated. A lot of small problems (technical but also in comprehension) must be taken into account at introducing a totally new tool.
Experiments, improvements and surveillance: The following phase concerned the work with the baseline project. The originally provided baseline project had to be dropped, because it fells in a status, where no further development. Therefore it was necessary to take an other baseline project. A project for an optical 3D-surface reconstructing tool was selected. This project is called DIBIT (Digital Image Observation System for Tunnelling). The DIBIT tunnel scanner (see the system in practical use in Fig. CGAS. 4) is a photogrammetic measuring system for the documentation of tunnel advances and underground constructions.
Fig. CGAS. 4: The DIBIT system in practical use.
It was developed during the last four years at our institute. At the moment there are three delivered systems in practical use. Each system has varying configurations for the different tunnels and requirements of the customers. Furthermore there must be a handling of error requests and new requirements, which come in permanently. The main problems to solve concern to configuration management.
After the approval for the exchange of the baseline projects the initial work for the new baseline project had to be performed. The theoretically outworked measurement program was adapted to practical use for the experiment with the new baseline project. For evaluation of appropriate metrics the GQM (Goal - Question - Measurement) method [14], [12] was used. Four classes of metrics together with the results of the SynQuest assessment form the essential points of the metric program (see Table 4):
QO.1 defines the transparency in amount of time for different tasks, performed either for IECS and the baseline project. The spent time is recorded in Excel forms (see Fig. CGAS. 5) by all involved employees. The forms are analysed each month and stored in a database.
Transparency in amount of time
Efficiency in maintenance
Efficiency of configuration management
Higher reuse of source code
Table 4: Defined metrics
QO.2 is concerned with the efficiency of maintenance. The three quality characteristics "Handling of customer requests", "SW – documentation" and "SW – complexity" are measured by number and duration of processing customer requests, by analysing the documentation and source code comments. All customer requests just as internal detected bugs and faults are registered.
QO.3 bother the efficiency of configuration management. Quality characteristics are the intensity of use of the CCM – tool (e.g. see Fig. CGAS. 6) and the documentation of the CIs.
QO.4 defines a quality objective of higher reuse of source code. It will be measured at the end of the project by the number of library modules, which are used by other
Additionally the results of the SynQuest assessment (see Fig. CGAS. 1 and Fig. CGAS. 2) will be used as an overall metric concerned to the subjective opinion about a performed improvement.
Fig. CGAS. 5: Example of a time recording form
Fig. CGAS. 6: Number of Check-In’s in DIBIT (ME.3.2.1)
The actual state of the measurement activities is described by intensive data gathering. The technical conditions and infrastructure were installed. The tool Provista/QS together with MKS Source Integrity supports the analysis of source-code, Releases and CIs. All data out of these tools are collected in central MS-Access databases.
Final assessment - Reporting and dissemination: The final phases of the experiment represent the Final SynQuest Assessment, which should provide a comparison to the initial assessment, and dissemination at international just as at internal events.
In the moment there exists not so many quantitative data to publish, establishing the results came out from the PIE. Especially trends are – at the moment - hard to observe. Therefore the time of data gathering is to short.
Nevertheless, we recognised, that the introduction of metrics is absolutely necessary. It initiates new sights on different areas of the software developing process. And the famous sentence from Tom DeMarco [3] "You cannot control what you cannot measure.'' is hard to disprove.
From a technical point of view, the new tools and defined processes, have a deep influence on practical working of the employees. In some cases the benefits in daily work is obvious and gets from there a great acceptance. E.g. the CCM-Tool integration in the developing environment enhances the version management compared to earlier hand made versioning, e.g. organised in directory structures.
The CCM-Tool makes it possible for the first time to reuse software components, which is one important objectives of the PIE. Before the experiment no common software pool was installed. A lot of different versions and variants of software components were spread over local PCs and local directories on workstations. In the meantime two common libraries are in the repository (17020 LOC in 51 modules and 1495 LOC in 12 modules), which are of great interest for other projects. QO.4 will measure the reuse of such libraries in other projects.
Also the Intranet DIBIS, as information pool, has some similar benefits. Before IECS, documentation spread over different platforms and file-formats, which made it impossible to integrate them. With the HTML – standard this problem can be solved. All the advantages from an Intranet can be used under a homogeneous environment. The acceptance of DIBIS is growing and can be measured by the access count of the Internet Server.
The very common objective of shorter development cycles is also influenced in a hard measurable manner by the CCM – tool as a repository. Indirect by the now possible reuse of software – components and directly by accessing a well defined baseline stored in the repository. Before the experiment, available components had to be searched at different places and in the care of other employees.
The request database provides an easy to use and automated tool, to document all requests. It was therefore well accepted by the baseline–project staff. There existed no structured documentation or recording of customer requests or internally detected bugs in the past. The actual statistic of recorded request (e.g. 22 entries in May, 39 in June, 14 in July, 21 in August, 24 in September, 5 in October) can be used to illustrate the improvement triggered by the new introduced technologies. Before recording requests, some of them were easily forgotten and had therefore not enhanced the confidence of the customers.
The improvement of the business matters of the baseline-project was the major goal specified by the project manager of DIBIT, however a major impact on the business operation is not yet measurable.
The expenditure for installing and maintaining the CCM – tool was higher than calculated. It took some time to cope with some initial problems. An experiment with a small pilot project or better, but more expensive, the commissioning of the tool supplier to installation and training could reduce time. About 143 person days (only 60 were planned) were used for installation and training and solving initial problems.
The more detailed time recordings supports the project controlling and thereby indirect the project business results. For the quality objective QO.1 (see Table 4) the record of time spent for all project tasks is unavoidable. Before the experiment only monthly time recordings had to be prepared. For the internal use a more detailed recording scheme was installed. Based on MS-Excel a detailed time recording form had to be filled out daily (see Fig. CGAS. 5). All times spend for work packages defined in the DIBIT project had to be recorded. For some employees this was a great readjustment in their working practices, other uses this forms now for their whole time recording.
The now available common repository showed us the strength’s of parts of our software. Currently we are engaged in a negotiations, where we have the opportunity to sell great parts of our image processing software as a library for a hardware manufacturer.
Support and resistance of concerned employees depends strongly on the individual. People with higher responsibilities and more experience in handling problems are more interested but also more critical than people with lower responsibilities. The greatest resistance raised at the programmer’s level.
Resistance could best overcome by convincing that changes are necessary and that the initial overhead (e.g. administrative, settling into new tools) becomes smaller and the benefits of the new methods will predominate in long term. Several times there exist convincing arguments with a practical context. Therefore only examples and data in good literature can taken to argue.
It has turned out, that it is very important to include all concerned people into the decision making process. A commanding style doesn’t work really today.
6. Lessons learned
The following lessons have we learned up to now. Maybe some of them will be overruled in the last time of the experiment.
Technological point of view
Business point of view
Strengths and weaknesses of the experiment
Without listing quantitative data, the most objectives seems to be achieved. Especially the software reuse and the configuration and change management goals are obviously achieved. Also documentation and the higher degree of reproducibility and readability of code has been improved. Most of the concerned employees are high motivated and very interested in the new methods and technologies.
Many of the processes introduced in the first half of the PIE have to be consolidated during the second half of the experiment. Especially the documentation of the new procedures must be improved. Precise, but if possible not voluminous descriptions of procedures, templates and checklists shall make it possible to support others than the baseline project, which stands in a privileged position to the PIE.
The higher degree of automation of the metric data gathering process, but also the exploitation and interpretation of the gathered data must be improved.
Appendix I: About the authors
Clemens Gasser is a staff member of 3D Vision Group and some other project groups at the JOANNEUM RESEARCH Industrial Analysis department in the Institute of Digital Image Processing (DIB). He is Diploma Engineer in Telematics (University of Technology Graz) on studies in the field of artificial intelligence (neural networks, fuzzy reasoning and control) and industrial image analysis. He has collected experience on DSP, parallel computing, software engineering and software quality. In 1995 Clemens Gasser received his Diploma with thesis on intelligence adaptive fuzzy control systems. Since May 1995 he is a fixed employee at JOANNEUM RESEARCH. Between 1995 – 1997 he leads and assists in different industrial image processing projects. He was a staff member of the project team, which developed a system for stereo-vision based tunnel surface reconstruction (DIBIT). Since March 1997 he is the project manager of the ESSI-PIE IECS. This ESPRIT - project (Nr. 23760) experiments with the Introduction of CCM into the software developing cycle. Since September 1998 he works as internal auditor of the ISO 9001 QM system.
Edwin Deutschl is a staff member of the Linescan Vision Group at the JOANNEUM RESEARCH Industrial Analysis department in the Institute of Digital Image Processing (DIB). He studied Applied Mathematics at the University of Technology in Graz, Austria. In 1994 he received his Diploma with a thesis on light reflections. Since 1995 he works at the JOANNEUM RESEARCH Center in Graz as application developer and project manager of Online Quality Assurance Systems. His research interests are in real time and parallel computing for applications in wood- and pharmaceutical industry as well as related Software Engineering issues. Since 1997 he is a permanent member of the Software Quality Group at the Institute of Digital Image Processing.
Appendix II: About JOANNEUM RESEARCH
JOANNEUM RESEARCH is one of the largest technology centres in central Europe, making its expertise available to business, industry and administration. Our highly-qualified staff of 300 people that work in 20 research units implement their know-how in all sectors of innovation, both at national and in international levels. Our service includes specifically geared development tasks for small- and medium-sized companies, complex interdisciplinary national and international research assignments as well as tailored techno-economic consulting. Our highest priority in all our activities is to meet the top quality standards demanded by our clients.
References