Empirical Software Engineering Issues. Critical Assessment and Future Directions: International Workshop, Dagstuhl Castle, Germany, June 26-30, 2006, Revised Papers (Lecture Notes in Computer Science)

Victor R. Basili, Dieter Rombach, and Kurt Schneider Introduction In 1992, a Dagstuhl seminar was held on “Experimental Software Engineering Issues” (seminar no. 9238). Its goal was to discuss the state of the art of empirical software engineering (ESE) by assessing past accomplishments, raising open questions, and proposing a future research agenda. Since 1992, the topic of ESE has been adopted more widely by academia as an interesting and promising research topic, and in industrial practice as a necessary infrastructure technology for goal-oriented, sustained process improvement. At the same time, the spectrum of methods applied in ESE has broadened. For example, in 1992, the empirical methods applied in software engineering were basically restricted to quantitative studies (mostly controlled experiments), whereas since then, a range of qualitative methods have been introduced, from observational to ethnographical studies. Thus, the field can be said to have moved from experimental to empirical software engineering. We believe that it is now time to again bring together practitioners and researchers to identify both the progress made since 1992 and the most important challenges for the next five to ten years. Front Matter....Pages - The Empirical Paradigm Introduction ....Pages 1-3 Techniques for Empirical Validation....Pages 4-9 Status of Empirical Research in Software Engineering....Pages 10-19 Aggregation of Empirical Evidence....Pages 20-20 Empirical Evaluation in Software Engineering: Role, Strategy, and Limitations....Pages 21-21 New Opportunities for Empirical Research....Pages 22-22 Empirical Paradigm: Position Paper....Pages 23-23 The Value of Empirical Evidence for Practitioners and Researchers....Pages 24-24 Empirical Paradigm – The Role of Experiments....Pages 25-32 The Role of Controlled Experiments in Software Engineering Research....Pages 33-37 Creating Real Value in Software Engineering Experiments....Pages 38-38 From Silver Bullets to Philosophers’ Stones: Who Wants to Be Just an Empiricist?....Pages 39-39 Social and Human Aspects of Software Engineering....Pages 40-40 Longitudinal Studies in Evidence-Based Software Engineering....Pages 41-41 The Use of Grounded Theory in Empirical Software Engineering....Pages 42-42 Exploration and Confirmation: An Historical Perspective....Pages 43-49 Combining Study Designs and Techniques Working Group Results ....Pages 50-53 Optimizing Return-On-Investment (ROI) for Empirical Software Engineering Studies Working Group Results ....Pages 54-57 The Role of Controlled Experiments Working Group Results ....Pages 58-62 The Empirical Paradigm Discussion and Summary ....Pages 63-67 Measurement and Model Building Introduction ....Pages 68-69 Data Collection, Analysis, and Sharing Strategies for Enabling Software Measurement and Model Building....Pages 70-76 Knowledge Acquisition in Software Engineering Requires Sharing of Data and Artifacts....Pages 77-82 Effective Data Interpretation....Pages 83-90 Software Support Tools and Experimental Work....Pages 91-99 Measurement and Interpretation of Productivity and Functional Correctness....Pages 100-100 Synthesising Research Results....Pages 101-101 On the Quality of Data....Pages 102-102 Potential of Open Source Systems as Project Repositories for Empirical Studies Working Group Results ....Pages 103-107 Data Sharing Enabling Technologies Working Group Results ....Pages 108-110 Documenting Theories Working Group Results ....Pages 111-114 Measurement and Model Building Discussion and Summary ....Pages 115-120 Technology Transfer and Education Introduction ....Pages 121-124 Empirical Studies as a Basis for Technology Transfer....Pages 125-127 Relationships and Responsibilities of Software Experimentation....Pages 128-128 The (Practical) Importance of SE Experiments....Pages 129-129 How to Improve the Use of Controlled Experiments as a Means for Early Technology Transfer....Pages 130-130 Extending Empirical Studies to Cover More Realistic Industrial Development and Project Management Issues....Pages 131-131 Empirical Case Studies in Industry: Some Thoughts....Pages 132-132 Software Process Simulation Frameworks in Support of Packaging and Transferring Empirical Evidence....Pages 133-133 Structuring Families of Industrial Case Studies....Pages 134-134 Empirical Software Engineering: Teaching Methods and Conducting Studies....Pages 135-142 Educational Objectives for Empirical Methods....Pages 143-150 On “Landscaping” and Influence of Empirical Studies....Pages 151-151 Involving Industry Professionals in Empirical Studies with Students....Pages 152-152 Industry-Research Collaboration Working Group Results ....Pages 153-157 Teaching Empirical Methods to Undergraduate Students Working Group Results ....Pages 158-162 Technology Transfer and Education Discussion and Summary ....Pages 163-167 Empirical Software Engineering Research Roadmap Introduction ....Pages 168-171 Roadmapping Working Group 1 Results ....Pages 172-174 Roadmapping Working Group 2 Results ....Pages 175-177 Roadmapping Working Group 3 Results ....Pages 178-180 Roadmapping Working Group 4 Results ....Pages 181-183 Empirical Software Engineering Research Roadmap Discussion and Summary ....Pages 184-187 Back Matter....Pages - This book constitutes the thoroughly refereed post-proceedings of the International Dagstuhl-Seminar on Empirical Software Engineering, held in Dagstuhl Castle, Germany in June 2006. The purpose of this workshop was to identify the progress of empirical software engineering since 1992, to summarize the state-of-the-art in ESE, to summarize the state-of-the-practice in ESE in industry, and to develop an ESE roadmap for research, practice, education and training. The 54 revised full papers in this state-of-the-art survey were carefully reviewed and selected from the presentations during the workshop, that provided a forum for a small but representative group of leading experts in software engineering with an emphasis on empirical studies from both universities and industry to meet and reflect on past successes and failures, assess the current state of the practice and research, identify challenges, and define future directions. The papers are organized in topical sections on the empirical paradigm, measurement and model building, technology transfer and education, as well as roadmapping