EncycloZine

Software engineering is the technologies and practices that are used to create computer software while improving productivity and quality. In the year 2000, these technologies and practices encompass languages, databases, tools, platforms, libraries, standards, patterns, and processes.

Software engineering is the spectrum of applications that create economic and social value. In the year 2000, practitioners have built many successful and important systems, such as email, embedded software, graphical user interfaces, office suites, operating systems, optimizing compilers, relational databases, robotics controllers, video games, and the world-wide web. Other important applications include accounting, airline reservations, avionics, banking, and telephony.

Software engineering is the community of practitioners who create programs. In the year 2000, this community included about 640,000 people in the U.S. and many more around the world. They mostly have computer science degrees, though some have science or engineering degress, some have business or other non-technical degrees, and some have no degrees. They use diverse technologies and practices and work on diverse applications.

Software engineers have been very successful. Economic: Software engineers created a large part of the growth of the U.S. and world economies during the last half of the twentieth century. Software engineers drove 1/4 of all economic growth (increase in GDP) during 1990s in the U.S., about $90 billion per year. Software drove 1/6 of all productivity growth (efficiency within GDP) during the late 1990s in the U.S., about $33 billion per year. Social: Software engineering has improved U.S. and world culture. Email, the world-wide web, and instant messaging enable people to interact in whole new ways. Software lowers the cost of many important social goods and services, such as healthcare and fire departments.

Software Engineering versus Programming

Software engineers work on all sizes of applications: small and large. Software engineering often emphasizes large applications that individual programmers cannot write alone. Some projects have more than 1000 programmers at once.

Programming emphasizes coding, independent of budget and schedule. Software engineering tries to encompass software projects more completely, including budget and schedule. On a larger scale, software engineering recognizes that software development fits in a large business context with relationships to marketing, sales, production, installation, training, support, and operations.

Software Engineering versus Computer Science

Whether software development is more like science or engineering has been fiercely debated for many decades. Software development shares attributes of both and many software projects have elements of both, but distinctions exist.

Metaphorically, software engineering is to computer science and information science as traditional engineering is to physics and chemistry.

Software Engineering versus Traditional Engineering

Software engineers borrow many metaphors and techniques from traditional engineers. Software engineers aspire to build low-cost, reliable, safe applications, just like traditional engineers build. For example, software engineers borrow requirements analysis, quality control, and project management techniques.

Traditional engineers also borrow many metaphors and techniques from software engineers. Today, many traditional engineers use software tools to design and analyze systems, such as bridges and buildings. These new kinds of design and analysis resemble programs in many respects, because they exist as electronic documents and they go through analysis, design, implementation, and testing phases, just like software. Many traditional engineers now experience bug fixing cycles and release pressures, just like software engineers.

Software engineering and traditional engineering are independent. The scientific foundations, cost, status, role of innovation, and replication differ.

Software engineering is much younger than civil engineering and electronic engineering and much older than space engineering and biological engineering.

Software engineering is already as predicatable and reliable as many fields of engineering, such as space engineering, biological engineering. Although large, reliable software systems can be and are constructed, software projects that fail during construction or in service are still too common. However, large traditional engineering systems that fail, like Three Mile Island, Chernobyl, Bhopal, Space Shuttle Challenger, are also too common.

The Software Crisis

Software engineering arose out of the so called software crisis of the 1960s, 1970s, and 1980s, when many software projects had bad endings. Many software projects ran way over budget and schedule. Some projects caused property damage. A few projects caused loss of life. As software becomes more pervasive, we increasingly recognize the need for better software. The software crisis was originally defined in terms of productivity, but evolved to emphasize quality.

Time and Money: One classic example is the OS 360 operating system, still used on the IBM 360 Series and its descendants. This decade-long project eventually produced a working system, which is amongst the most complex software systems ever designed. OS 360 was one of the first very large (1000 programmer) software projects. IBM eventually switched to Linux in the late 1990s, partly because OS 360 never lived up to expectations. Fred Brooks admits in Mythical Man Month that he made an $X million mistake.

Property Damage: Software bugs can cause property damage. Poor software security allows hackers to steal identities which costs time and money and hassle. The explosion of a European Ariane rocket and other disasters spur further developments in the field.

Life and Death: Software bugs can kill. Several embedded systems used in radiotherapy machines failed so catastrophically that they administered lethal doses of radiation to patients.

Silver Bullets and No Silver Bullets

For decades, solving the software crisis was paramount. Seemingly, every new technology and practice from the 1970s to the 1990s was trumpeted as a silver bullet to solve the software crisis.

Technologies: Every new technology and practice for decades was touted as the solution to the software crisis: structured programming, object-oriented programming, process, CMM, UML, Ada, methodologies, and so on.

Practices: Some pundits argued that the software crisis was due to the lack of discipline of programmers. Many practitioners resist process strongly. Some believed that if formal engineering methodologies could be applied to software development, the production of software would become as predictable an industry as other branches of engineering. This led to work on code of ethics, professionalism. The issue is so pervasive that it cannot be deliberate. It must be a natural response to typical experience.

In 1987, Brooks published the famous paper No Silver Bullet, arguing that no individual technology or practice can make a 10 fold improvement in productivity in 10 years. All of the technologies and practices that practitioners use today make incremental improvements. Up to now he has been proven right, and he looks to remain correct for the foreseeable future. http://www.virtualschool.edu/mon/SoftwareEngineering/BrooksNoSilverBullet.html

Historical Notes

Women's Issues: In the 1940s, 1950s, and 1960s, software was a women's ghetto. Men preferred the higher prestige hardware engineering roles. So, women filled the lower prestige programming roles. Women like Grace Hopper were common. Many unsung women wrote code prior to 1968. Today, few women work in software engineering. Women have largely moved into analysis and testing roles. Saying that this is sexual discrimination is too simple, because it related directly to individual identity. In this sense, software engineering is the masculization of programming.

NATO: NATO sponsored two conferences on software engineering in 1968 and 1969, which gave the field its initial boost. Many consider these conferences to be the start of the field. However, the term software engineer had been used for many years prior to 1968. There has been a continuity of practice between the early projects from the 1940s to the 2000s.

Cost: The cost of software versus hardware has changed substantially, from the days of the mainframes, to the days of the PC. When mainframes were expensive, software projects could be expensive. Because powerful PCs are cheap, software costs must become cheaper, in comparison.

Criticisms of Software Engineering

Critics charge that the assumptions that underlie software engineering are inherently flawed.

Managing Expectations: Managing the customer's expectations to something that can be built and delivered is the key to successful "software engineering" methodologies. Thus, the field resembles marketing, sociology, or voodoo, more than traditional engineering with its responsibilities to society at large and the perils of legal liability with failure to protect the public interest. — Every profession manages expectations.

Poor Requirements: The requirements for most projects are incomplete and/or inconsistent. Some customers are first-time customers with little previous experience in writing requirements for a software project. Other customers really do not know what they want, and say "I'll know it when I see it" (IKIWISI). Even experienced customers who know exactly what they want may be unable to articulate their requirements. And, complete requirements may describe programs that have no computable or practical solution. Embedded systems are unique in that they are designed by other engineers, who can sometimes define interfaces completely. Users almost always expect far more than they write in the specifications. — One response is to not even try. Another response is rapid prototyping.

Rising Complexity: Critics argue that the probability of failure increases as project size, scope and complexity increase. Technologies and practices have improved through the 1970s, 1980s, and 1990s, but the complexity in requirements and user expectations increases fast enough to overwhelm the improvement in technologies and practices. So, the gap between what is expected and what is delivered has not improved. — This is not the fault of practitioners. This is actually a measure of the success of practitioners.

Ongoing Change: Practitioners are continually eager to develop new technologies and practices and try the newest tools in every project. Innovation versus conservatism. — This ongoing improvement actually symbolizes the success of software engineers.

Ongoing Failure: Critics charge that incomplete or poorly designed systems are still too common. The early lessons of the field were not sufficient to prevent subsequent disasters. — All fields that try new things experience some failure.

Nothing New: Critics argue that software engineering has not created anything on its own, it merely uses what computer scientists already know. This is true, but software engineers emphasize applying their skills. Computer science concepts, like algorithms and complexity, are independent of budget and schedule. — Compilers, make, and cvs were originally built by practitioners who simply needed tools.

Anyone Can Do It: Many bright people from other fields (engineers, scientists, business people) write spreadsheet templates or calculator programs, and eventually switch to writing large applications, and believe that they do software engineering. And many of them are. — Software engineering is a skill that is refined through long practice. Software engineers are the ones who already have the education and experience, and they keep up. This is true of any skill.

We Do Not Know What It Is: This is often said by people who want to redefine software engineering on their own terms. — Software engineering is the technologies and practices used to create office suites, databases, and engine controllers.

Methodologies

Software engineering methodologies span many disciplines, including project management, analysis, specification, design, coding, testing, and quality assurance. All of the methodologies guiding this field are collations of all of these fields.

Software design methodologies can be informally classified into thick and thin. Thick methodologies include a large amount of formal process paperwork and documentation. The well-known thick methodologies include Cleanroom and Rational Unified Process (RUP). Thin methodologies eschew formal process paperwork and documentation. The well-known thin methodologies include Extreme Programming (XP) and Agile Processes.

Recently, some (like Karl Weigers) have argued for no more methodologies. Methodologies tend to list the contemporary technologies and practices and insist that everyone use them. This advice is obvious for those who have the opportunity to use contemporary technologies and practices, and useless for those who maintain legacy systems and must use legacy tools cannot do so, due to circumstance. So, methodologies are not specfically useful. And methodologies must be updated as technologies and practices evolve.

Process

There is a growing body of software development organisations that are implementing process methodoliges. This is mostly driven by the defence industry that are requiring a 'Rating' baised on 'Process models' as a part of their contracts.

For example the Capability Maturity Model (CMM) describes the experience level on which a software creation organization works. ISO 9000 is a standard of describing and organizing the process with the documentation in a formalized way.

ISO 15504, The software process life cycle is also gaining wide usage. This standard is aimed at setting out a clear model for process comparison. "SPICE" as it is known is used in a similar manner to CMM or the new CMMi. That is as a model of a set of processes that can be used to manage, control, guide and monitor software development. This model is then used for comparison against what a development organization or project team actually do during software development. This information is analyzed to identify weaknesses to drive improvement, and strengths so that they can be continued or integrated into common practice for that organization or team.

Different methodologies advocate conflicting solutions. Proponents of different methodologies often get into heated debates over their merits.

Books

Many books have been written about software engineering. The following books are milestones by ex-IBM people.

• Fred Brooks wrote the The Mythical Man-Month, at UNC describing what he learned from the OS 360 project.
• Watts Humphrey wrote The Capability Maturity Model for the Software Engineering Institute, emphasizing management.
• Gerald Weinberg wrote The Psychology of Computer Programming as an independent consultant.

Professional Software Engineering

In the mid-1990s, The National Society of Professional Engineers sued in all states to prohibit anyone from using the term software engineer as a noun or field of employment. They won in most states. Utah does not license professional engineers, so the point was moot there.

In response, the IEEE and ACM began a joint effort called JCESEP in 1993, which evolved into SWECC in 1998 to explore making software engineering into a profession. The ACM pulled out of SWECC in [1998?] objecting to its support for the Texas professionalization efforts. The IEEE continued to support making software engineering a branch of traditional engineering.

Donald Bagart of Texas became the first professional software engineer in the U.S. on September 4, 1998 or October 9, 1998. As of May 2002, Texas had issued 44 professional enginering licenses for software engineers.

The professional movement has been criticized for many reasons. The professional movement uses the traditional engineering model, so licensed software engineers must eventually learn years of physics and chemistry, which is irrelevant to most practitioners. It ignores the question of what to do with all of the computer science majors, who don't earn degrees in engineering schools. Another criticism of the Canadian effort is that most professional software engineers studied computer or electrical engineering (who already qualify for traditional licenses) and not computer science. Thus, it affected a different community than was intended.

• http://www.faqs.org/faqs/engineering/pe-eit-exam/ Apr 10, 2002
• http://www.sdmagazine.com/documents/s=746/sdm0003g/0003g.htm

The fields of data engineering, knowledge engineering, and so on have similar concerns over engineering issues.

Software Engineering Technologies and Practices

Technologies and Practices

• Programming languages, like Java and C++. Also includes compilers, linkers, and loaders.
• Relational databases, like SQL.
• Software tools, like make, cvs, editors.
• UML, design language
• Patterns and Anti-patterns document many common programming and project management techniques.
• Software development processes, like CMM, ISO-9000, and Agile.
• Platforms, like WINTEL PCs and IBM mainframes. As platforms have become more powerful, tools have become available on less expensive platforms and more widely available.
• Several approaches to the programming in practice:
• Structured programming
• Object Oriented Programming
• Software components
• State oriented programming
• Aspect oriented programming
• Post-object programming

Life Cycle (Waterfall Point of View)

• Project lifecycle
• Requirements gathering and Analysis
• Software architecture
• Software design
• Functional decomposition
• Computer programming
• Testing, focuses on detecting (preventing?) bugs
• Quality assurance, ensures compliance with process, but it is a misnomer, it is not quality and not testing

Management (Management Point of View)

• Project management
• Configuration management
• Personnel management
• Risk management, everything is a risk.

Notable Problems of Software Engineering

• Complexity or scaling
• Software brittleness problem
• Second system syndrome
• Estimation

Notable Software Engineers

• John Backus: Fortran, first optimizing compiler
• Donald Bagert: First professional software engineer in U.S. in 1998.
• Kent Beck: Refactoring, XP (pair programming, test driven development).
• Barry Boehm: Economics, COCOMO, spiral model.
• Grady Booch: Object-oriented design, UML.
• Fred Brooks: System 360, OS 360, Mythical Man Month.
• Michael Fagan: Code reviews.
• Grace Hopper: First compiler (Mark 1), COBOL.
• Watts S. Humphrey: CMM, headed (founded?) the SEI.
• Jean Ichbiah: Ada
• Brian Kernighan: C and Unix.
• Peter G. Neumann: Computer Risks, ACM Sigsoft.
• David Parnas: Module design, social responsibility.
• Dennis Ritchie: C and Unix.
• Winston W. Royce: Waterfall model.
• Will Tracz: Reuse, ACM Software Engineering Notes.
• Richard Stallman: Founder of the Free Software Foundation
• Gerald Weinberg: The Psychology of Computer Programming.
• Ed Yourdon: Structured programming, The Decline and Fall of the American Programmer.

Notable Software Moguls

• Bill Gates: Microsoft
• Larry Elison: Oracle
• Mitch Kapor: Lotus (now with the Open Source Applications Foundation)

Notable Applications

• Computer graphics image processing and special effects for video and film
• Email
• Embedded systems
• Avionics software
• Automotive software
• Graphical user interfaces
• Office suites
• Operating systems
• Optimizing compilers
• Relational databases
• Video games
• World wide web (Tim Berners-Lee)

For more information on the software engineering community, see:

• http://www.software-engineer.org
• http://www.agilealliance.org
• http://www.acm.org The Association for Computing Machinery
• http://www.computer.org The IEEE Computer Society
This article is licensed under the GNU Free Documentation License. It uses material from the Wikipedia article "Software_engineering"