Software Engineering in the Age of AI

From programming to software engineering

The term software engineering was made popular at a NATO conference in Garmisch, Germany, in 1968. Computers had become powerful, projects had become large, and many of them were late, over budget or simply did not work. People called this the software crisis. The organizers chose the word engineering on purpose: they wanted software to be built with the same discipline as bridges and aircraft.

The classic IEEE definition says software engineering is "the application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software." Notice the last three words: development is only the beginning. A shorter modern definition, from Google's engineers, is: software engineering is programming integrated over time.

Why software projects fail: five famous stories

Studying failures is the fastest way to understand why the discipline exists. In each story below, look at where the problem really came from. It is almost never "a programmer who could not code".

1. Ariane 5, Flight 501 (1996). Thirty-seven seconds after launch, Europe's new rocket turned sideways and was destroyed. The navigation software had been reused from Ariane 4. One line converted a 64-bit floating-point number (a horizontal velocity value) into a 16-bit signed integer. On the faster Ariane 5 the value was larger than 32,767, the conversion overflowed, the navigation computer shut down, and its backup — running the same code — failed the same way. Root cause: reuse without re-checking the assumptions (requirements) of the new environment, and no test with realistic Ariane 5 flight data.

2. Therac-25 (1985–1987). A radiation therapy machine gave massive overdoses to at least six patients; several died. Earlier models had hardware safety locks; the Therac-25 relied on software alone. A race condition appeared only when an experienced operator typed quickly and edited the settings within a few seconds. Root causes: removing independent safety barriers, no proper safety analysis, reused code, confusing error messages ("MALFUNCTION 54"), and a manufacturer that did not believe the first reports.

3. Knight Capital (2012). A trading company lost about 440 million US dollars in 45 minutes. A new version was deployed manually to only seven of its eight servers. On the eighth server, a reused configuration flag switched on old, dead code that bought and sold shares in a loop. Root causes: manual deployment with no check, dead code left in the system, a flag reused with a new meaning, and no automatic "kill switch".

4. Healthcare.gov (2013). The US health-insurance website crashed on its first day; only a handful of people could register. Root causes: many contractors with no single technical leader, requirements that changed until the last weeks, and almost no end-to-end testing under real load before launch.

5. CrowdStrike (July 2024). A faulty content update (a configuration file, not a program) for a security product caused about 8.5 million Windows computers to crash and restart in a loop. Airlines, hospitals and banks stopped working. The file triggered an out-of-bounds memory read in code that runs inside the operating system kernel. Root causes: the update was sent to all customers at once instead of step by step (no staged rollout), and the validation of that kind of file had a gap.

Why software is hard: Brooks's four difficulties

In 1986 Fred Brooks (the manager of IBM's OS/360 project) wrote a famous essay, "No Silver Bullet". He argued that no single tool or method would ever make software development ten times easier, because software has four essential difficulties:

1. Complexity. A program has an enormous number of states; no two parts are alike (if they were, we would make them one function). Complexity grows faster than size.
2. Conformity. Software must fit the messy world around it: laws, other systems, file formats, human habits. Physics has simple laws; tax rules do not.
3. Changeability. Software is always asked to change, because it is "easy" to change — it is only text. Successful software changes the most.
4. Invisibility. You cannot see software. A building has a floor plan; software needs many different diagrams, and none of them shows everything.

Brooks separated two kinds of difficulty:

• Accidental complexity comes from our tools: syntax, memory management, slow compilers, writing the same boilerplate code again and again. Better tools remove it.
• Essential complexity comes from the problem itself: what the system must do, the rules, the special cases, how the pieces fit together. No tool removes it, because it is the job.

This distinction is the key to understanding AI in software engineering. AI assistants are excellent at removing accidental complexity: they remember APIs, write boilerplate, and translate between languages. They do much less about essential complexity: knowing what the users need, which rule applies when two rules conflict, and whether the design will survive the next five years of changes.

The lifecycle: what engineers actually do

Every software product, whatever process a team follows (Chapter 2), goes through the same basic activities:

Verification asks "Are we building the product right?" (does it meet its specification?). Validation asks "Are we building the right product?" (does it meet the users' real needs?). You will meet these two words many times.

Another classic result: the later a defect is found, the more it costs to fix. A misunderstood requirement caught in a 10-minute conversation costs almost nothing. The same misunderstanding found after release may require redesign, new code, new tests, a new release, data repairs and apologies to users.

What does "good software" mean? Quality attributes

A program can do exactly the right thing and still be a bad product: too slow, impossible to use, easy to hack, or impossible to change. Engineers describe quality with quality attributes (also called non-functional requirements). The international standard ISO/IEC 25010 lists them; the most common are:

Quality attributes often conflict. More security (two-factor login) can reduce usability. More performance (caching) can hurt data freshness. Engineering means choosing the right trade-off for these users and writing the reason down.

The AI angle: what changes and what does not

AI coding assistants appeared in three waves: autocomplete (suggesting the next lines), chat (answering questions and writing whole functions), and agents (tools that read a code base, run commands, edit many files and open pull requests on their own). By 2026 most professional developers use at least one of them.

What does the evidence say? It is mixed — and that is itself an important lesson:

• In a 2022 controlled experiment by GitHub, developers using Copilot finished a small, well-defined task (writing an HTTP server in JavaScript) about 55% faster than those without it.
• A 2022 study from New York University ("Asleep at the Keyboard?") generated programs with Copilot in security-sensitive scenarios and found that about 40% of them contained a known type of vulnerability.
• A 2025 randomized study by METR followed experienced open-source developers working on their own large projects. With AI tools allowed they took about 19% longer — while believing they had been about 20% faster.

The pattern: AI helps most on small, clear, familiar tasks (accidental complexity) and least — sometimes negatively — on large, unfamiliar code bases with unwritten rules (essential complexity). And people are poor judges of their own productivity with AI.

What AI does not change:

1. Responsibility. When code reaches users, the team that shipped it is responsible — legally and ethically. "The AI wrote it" is not a defence (Chapter 12).
2. Understanding the problem. An AI answers the question you asked. If you asked the wrong question (a wrong requirement), you get a perfect solution to the wrong problem.
3. Verification. AI output must be reviewed and tested at least as carefully as human code, because its mistakes look plausible: correct style, confident comments, wrong logic.
4. Security and privacy. Pasting StudyBuddy's database of student emails into a public AI chat is a privacy incident, however useful the answer.

The engineer's rule: verify, then trust

Throughout this course you will use one habit for anything you did not write and check yourself — a library, a colleague's pull request, a Stack Overflow answer, or AI output:

1. Read it. Can you explain every line? If not, you cannot own it.
2. Check it against the requirement, not against what the code seems to do.
3. Test the edges: empty input, zero, negative numbers, very large values, special dates, missing data.
4. Check the risks: security (input from users), privacy (personal data), licences (copied code).
5. Only then merge it.

Here is a typical example. A student asks an AI for "a Java method that checks whether a year is a leap year" and receives:

static boolean isLeapYear(int year) {
    return year % 4 == 0;   // leap years are divisible by 4
}

It compiles, the comment sounds right, and it works for 2024 and 2023. But the real rule is: divisible by 4, except years divisible by 100, unless also divisible by 400. So 1900 was not a leap year and 2000 was. An engineer finds this by testing the edge cases — not by trusting the comment.

static boolean isLeapYear(int year) {
    return (year % 4 == 0 && year % 100 != 0) || year % 400 == 0;
}

Meet StudyBuddy, our case for the semester

Every chapter uses the same system, so you can see how requirements, design, testing, security, privacy, law and operations connect.

StudyBuddy is a course-help app for Wenzhou-Kean University students, built by a five-person student team: Lina (product owner), Omar (backend), Mei (mobile), Daniel (testing) and Aisha (UX design).

• Users: students, teaching assistants, instructors and a department administrator.
• Features: a Q&A board for each course; Buddy, an AI tutor that answers from the instructor's uploaded course materials by calling an external large-language-model service; study-group matching by course and free time; flashcards and practice quizzes; deadline reminders; a moderation queue for reported posts.
• Data: student ID, name, WKU email, enrolled courses, posts, AI-chat history, availability, optional photo. No grades.
• Technology: a web app and an Android app, a Java backend with a REST API, a MySQL database, hosted on a cloud server in mainland China — while some users study at Kean in the United States.

Even this small description already raises engineering questions for later chapters: What exactly may Buddy answer (requirements)? What happens when the AI service is down (design, reliability)? Who can read chat histories (security, privacy)? Which laws apply to data of students in two countries (legal)? How much energy do thousands of AI calls use (sustainability)?

Key takeaways

• Software engineering is programming integrated over time: teams, users, change, and years of operation.
• Famous failures (Ariane 5, Therac-25, Knight Capital, Healthcare.gov, CrowdStrike) came from requirements, reuse, testing, deployment and organization — rarely from a lack of coding skill.
• Brooks: software is complex, must conform, keeps changing, and is invisible. Accidental complexity can be removed by tools; essential complexity cannot.
• The lifecycle activities are requirements, design, construction, verification and validation, deployment and operation, and maintenance; most of the cost comes after the first release, and late defects are the most expensive.
• Quality is described by measurable quality attributes that often conflict and must be traded off.
• AI assistants mostly reduce accidental complexity; evidence on productivity is mixed; they add risks (plausible wrong code, vulnerabilities, data leaks). Responsibility stays with the engineer.
• Habit for the whole course: verify, then trust.