Some insights on the software development craft, from a philosophical viewpoint.
Ludwig von Mises — probably the greatest economist of the 20th century — describes in his book "Human Action", an extremely useful method, employed by economic science: the method of imaginary constructions. According to Mises, "an imaginary construction is a conceptual image of a sequence of events logically evolved from the elements of action employed in its formation. It is a product of deduction, ultimately derived from the fundamental category of action, the act of preferring and setting aside. In designing such an imaginary construction the economist is not concerned with the question of whether or not it depicts the conditions of reality which he wants to analyze."
Before we go about understanding this seemingly obscure definition, we should first understand what the term action means in this context, since this is the essential fact underlying every economic phenomenon. Human action is purposeful behavior, whose end is always the relief from a felt uneasiness. Acting man, through a rational effort, attempts to substitute a more satisfactory state of affairs for a less satisfactory one.
With the concept of action in mind it is easier to grasp the concept of imaginary constructions, using examples. Let's consider the state of rest. It refers to a hypothetical market in which no exchange will be performed at all, either because every uneasiness has been removed or because it is no longer possible to remove them.
Two special cases are distinguished: the plain state of rest, a momentary state attained when buyers and sellers do not agree on the prices or when the conditions are not favorable for trade; and the final state of rest, in which the market activities come to an irreversible halt, no action being performed any longer. The prices of products in this final state are called final prices. The plain state of rest is real and is reached again and again (possibly everyday) in every market, but in the next instant the conditions change and the market starts "working" again. The final state of rest, on the other hand, is only hypothetical.
The evenly rotating economy — another imaginary construction — is a "fictitious system in which the market prices of all goods and services coincide with the final prices." This construction is different from the final state of rest in that buyers and sellers are in perpetual activity, but the economic data do not change. Everything stays constant, from market prices to population figures. In this hypothetical world, any medium of exchange would be of no use, since there is no uncertainty about the future. Supply would equal demand and every human dissatisfaction would be automatically removed. There would not be preferring nor setting aside, i.e., there would be no action.
Several other constructions like these are employed by economics. But it is important to remember that imaginary constructions are useful only as a limiting concept. Every one of those concepts lacks some feature of the real economy. This difference allows us to compare the real world with the imaginary world to better understand the nature of the missing feature. For instance, the evenly rotating economy is very useful for the economist trying to understand the nature of money.
Therefore, it is a serious mistake to deal with these constructions as if they were real. Take an example given by Mises himself: "the socialist scheme is logically compatible with the unrealizable imaginary constructions of an evenly rotating economy and of a stationary economy". But the socialist scheme is not compatible with the real world of human action. Indeed, all the socialist projects for a new society are based on a gross economic fallacy. No socialist government in history was (and never will be) able to succeed.
The waterfallish processes are the software development counterparts of these economic fallacies. As in economics, we might make a mental experiment and produce an imaginary construction of a software development environment with some desired characteristics. I am not saying that this method is actually suitable to software development. Anyway, the logical consequences derived from this approach are of great importance to the present discussion.
Following the procedure adopted by economics, we need to choose the features of the real world that are missing in this imaginary world. For example, we could create a world in which there would be no uncertainties about the requirements, so that they could be completely, correctly and unambiguously gathered before writing a single line of code. The requirements, moreover, would stay constant over time (as well as the business circumstances that usually cause requirements changes). Additionally, in that imaginary world, every software activity (requirements, design, test etc) could be performed in isolation, at the point of being executed by different people at different times. In other words, the inherent coupling between all the activities would not exist.
By making some additional abstractions like these from the real world, we could arrive at an imaginary construction that would be logically compatible with every flavor of waterfallish process. But this logical compatibility is meaningful as long as it is treated as a mental tool for the understanding of some aspect of software development. Unfortunately, the champions of heavyweight processes often forget that this is just an imaginary construct and starts treating it as real. As a consequence, developers are told to follow a set of rules that comply only with an imaginary and unrealizable world. So, it is easy to see that their appeal to the logical consistency of their rules is deceiving. Logical consistency means nothing if detached from reality. This is the reason why all such processes fail miserably. In a sense, they are not much different from socialist regimes.
Labels: economics, fallacies, understanding
As I have been writing about the ontological nature of software development, it is becoming more and more clear that there is central thesis in this blog: that the division of software development in activities is methodological, and not ontological. All other themes seem to gravitate around this one. So, I would like to go a little deeper into that discussion, by presenting a few examples.
Let's start with requirements. Several authors and processes advise their readers to gather requirements from the customer at the initial stages of a project, register them in requirements documents, then declare that phase completed, handing all the resulting artifacts to other developers to continue the work from that point. Note that in this advise there are some hidden premises: first, that this is a time-framed activity, precisely scheduled by some project manager. Second, the duration of that work is supposed to be relatively short, compared to the whole project schedule. After all — goes the advice — once the requirements have been collected, there is no longer a need for customer feedback. And third, there is the idea of avoiding the overlapping of activities. "Don't move on to the next phase until the customer has approved the final version of the requirements documents."
This advice is based on the assumption that there is an independent entity in the real world called "software requirements". Being an entity in and of itself, it could be tackled separately from all the other activities. Since every other activity depends on requirements, the latter should be done first and completed as soon as possible.
But it is important to recognize that the requirements aren't just wishlists dictated by the customer. The requirements are behavioral and structural expectations for a software application. The whole purpose of any application is to solve a particular problem set for some particular domain. So, some kind of conceptual domain model — albeit a rough one — must be constructed while the requirements are collected. In addition, as Eric Evans points out, "new systems almost always have to be integrated with legacy or other systems", so it is inevitable to think of some form of high level architecture. But these requirements, when implemented, will have to be tested. So test is another concern the requirements analysts should not overlook. In fact, automated tests are an excellent tool to register requirements.
Because of all this, it's easy to see that requirements don't exist in a vacuum. They can only be separated by abstraction, a mental process that is necessarily done a posteriori. A similar analysis could be done for all the other activities. For instance, software design, as discussed above, is intertwined with requirements. Likewise, professional developers write unit tests for every piece of production code (preferably written before the production code itself). So, one more time, tests come into play. Since the best tool to represent a domain model for software applications is the source code itself, software design and implementation are indistinguishable. And don't forget that the application will eventually be deployed (another software activity), which will spark a torrent of ideas from the end users, resulting in more requirements, starting it all over. The interconnections are virtually endless and have only been superficially addressed here.
It doesn't take much effort to realize that all concerns involved in a software project are intertwined. It doesn't make sense to set them apart and expect to get better results from such a strategy. Using an OO-inspired terminology, one could say that these concerns are highly coupled. High coupling is usually solved with encapsulation. But, as Martin Fowler puts it, "encapsulation is for objects, not for people".
Labels: functional silos, software activities
Software development is a creative process, in which ingenuity and aesthetic concerns play a major role. Using an OO language, for instance, developers are free to choose how to distribute the behavior among classes, how these classes interact with each other and which layers will be used, just to name a few. But, although the set of possibilities is very large, it is not limitless. There are several technological and theoretical constraints, such as memory consumption, turnaround time, computational complexity, design complexity, coupling, cohesion, computability and formal correctness, among others.
According to the scale of four discourses proposed by prof. Olavo de Carvalho (see part I), we could say that, during the course of a project, developers go through all the levels, starting at the poetical one, when a great number of possible solutions are devised. The next step is to select a subset of those solutions that could possibly work (the rhetorical level). By confronting these apparently suitable solutions to one another, it is possible to identify which of them are inconsistent or incomplete and which ones are satisfactory. Sometimes, this confrontation of ideas can lead to the creation of a whole new set of acceptable solutions to the problem. This is what we could call a dialectical practice. Finally, the most likely solutions the team came up with are further refined and formalized, in order to be implemented them in some programming language. Evidently, this is not a linear process, and developers usually go back and forth, as the circumstances demand it.
Eventually some artifacts are produced as a result of the proceedings above, including use case specifications, UML diagrams, test reports, quality control reports and the like. This is especially true in the most bureaucratic, waterfallish companies. Every one of those artifacts, being instances of human discourse, can also be classified according to the four levels. But note that there is a key difference between a creative process and its resulting artifacts. The process, although it spans throughout all levels, is just a means to an end. What really matters is delivering a valuable product to the end user. It does not matter whether developers underwent a poetical stage early on; at the end, we all wish to deliver an executable and correct source code, amenable to lexical and syntactical analysis, mathematical proofs of correctness, style checking and so on. In other words, the final cause of this process is the logical discourse.
The artifacts, on the other hand, are crystallized forms of discourse, registered during the early stages, that are expected (a naive expectation, of course) to remain valid long after it has been produced. As a consequence, some artifacts stand at the level of high probability, such as the design diagrams and test result reports. Others fall at a lower level, the rhetorical one: examples include the process body of rules (usually a distorted version of RUP) and quality management reports (usually a bunch of finger-pointing paperwork). And there are those artifacts that could be put in the lowest level of the scale: use case documents are an outstanding specimen of the kind. Everything written down in such documents is nothing but poetical formulation. It represents just a possible world, very useful for hypothetical considerations, but with nothing that binds it to the actual world.
Actually, I think this is the ultimate criterion for classifying software artifacts according to the four discourses scale: how much it is bound by reality. In the case of software development, the reality forces come from the technological and theoretical constraints I mentioned above. After all, it is possible to write pretty much everything you want in a use case. But only a small subset of that is actually implementable. Consider this dramatic scenario: a requirements document which states that the application should divide integers by zero. It is even possible to get some client to "approve it", but no well-implemented compiler in the world would accept a tentative implementation of that requirement. This is a rather contrived example, but it is helpful to get the point: software development is a creative process, but it ultimately seeks to produce source code, a reality-bound artifact, which should be treated as the most important asset in the project.
Labels: Dialectic, Logic, Poetics, Rhetoric
One of the most important and well-known theories published by Olavo de Carvalho is that of the Four Discourses [book excerpt in Portuguese]. The theory states that in the works of Aristotle there is a core idea: that every human discourse may be classified as belonging to one the following four categories: poetical, rhetorical, dialectical or analytical (logical). Each of these is associated with a level of credibility that is expected from the audience.
Roughly speaking, the poetical discourse is targeted at the level of possibilities: people watching a play know that everything being presented at the stage is not true (they are "pretending", so to say). But nonetheless they realize that all those events could be true. The spectators can even feel the pain and enjoy the good fortunes experienced by the characters. This is what Samuel Taylor Coleridge called "suspension of disbelief".
The rhetorical discourse aims at creating a strong belief in the audience, in order to persuade them to take some decision. The rhetorical discourse is that of lawyers at courts and politicians in election campaigns, for example. The level of credibility intended here is similitude. It suffices to seem to be true, even though there is no certainty if what is being said is actually true.
The dialectical discourse is an attempt to reach a level of high probability. Beliefs and theories are tested and confronted with each other, so as to reduce the margin of error to a minimum acceptable threshold. This is the method employed by science and philosophy. As an investigation progresses (a process that could last for centuries), scientists come up with theories that are most likely to be true.
The last discourse — analytical — addresses absolute certainty. A mathematical proof is the canonical example of the analytical discourse; a (possibly long) chain of reasoning that begins with the first principles (axioms, definitions) and leads to conclusions that are necessarily true. Not the slightest error or doubt is accepted.
In the next post, I will be talking about the relationship between these levels of discourse and the artifacts produced in a software development project.
Labels: Dialectic, Logic, Poetics, Rhetoric