Forrest roads

In forests we usually find unpaved roads like this:

Forrest Road

Forest Road

Now we should observe how they are constructed. Usually the need to somehow allow access to areas in the forest. It does not have to be the shortest connection, it does not have to be as flat as possible, it is not required to build for high speed and it is not at all required to build for high capacity.

Often it is necessary that trucks and heavy machinery can use the road. So it needs to be constructed in a way that it withstands occasional usage by heavy vehicles, for example to extract wood from the forest or in bad cases to fight a forest fire. And the network usually provides a reasonable way for a truck with a trailer to get out again by driving forward only.

Apart from this there is one very important requirement. The construction must be cheap. It should usually be so cheap that it can easily be paid by a relatively small fraction of the money that is obtained by selling the wood from the forest. There can be a significant secondary use for recreational purposes or even as route through the forest, typically for MTBs and pedestrians, which might justify using money from other sources than the revenue from the forest. Interestingly forests in Switzerland have much denser road networks than forests in Norway or Sweden.

What we never see is forest roads that are somehow prepared for the case that it might be decided in the future to expand them to eight lanes or to transform them into high speed highways. It would be good to use a route that allows this, to already pass hills with cuts or tunnels and to build bridges already much wider than currently necessary. Real highways usually run on a dam or at least they include a thick sequence of layers that often add up to a few meters under the upper asphalt layer. There are „best practices“ about building highways, even relatively narrow highways like this one:

Highway E45 in the Taiga in northern Sweden 2014

Highway E45 in the Taiga in northern Sweden 2014

They are mostly ignored, apart from very universal principals that apply to any kind of construction.

One kilometer of forest road costs a very tiny fraction of one kilometer of such a highway with two lanes.

We should learn from this for our IT solutions. We should think how big our IT solution might actually become. How many customers do we need to serve? What kind of sophisticated functionality needs to be added? Very often we see the mistake that IT solution are built too small. They do not scale, cannot easily be expanded or simply not serve the load or the availability requirements.

Companies like Google. Facebook, Twitter, Netflix or VK serve millions of users 7×24. There is even an implicit promise that there will be no down times and people start relying on this.

We expect banking software to be accurate to the cent. Not sometimes, but always.

Running a device in a chemical plant or steering a rocket requires absolutely reliable software. Errors can be very expensive and cost human lives.

On the other hand there is plenty of software that is useful. Off course it needs to work properly. But the requirements are much lower. A typical app for a mobile phone does not need to be able to run on millions of servers simultaneously. Downtimes during software updates are no problem. Usually a small development team is sufficient to build them. And when it comes to money, the real business logic for this is usually on the server. And we just should not control a chemical plant by a mobile phone app. But mobile phone apps usually have to come at neglectable prices to the users. They either have to pay themselves by the business that they indirectly promote or by a very small amount of money per user, usually combined with a very small number of users.

It is important to really understand the requirements well and build the right size of application. Building too small is very bad, but building too big can be as bad for the project and the organization. And even if the company is lucky and discovers that the software is so attractive that a bigger solution is necessary, then maybe this is a good moment to rewrite it and consider the first version as proof of concept.

Whenever a new highway is built on a route that is previously only covered by a forest road, the highway will usually be built from scratch, ignoring the routing of the old forest road. If the forest road becomes obsolete by that, then the money for the original construction of the forest road is neglectable. Trying to transform a forest road into a highway did happen in many small steps over centuries to create part of our current road network, but it is usually not a recommended approach.

Share Button

DB Persistence without UPDATE and DELETE

When exploring the usage of databases for persistence, the easiest case is a database that does only SELECT. We can cache as much as we like and it is more or less the functional immutable world brought to the database. For working on fixed data and analyzing data this can sometimes be useful.

Usually our data actually changes in some way. It has been discussed in this Blog already, that it would be possible to extend the idea of immutability to the database, which would be achieved by allowing only INSERT and SELECT. Since data can correlate, an INSERT in a table that is understood as a sub-entity via a one-to-many-relationship by the application actually is mutating the containing entity. So it is necessary to look at this in terms of the actual OR-mapping of all applications that are running on that DB schema.

Life can be simple, if we actually have self contained data as with MongoDB or by having a JSON-column in PostgreSQL, for example. Then inter-table-relations are eliminated, but off course it is not even following the first normal form. This can be OK or not, but at least there are good reasons why best practices have been introduced in the relational DB world and we should be careful about that. Another approach is to avoid the concept of sub entities and only work with IDs that are foreign keys. We can query them explicitly when needed.

An interesting approach is to have two ID-columns. One is an id, that is unique in the DB-table and increasing for newly created data. One is the entity-ID. This is shared between several records referring to different generations of the same object. New of them are generated each time we change something and persist the changes and in a simple approach we just consider the newest record with that entity-ID valid. It can off course be enhanced with validFrom and validTo. Then each access to the database also includes a timestamp, usually close to current time, but kept constant across a transaction. Only records for which validFrom <= timestamp < validTo are considered, and within these the newest. The validFrom and validTo can form disjoint intervals, but it is up to the application logic if that is needed or not. It is also possible to select the entry with the highest ID among the records with a given entityID and timestamp-validTo/From-condition. Deleting records can be simulated by this as well, by allowing a way to express a "deleted" record, which means that in case we find this deleted record by our rules, we pretend not having found anything at all. But still referential integrity is possible, because the pre-deletion-data are still there. This concept of having two IDs has been inspired by a talk on that I saw during Clojure Exchange 2017: Immutable back to front.

Share Button

Brass Music and what we can learn for IT

The English term „music“ refers to what we actually listen to, but also to how we write it down on paper, like this:

Music handwritten by Johann Sebastian Bach

Music handwritten by Johann Sebastian Bach


This musical notation is actually like a programming language, because it allows to write down complex musical pieces on paper.

But there is off course more to it than just mechanically playing what is on the paper and dealing with the inconveniences of the musical instruments. What makes it pleasant is the interpretation and that requires skill and intuition and experience and feelings. Since this is not a music blog, I will leave this as it is and stick with the relatively irrelevant side issue of the musical notation language, how we write music.

Generally there might be issues that it is hard to read, because things look too similar, but on the other hand musicians just see it immediately and at least fast enough to work efficiently with it, so I guess that this way of writing music is generally OK.

Now we have the possibility to cover a certain range, slightly more than two octaves, efficiently. Beyond that it will get hard to count the auxiliary lines. To cover different instruments, at least three kinds of Clefs are in use and the same note usually means the same. I think that there are ways to shift the whole system by one ocatve, at least for beginners, but usually with the three clefs that is not necessary for the whole piece of music.

Now for some brass instruments we have different sizes, as for other instruments as well. So the same way to play it yields a different tone on different sizes of the instrument. Just take the recorder, which has five common sizes. They are based on different f and c notes and when you play an f-based recorder you have to adopt to this by playing an f when reading an f-note, for example by closing all wholes. On a c-based recorder you read a c (the deepest you can regularly play) and close all holes to play this. Normally people know this and can deal with this. For brass instruments a different approach was chosen. For situations where you actually want to hear an F and might actually write an F in the notes for one size of the instrument, the larger or smaller instruments just call something an „F“ which is actually not an F at all for the rest of the musical world. So for these instruments „F“ does not mean the tone that you hear, but the grip combination that you do to achieve the tone, simplified. It was supposedly meant to make it easier for relatively unskilled musicians to adapt to different sizes of their instruments, but now even professionals have to live with this.

So they invented a new mechanism to simplify things, which in the overall view makes things a lot more complicated and simplifies just something trivial that even average skilled musicians can easily learn.

I respect the musicians for what they do and I guess since they can deal with this irregularity, it is kind of OK. Or at least up to the musicians to decide if they want to fix this or not.

But we can learn a lot for IT solutions from it.

We often have the situation that we need to adapt a software for another related, but slightly different use case. And we often get the request to simplify things.

It is important to think carefully at which level we do the adaption, so that it will make sense in the long run.

And we should simplify things, but there is no point in trying to make things simpler than they actually are, this simply cannot work an will backfire.

Share Button