Addresses

Postal addresses seem to be easy:
– Name and/or Company
– Street and/or PostBox
– ZIP Code
– Municipality
– Country

This is true in Germany or Switzerland and a few other countries. I would like to add, that in some large buildings it can be a good idea to add the apartment number, but these buildings are rare in these two countries and the name is really almost always sufficient.

For relational databases please keep in mind, that these fields may get just a bit or even a lot longer than we anticipated. US-ZIP-Codes are now something like NY-11713-5532. And others may be longer. How long are names, street names and names of villages and towns? Even the country part, which seems to be relatively easy, brings some challenges. Countries like Switzerland and Germany and Canada are no problem. But what about semi-independent countries like Guernsey, Jersey, …? And what about areas, that are de-jure part of another country, but de-facto an independent country? Or an independent country that is not accepted by each country? There are lists of countries that we can find and usually they include also the „semi-independent“ and „semi-accepted“ countries. But it is a good idea to check if the list is complete enough for our purpose. I would not recommend to define it yourself.

But now the other parts of the address: If we want to describe the location where a person lives, and this person does not live in a housing area, but for example in a nomadic life style, it becomes a bit harder. Or we might observe a different number of lines for the address. Or even that streets are not named consistently and the only relyable address is a postbox. Many countries do not write the names on the door and on the letterbox, but just an apartment number. So this number with some word for „apartment“ that is understood by the local post officer with possibly limited language skills is needed. Also in some countries there are a lot of buildings for „streetname 3053A“, so we need to add the building number also.

My point is, that postal addresses are not as easy as it might seem. So I recommend to do some research in the internet to find a library or a documentation that handles this or can be used as a basis instead of trying to invent a new wheel that will probably be incomplete and suffer from its insufficiencies at some point. It is sometimes more important to recognize which seemingly trivial problems are actually harder than being able to invent solutions for such problems. We should invest our energy on solving problems that have not been solved with publicly available documentations or even libraries and that make up our business. In this case the actual implementation is rather trivial, but the specification of the requirements is the hard part, so it is enough to find some useful documentation for this. It might of course be sufficient to handle only for example French addresses, if the system will never be dealing with foreign addresses, but the experience shows that at least some thinking about what it means to extend the system later are a good idea.

And please, handle too long entries properly instead of displaying a stack trace on the end users screen or even providing a spot to attack the server.

Share Button

Happy New Year 2021

Un an nou fericit! — Happy new year! — Срећна нова година! — Frohes neues Jahr! — Onnellista uutta vuotta! — С новым годом! — Gullukkig niuw jaar! — FELIX SIT ANNUS NOVUS — ¡Feliz año nuevo! — Feliĉan novan jaron! — عام سعيد — Щасливого нового року! — Gott nytt år! — Bonne année! — Καλή Χρονια! — Felice anno nuovo! — Godt nytt år!

This was generated with JavaScript using Rhino:

a = ["Frohes neues Jahr!",
"Happy new year!",
"Gott nytt år!",
"¡Feliz año nuevo!",
"Bonne année!",
"FELIX SIT ANNUS NOVUS",
"С новым годом!",
"عام سعيد",
"Felice anno nuovo!",
"Godt nytt år!",
"Gullukkig niuw jaar!",
"Feliĉan novan jaron!",
"Onnellista uutta vuotta!",
"Срећна нова година!",
"Un an nou fericit!",
"Щасливого нового року!",
"Καλή Χρονια!"];
b = a.map(function(x) { return Math.floor(1000000000 + Math.random()*1000000) + " " + x; });
b.sort();
c = b.map(function(x) { return x.replace(/^\d+\s+/, ""); });
print(c.join(" — "));

Share Button

Christmas 2020

¡Feliz Navidad! — καλά Χριστούγεννα! — Buon Natale! — З Рiздвом Христовим! — クリスマスおめでとう ; メリークリスマス — Natale hilare! — Merry Christmas! — ميلاد مجيد — Hyvää Joulua! — С Рождеством! — Joyeux Noël! — God Jul! — Feliĉan Kristnaskon! — Crăciun fericit! — God Jul! — Frohe Weihnachten! — Срећан Божић! — Prettige Kerstdagen!


This was generated by a bash script. I am using Perl instead of sed, but not for program logic:

#!/bin/bash
set -e

mkfifo /tmp/tmp_pipeA-$$
mkfifo /tmp/tmp_pipeB-$$

cat << EOTXT |head -18 > /tmp/tmp_pipeA-$$ &
С Рождеством!
Hyvää Joulua!
καλά Χριστούγεννα!
Buon Natale!
Prettige Kerstdagen!
З Рiздвом Христовим!
Merry Christmas!
Срећан Божић!
God Jul!
¡Feliz Navidad!
ميلاد مجيد
クリスマスおめでとう ; メリークリスマス
Natale hilare!
Joyeux Noël!
God Jul!
Frohe Weihnachten!
Crăciun fericit!
Feliĉan Kristnaskon!
EOTXT

od -x /dev/urandom \
|head -18 \
|perl -p -e ’s/^\d+\s//;‘ > /tmp/tmp_pipeB-$$ &

paste /tmp/tmp_pipeB-$$ /tmp/tmp_pipeA-$$ \
|sort \
|cut -f 2 \
|perl -p -e ’s/\n/ — /g;‘ \
|perl -p -e ’s/ — $/\n/;‘

rm -f /tmp/tmp_pipeA-$$
rm -f /tmp/tmp_pipeB-$$

Share Button

Functional Scala

I participated online in the conference „Functional Scala 2020“ in London. That it was in London had mostly one relevance, which was the time zone. There was no physical location and all talks were done online. An interesting idea was a virtual location. It consisted of rooms and we could move a dot representing ourselves around. Each room consisted of a beautiful landscape as a map of a different climate zone. I could hear what others said, when I moved my dot, representing myself, closer to them, as in real life, and do some nice conversations like that.

A lot of things were said about Scala 3, which will be a big step forward, but also a big step, because it is not compatible with Scala 2. So some work will be necessary to move on to Scala 3, but we will gain a better language for beginners, intermediates and advanced Scala developers.

I am really looking forward to Functional Scala 2021, hopefully in London.

Share Button

How to disable touchpad (on Linux/X11)

For me it is much better to use an external mouse than the touchpad, which I sometimes touch accidentally.

So, here is how to disable it with a short Perl-Script. A bash script with a bit of Perl would do the same, btw.


#!/usr/bin/perl
my $tp = `xinput list | egrep -i touch`;
chomp $tp;
$tp =~ s/.+id=(\d+).+/$1/;
system "xinput set-prop $tp 'Device Enabled' 0";
print "Touchpad disabled\n";

Share Button

Devoxx UA 2020 (talks)

I watched the conference onlie and picked the following talks:

And on the second day:

Share Button

Devoxx UA

Most conferences have been cancelled, since it is difficult to hold a conference these days. The idea to move the conference online has been obviously around, but it was rejected by most organizers, because it it not the same and the all important chance to meet other people is just not the same.  So the Devoxx in Antwerp, which I like to visit every year, did not happen.  But Devoxx Ukraine decided to go for online.

So how did it work:  There were three tracks.  Each track was represented by a youtube channel, on which the live talk was transmitted.  Before and after the talks, professional moderators appeared in these channels, announced the speakers and did what moderators do in normal conferences.  The devoxx App worked on my cell phone and that seemed to be the most up to date schedule.

Some talks were really excellent.  I enjoyed them a lot even online.  For talks that are not so good, it requires more discipline to stay tuned.

The discussion was done in zoom channels that belonged to the three tracks.  So discussions could technically last until the end of the next talk.  The discussions in zoom also worked quite well, which was a surprise.

I think it is probably the right reaction, to have fewer conferences than usually in a year and to cancel some and move some to online, exactly as it is happening.  I think DevoxxUA had around 10’000 visitors, so they absorbed audiences of several conferences.  Also some common speakers at Devoxx conferences were not giving talks, so I assume that also part of the speakers decided that the online format is not ideal for them.

About the contents I will write in another Blog article.

Share Button

How to rename files according to a pattern

We often encounter situations, where a large number of files should be copied or renamed or moved or something like that.
This can be done on the Linux command line, but it should be possible in almost the same way on the Unix/Linux/Cygwin-command line of newer MS-Windows or MacOS-X.

Now people routinely do that and they have developed several ways of doing it, which are all valid and useful.

I will show how I do things like that. It works and it is not the only way to do it.

So in the most simple case, all files in a directory ending in ‚.a‘ should be renamed to ‚.b‘.

What I do is:


ls *.a \
|perl -p -e 'chomp;$x = $_;s/\.a$/.b/;$y = $_; s/.+/mv $x $y\n/;' \
|egrep '^mv '\
|sh

You can run it without the last |sh, to check if it really does what you want.

So I use the files as input to a short perl script and create shell commands. It would be possible to do this actually in Perl itself, without piping it into a shell:


ls *.b \
|perl -n -e 'chomp;$x = $_;s/\.b$/.c/;$y=$_;rename $x, $y;'

You could also read the directory from perl, it is quite easy, but for just quickly doing stuff, I prefer getting the input from some ls.

To go into sub directories, you can use find:


find . -name '*.c' -type f -print \
| perl -n -e 'chomp;$x = $_;s/\.c$/.d/;$y=$_;rename $x, $y;'

a
You can also rename all the files that contain a certain string:

find . -name '*.html' -type f -print \
|xargs egrep -l form \
|perl -n -e 'chomp; $x=$_;s/\.html$/.form/;$y=$_;rename $x, $y;'

So you can combine with all kinds of shell commands and do really a lot of things in one line.

Of course you can use Raku, Ruby, Python or your favorite scripting language instead, as long as it allows some simple pattern matching and an efficient implicit iteration over the lines.

For such simple tasks there are also ways to do it directly in the shell like this

for f in *.d ; do mv $f `basename $f .d`.e; done

And you can always use sed, possibly in conjunction with awk instead of perl for such simple tasks.

Another approach is to just pipe the files into an texteditor that is powerful enough and create a one time script using powerful editing commands.
On Linux and Unix servers we almost always use vi, even people like me, who prefer Emacs on their own computer:

ls *.e > tmpscript
vi tmpscript

and then in vi


:0,$s/\(.*\)\(.e\)$/mv \1\2 \1.f/
ZZ

and then

sh tmpscript
rm tmpscript

So, there are many ways to achieve this goal and they are flexible and powerful enough to do really a lot more than just such simple pattern renaming.

If you work in a team and put these things into scripts, it might be necessary to follow a team policy about which scripting languages are preferred and which patterns are preferred. And you need to know the stuff that you write yourself, but also the stuff that your colleagues write.

Please, do not do

mv *.a *.b

It won’t work for good reasons.
On Linux and Unix systems the shell (usually bash) expands the glob expression (the stuff with the stars) into a list of strings and then starts mv with these strings a parameters. So calling mv with some file names ending in .a and .b, mv cannot have any idea what to do. When called with more than two parameters, the last one needs to be a directory where to move the stuff, so usually it will just refuse to work.

Share Button

GIMP

In spite of working mostly for server software and server setup using powerful non-graphic command line tools and scripting languages, it is sometimes fun to work with something very graphical. I did talk about Clojure Art, which is fun and creates interesting visual results and helps getting into the phantastic language Clojure. But more than twenty years ago I have discovered GIMP, which is the main image editor on Linux computers. I keep hearing that Photoshop is even a bit better, but it does not work on my computers, so I do not really care too much about it.

To be clear, I am not a professional image editing specialist, I just do it a bit for fun and without the claim of putting in all the knowledge about colors and their visual appearance, the functionality of gimp and image editing in general… I am just experimenting and finding out what looks interesting or good to me and how to work efficiently. Actually it brings together my three interests, programming, photography and bicycle touring, the combination of the latter two being the major source of my input material.

Now you start working with layers and with tools that increase or decrease the brightness, contrast and saturation of an image or of the layer being worked on. Now I would like to explore how certain functions can be brought into this. Either by putting them into my fork of gimp or into a plugin or into a script within gimp or a standalone script or program.

Some things that I find interesting and would like to explore: additional functions for merging layers. The function has the input of n (for example n=2) pixels from the same position and different layers and it produces another pixel. These are the twenty or thirty layer modes, that describe how a layer is seen on top of the next lower visible layer. So two images of the same size or two layers could be merged. It could be as nicely as in Gimp or just desctructively to make things easier. If it is worth anything for anybody else, maybe making it work as the current modes would be a noble goal. But for the moment it is more interesting what to do. A very logical thing to do is taking just the average of the two layers. So for rgb it could be the arithmetic mean of the r, g and b values of the n layers (or images) belonging to the same x-y-position. Now what would alpha values mean? I would think that they are weighting the average and the new alpha value could be the average of the input alpha values. Now we could use geometric, quadratic and cubic means and with some care concerning the 0 even harmonic means. Very funny effects could be created by combining these byte-values with functions like xor.

When working with any functions, it is always annoying that the r-g-b-values are always between 0 and 255 (or something like that). So this can be changed to real numbers, by doing something like

    \[s = \tan(\frac{(r-127.5)\pi}{256})\]

or to the non-negative real numbers by doing something like

    \[s = \tan(\frac{r\pi}{512})\]

Then some functions can be applied to these double values and in the end the inverse function will just be applied and result in a rounded and limited integral value from 0 to 255.

Do we need this? I do not know. But I think it would be fun to be able play around with some functions on one pixel, a vicinity of pixels, the same pixel-position from different layers and the like.

The nice thing is: we can see the result and like it or throw it away. Which function is correct or useful can be discussed and disagreed on. That is more fun than formally proven correctness, assuming of course, that the functions itself are implemented correctly.

I have not looked into the source code of plugins to tell what can reasonably be done without too much effort. But if someone reading this has some ideas, this would be interesting to hear about.

And finally we have one more advantage of GIMP, because it is open source and it is possible to make changes to it.

Share Button

How procurement can create value for IT projects

We know this, in many IT projects we need to make use of services and software and hardware that needs to be bought.

Actually it often makes a huge difference, what kind of deals are made and how efficient the projects can work on this basis.

I will just briefly tell a few stories and tell a bigger story by that.

A common pattern is a „preferred supplier“. It is nice to be a preferred supplier and in the phase when the partner is chosen and the contracts are made, companies often offer their best people and services to show how good it will be later to have them as preferred supplier. And then, when the deal has been fixed, they send the juniors for the same hourly rate and make a lot of profit. Or the price has been made so low, that only the juniors can be sent. This can be a problem in the long run, because it might get really difficult to get enough really good people in order to progress with strategic long term projects and not just maintenance of the daily business. Another interesting pattern can occur, when the preferred supplier is very strong with their employees in a project. Now they are getting some money from the hourly rates and in order to make profit the salaries should be significantly lower than this. In order for this to make sense for the employees they can provide non-monetary incentives, like some kind of career steps. Being in a powerful position in the project, the preferred supplier has some possibilities to choose who is in the project and who gets more responsible positions. So there is a temptation to kick out people who are not from their company and to provide these attractive positions not to the person, who would be the best choice for the customer, but to those whom they want to give an incentive. This is on the expense of their customer. So in the end of the day it is usually good to rely on multiple providers for „external“ people. There are serious companies who behave professionally and correctly even when they have become a „preferred supplier“, but this is not always the case.

When the preferred supplier is providing software, for example a database, it may be possible to get a really good deal for five years. Then in five years the deal needs to be extended and becomes magically more expensive. Especially if the company knows itself in the position of a „preferred supplier“. And when this issue is discovered, maybe even a year before, it is already too late to migrate. And then the expensive software needs to be used for another four years until it is again too late… And being from a big, impressive company does not necessarily make the software good. Counterexamples exist. In the case of databases I have seen companies that follow a strategy of multiple databases and that require a good reason for using the more expensive solutions. And magically the position of the buyer becomes much stronger when the deal needs to be extended for another five years. Maybe the overly expensive database will even be kicked out at some time. And yes, this expensive database has some really cool and pretty unique features. Unfortunately they only come in some enterprise edition that would be even much more expensive than the regular one, while open source databases provide decent, but less sophisticated variants of these enterprise features for a price that is less than the base version of the expensive database product. But, since the DB product cannot easily be changed, it is important to make a wise choice and to consider different options, including the more expensive ones, when starting a project. And to pick what makes sense for the specific needs.

Some interesting observations where made, when some preferred supplier made a really tempting offer for operating all the servers of a larger company. The annual price was really low. Much cheaper than doing it with their own team. Now suddenly the need arose, to store some really large amount of log files. I mean, I am talking about what could be stored on a few USB-discs, that could have been bought in the supermarket for a few hundred Euros. But of course this was forbidden, because the servers had to be run by the preferred supplier and even putting Linux on a few PCs that were no longer needed and attaching a few cheap disks would have been ruled out by the cheap overall contract. But this cheap solution would have been absolutely sufficient for the purpose. Now the diskspace could be bought from this supplier. Or more precisely rented. It was not a few hundred Euros, but a few hundred thousand Euros a year. Yeah, trey needed to make some money somehow… And a few hundred Euros once every few years, or maybe even a few thousand Euros every year would have been totally acceptable to pay by the project. But there are limits. You cannot do certain things under such conditions. The deal kills important possibilities of the IT people. I am not going to write, how this was resolved…

Another story, with a really cheap preferred supplier: They actually ran an important database for a stunningly low fixed base price. And on top of that it was paid per query. So what did they do? They designed the software in such a way that it used an Oracle database as a cache for the pay-per-query DB2 database. So the same query had to be made only once to DB2 as long as the data did not change. And when the data changed, the Oracle database just had to be cleaned up. Since this happened only a few times a year, this technically stupid architecture saved really a lot of money every year. Big money.

Yet another example: The management had already bought clearcase licenses. They were really expensive and the money was already gone. Now the setup that was used for clearcase and that was allowed by the licensing was not really optimized for part of the team working remotely. To do that efficiently would have required a much more expensive license that no one wanted to pay for. So every day synchronizing the software took like 30 to 45 minutes. And one team member had to work full time to maintain clearcase. There were some other pains, like it crashed when files contained only linefeeds instead of carriage return-linefeed and some other annoying details that I do not really remember. Just for the record, some of these issues have been fixed in later releases… And clearcase had a lot of really interesting features that were not at all used. The seriously useful features can all be found in git now, in a contemporary way, of course. But not in those days, when there was still neither git nor subversion. So some tests were performed and it looked like the free software CVS (which really sucks big time when compared with contemporary systems like git) would have worked much much better for the concrete project. But clearcase had to be used because it was so expensive and the money had already been paid.

So in the end of the day, when the procurement does make good deals, this can create a lot of value for the project and allow for efficient and innovative work and for solutions that make sense technically instead of finding tricks how ot bypass the worst parts of the contract.

So a good procurement team and a good communication with the technical staff that knows what is needed for their work is a big plus for everybody, for the project and for the company.

Share Button