Some thoughts about testing

We all do a bit of testing, if we are in IT. Even if I would by no means call myself a professional test engineer or test manager, I could make some observation about the topic.

I did hear a lot of times, that testing is kind of the easy task in the IT industry, just for those, who cannot do the „real“ stuff. Or from the other perspective the area, where better carriers can be made, because the task by itself is not so demanding… And I have seen test teams that resembled this. But a good test team can add a lot of value and has a lot of hard tasks to solve for this.

Some ideas:

There was a test team that tested ticket vending machines. Mostly the software, but to some extent also the hardware. Out of around ten people one guy was finding about two thirds of the defects. This does not necessarily really mean two thirds, because it might include a lot of trivial errors, while the more deeply lying errors need much more time to uncover. But while usually one person was running tests on one machine, he ran a second set of tests on his laptop on a simulation to make use of the waiting time. And he asked for a better laptop, in order to run three tests at the same time, which was declined by the test manager, because „running two tests at the same time is more than enough“. Please, give good people the resources to empower to contributing more. And running two tests at the same time is not easy, I tried it myself with tests that had quite significant waiting time and ended up needing help of others, thus defeating the whole idea. Maybe it can be learned to some extent.

In another project, a server application with very demanding logic, algorithmic complexity and no GUI whatsoever was developed. Nobody can test such a software, especially not the people who cannot do the „real“ stuff. But the test engineer in this project was really good. He was part of the team during development and new very well, what the software was supposed to do and which features could already be tested. While we wrote the software, he figured out how he could test it and when we were ready, he was ready too with some extremely intelligent tests that really did a great coverage and with workable ways how to do testing at all in the first place. And this actually worked great in practice.

We all talk about test automation. This is a good thing and should be done. It should be considered wisely. While it is possible to automate end to end, if we really want to go there, this can be hard. I have seen that robots where used for testing ticket vending machines. Payment was simulated through software, but with better robots that might work as well and it is an important part. It is quite possible to automate GUI testing with tools like Selenium, Selenide and others, but this is getting extremely hard, depending on the GUI. And if the UX team want a minor change the tests might become worthless in a second and we have to start almost from scratch. Using IDs instead of positions or more generally a good abstraction and a reasonably good separation of logic and design helps a lot, but still the problem remains. I recommend to write such tests in a late phase, when the GUI is already pretty stable in terms of Design and UX. This is no place for test driven design, unless the UX-guys are very test-automation-affine. And trust me, they are not.

I have seen projects, where there was not test team at all and software that was committed to „trunk“ just had to pass the unit test suite and went immediately to production in case of success. At least during regular working hours. Non trivial changes to trunk had to undergo a code review via a pull request before being merged into trunk and the unit test suite was of course really good. I am not saying that I recommend this, but when it is well done, it actually works, at least in some cases.

What is important for the test team is to know what is going on in the development, to know where the documentation is and to contribute to it, to have a feeling for the whole system to understand how good it is. Mathematics on the defect detection rate is not everything, even thought it can help. It is important to understand the people. It is important to understand the system landscape. And the business logic. And to write good test cases. And for many projects it is important to understand multiple languages, if the software should run in a multilingual or international environment. And in the end of the day it is a hard decision to make, if there can be a „go“ or if it needs to be delayed and by how much. The guys from the top management might already be extremely eager to release the software and probably they do have good reasons. So a wise decision on when it is ok to go is really hard to make and valuable.

There is a lot of things that good test teams can contribute to add value to the product. And a lot of „real“ stuff that can and should be done by them.

Share Button

Internetexplorer and Microsoft Edge discontinued

It seems that Microsoft will discontinue development of its Edge Browser, which was meant to be an replacement for Internet Explorer. Instead they will base it on Google Chromium, so they will use the same HTML- and JavaScript-Engine as Google Chrome and the open source variant Chromium. The HTML- and JS-Engines of Edge will not be used any more. If you like, you can of course say, Edge will continue to exist, just undergo some changes. It is of course a matter of perspective. But the new Edge will not have very much in common with Internet Explorer and the old Edge, but very much with the Google browsers. So the way I see it, they discontinue their own browser and their own browser engine.

Is it a good or a bad thing? I think, it is not very important any more, because both Microsoft browsers combined have a combined share of only slightly above 5% of the human web usage.

Links

Share Button

TruffleRuby

The language Ruby is one of the most beautiful languages. A lot of things can be done, it has a good level of abstraction, it has chosen some very good defaults, has provided some great ideas that I have not discovered in any other language that I know well and provides a lot of flexibility. But I could no longer recommend it for projects that might require a good performance. I won’t go into the issue of static typing vs. dynamic here. Ruby is following the dynamic typing path and if you think that is a bad idea at all, then it will never become your favorite. But this is an issue with pros and cons. The big disadvantage of Ruby is that it is not very good in terms of performance. The single threaded performance is somewhat better in many reasonable languages like Java, Python, Scala, Clojure, C, C#, F# and some others .. And it gets worse when we want to use multiple cores, because Ruby does not run them simultaneously, but uses a global lock which ensures that only one thread at a time is running. Or in case of JRuby just crashes or yields wrong results in certain mulithreaded programs that we could write.

One approach is to go for immutability as a default, which allows quite painless multithreading. Scala and Clojure follow this route, for example. It is hard to write good code with this constraint or to make good use of very local mutability without leaking it outside, but under these conditions multiple threads are just working fine without deadlocks, crashes or falsified results. Another approach is to just copy structures and leave its own copy to each thread. There are ways to do a lot on this path, but the copying costs a lot of memory and performance and it is not always a gain.

Now Ruby heavily relies on mutable structures for strings and collections. It is not reasonable to go for a total paradigm change in this aspect. But there are some ways to get good and safe and fast operations on these collections and strings without breaking this. One idea is to work with chunks of collections or strings. For strings, the string that we are working with is described as a concatenation of such strings. Many operations can be made by just concatenating multiple strings together and possibly replacing one of them with a copy that can be made as needed. This is called Rope. A similar approach can be applied to collections. Then a smart locking mechanism is applied to the shorter string or collections when needed, but many operations can avoid locking or block much less of the structure.

Also the compiler can analyze the program and simplify it to a great extent, compile it to the JVM, which in conjunction with hot spot optimizations will make it run really fast. Now this TruffleRuby is much faster than other Rubies, by a factor of about 10. It uses GraalVM and it actually supports a lot of C-extensions for libraries through the feature of GraalVM that they can be eventually compiled to the JVM. It does not work if extensions rely on implementation details of the Ruby structures in C and it often does not work for C-extensions that go to low level OS functionality. The current version of TruffleRuby is not really ready to use in conjunction with Ruby on Rails, which is kind of a no go, because Ruby is usually used in conjunction with Rails. My impression is that it will be possible to use it with Rails in a year or two.

Hearing of this in a talk by Benoit Daloze in the local Rails user group in Zürich was a great and positive surprise. Ruby gets interesting again.

Links

Share Button

Documentation…

As developers we get often asked to write more, better and earlier documentation.

While it is very important and useful, to have the right documentation when needed and more often than not we are lacking important documentation, it is also important to avoid writing too much documentation.

This applies to all levels of documentation, including comments in the code (javadoc, scaladoc, rubydoc, perldoc,… as well as inline comments), design documentation, Wiki/Confluence, external documents.

Some projects measure the quantity of documentation, for example. Or the coverage of javadoc. And people start writing automatically generated or trivial comments on getters and setters. While there can be a comment documenting the attribute placed with the getter instead of the attribute itself, this is usually a bad idea, because it just clutters the code with no gain at all. Commenting should be limited to the non-obvious. And code should be written in such a way, that there is not more non-obvious than necessary. So code that is readable and understandable with few comments is actually a better goal.

For APIs we might want some more documentation. But we should keep it as close to the code as possible and use tools like swagger or scripts to create API-documentation from the code or code and documentation from the API-specification. This has a better chance of staying up to date, while hand maintained documentation always tends to get outdated over the time.

It is good to describe the overall working of the system. But details should be looked up in the code, where that is a viable option.

It is important to describe, how to get started. And even better, if getting started is easy because of automation, scripts or tools that do it in a few, relatively easy steps. This is a goal that everybody seems to have, but nobody seems to succeed with.

It is important to describe, how certain problems are solved in the team or have been solved, in terms of how it should be done to conform with other solutions or how it can successfully done. Guidelines are probably a good idea. Even better, if they can reference as much as possible well established guidelines published in the internet and just describe the exceptions or details about which the public guidelines do not make a decision, but that should be done coherently within the project.

And the business side of the software should of course be described also in documentation.

The agile manifesto recommends: „Working software over comprehensive documentation“.
But this should not stop us from writing documentation that really helps us.

Links

Share Button

Tips and Tricks: Typing Unicode

I found this on netzwolf.info:

You can enter arbitrary Unicode characters (more precisely code points) in X11 on Linux, if you know their Hex-Code:

  • Press Shift-Ctrl (keep them pressed)
  • Press also the letter u
  • Release the Ctrl-Key
  • Release the u-key
  • Keep the Shift-key pressed
  • Enter the Hex-Code of the Code point with the number of hex digits needed
  • Release the shift key

Let’s try with the Cyrillic Alphabet, more precisely with its Unicode Block:

  • Ѐ U+0400
  • Ё U+0401
  • Ђ U+0402
  • Љ U+0409
  • А U+0410
  • Б U+0411
  • В U+0412
  • Г U+0413
  • Д U+0414

Not very fast, but it works quite well for a few characters. Just open the table in one window and use this key sequence in another one.

For frequently used characters it is a good idea to remap the keyboard with xmodmap.

Share Button

Orthodox Christmas 2018/2019

Orthodox Christmas in some countries, for example in Ukraine is on 7th of January.
So to all readers, who have Christmas on 7th of January:


С Рождеством! — Hyvää Joulua! — καλά Χριστούγεννα! — Buon Natale! — Prettige Kerstdagen! — З Рiздвом Христовим! — Merry Christmas! — Срећан Божић! — God Jul! — ¡Feliz Navidad! — ميلاد مجيد — クリスマスおめでとう ; メリークリスマス — Natale hilare! — Joyeux Noël! — God Jul! — Frohe Weihnachten! — Crăciun fericit! — Feliĉan Kristnaskon!

I created the message and the random ordering using Perl and the Schwartzian transform:

#!/usr/bin/perl
use Math::Random::Secure qw(irand);
my @list = ( "Prettige Kerstdagen!",
          "God Jul!",
          "Crăciun fericit!",
          "クリスマスおめでとう ; メリークリスマス",
          "God Jul!",
          "Feliĉan Kristnaskon!",
          "Hyvää Joulua!",
          "ميلاد مجيد",
          "Срећан Божић!",
          "καλά Χριστούγεννα!",
          "З Рiздвом Христовим!",
          "Natale hilare!",
          "Buon Natale!",
          "Joyeux Noël!",
          "Frohe Weihnachten!",
          "С Рождеством!",
          "Merry Christmas!",
          "¡Feliz Navidad!" );
my @shuffled = map{ $_->[0] }
               sort {$a->[1] <=> $b->[1]}
               map { [ $_, irand() ] }
               @list;
print join(" — ", @shuffled);

Share Button

How to draw lines, circles and other curves

These ideas were developed more than 30 years without knowing that they were already known at that time…

Today the graphics cards can easily do things like this in very little time. And today’s CPUs are of course really good at multiplying. So this has lost a lot of its immediate relevance, but it is a fun topic and why not have some fun…

Let us assume we have a two dimensional coordinate system and a visible area that goes from x_{\min} to x_{\max} and y_{\min} to y_{\max}. Coordinates are discrete.

In this world we can easily measure an angle against a (directed) line parallel to the x-axis, for example up to an accuracy of 45^\circ=\frac{\pi}{4}:

  • y=0 \wedge x > 0 \implies \alpha = 0 (= 0^\circ)
  • 0 < y < x \implies 0 < \alpha < \frac{\pi}{4}(=45^\circ)
  • 0 < y = x \implies \alpha = \frac{\pi}{4}
  • 0 < x < y \implies \frac{\pi}{4} < \alpha < \frac{\pi}{2}(=90^\circ)</li> <li>x = 0 \land y > 0\implies \alpha = \frac{\pi}{2}</li> <li>x < 0 \land y > 0 \land |x| < |y|\implies \frac{\pi}{2}< \alpha < \frac{3\pi}{4}(=135^\circ)
  • x < 0 \land y > 0 \land -x = y\implies \alpha = \frac{3\pi}{4}(=135^\circ)

So let us assume we have a curve that is described by a polynomial function in two variables x and y, like this:

    \[f(x, y) = \sum_{j=0}^m\sum_{k=0}^n a_{j,k}x^jy^k = 0\]

We have to apply some math to understand that the curve behaves nicely in the sense that it does not behave to chaotic in scales that are below our accuracy, that it is connected etc. We might possibly scale and move it a bit by substituting something like c_1u+c_2 for x and c_3v+c_4 for y.

For example we may think of

  • line: f(x,y)=ax+by+c
  • circle: f(x, y)=x^2+y^2-r^2
  • eclipse: f(x, y)=\frac{x^2}{a^2}+\frac{y^2}{b^2}-1

We can assume our drawing is done with something like a king of chess. We need to find a starting point that is accurately on the curve or at least as accurately as possible. You could use knights or other chess figures or even fictive chess figures..

Now we have a starting point (x_0, y_0) which lies ideally exactly on the curve. We have a deviation from the curve, which is f(x_0, y_0)=d_0. So we have f(x_n, y_n)=d_n. Than we move to x_{n+1}=x_n + s and y_{n+1}=y_n+t with s, t = \{-1, 0, 1\}. Often only two or three combinations of (s, t) need to be considered. When calculating d_{n+1} from d_n for the different variants, it shows that for calculating d_{n+1}-d_n the difference becomes a polynomial with lower degree, because the highest terms cancel out. So drawing a line between two points or a circle with a given radius around a given point or an ellipse or a parabola or a hyperbola can be drawn without any multiplications… And powers of n-th powers of x can always be calculated with additions and subtractions only from the previous x-values, by using successive differences:
d_{m,1}=(x-m)^n-(x-m-1)^n
d_{m,l+1}=d_{m+1,l}-d_{m,l}
These become constant for l=n, just as the lth derivatives, so by using this triangle, successive powers can be calculated with some preparational work using just additions.
It was quite natural to program these in assembly language, even in 8-bit assembly languages that are primitive by today’s standards. And it was possible to draw such figures reasonably fast with only one MHz (yes, not GHz).

We don’t need this stuff any more. Usually the graphics card is much better than anything we can with reasonable effort program. Usually the performance is sufficient when we just program in high level languages and use standard libraries.

But occasionally situations occur where we need to think about how to get the performance we need:
Make it work,
make it right,
make it fast,
but don’t stop after the first of those.

It is important that we choose our steps wisely and use adequate methods to solve our problem. Please understand this article as a fun issue about how we could write software some decades ago, but also as an inspiration to actually look into bits and bytes when it is really helping to get the necessary performance without defeating the maintainability of the software.

Share Button

2019 — Happy New Year

Gott nytt år! — Godt nytt år! — Felice anno nuovo! — Καλή Χρονια! — Щасливого нового року! — Срећна нова година! — С новым годом! — Feliĉan novan jaron! — Bonne année! — FELIX SIT ANNUS NOVUS — Gullukkig niuw jaar! — Un an nou fericit! — Frohes neues Jahr! — Happy new year! — ¡Feliz año nuevo! — Onnellista uutta vuotta! — عام سعيد

This was created by a Java-program:

import java.util.Random;
import java.util.List;
import java.util.Arrays;
import java.util.Collections;

public class HappyNewYear {

    public static void main(String[] args) {
        List list = Arrays.asList("Frohes neues Jahr!",
                                          "Happy new year!",
                                          "Gott nytt år!", 
                                          "¡Feliz año nuevo!",
                                          "Bonne année!", 
                                          "FELIX SIT ANNUS NOVUS", 
                                          "С новым годом!",
                                          "عام سعيد",
                                          "Felice anno nuovo!",
                                          "Godt nytt år!", 
                                          "Gullukkig niuw jaar!", 
                                          "Feliĉan novan jaron!",
                                          "Onnellista uutta vuotta!",
                                          "Срећна нова година!",
                                          "Un an nou fericit!",
                                          "Щасливого нового року!", 
                                          "Καλή Χρονια!");
        Collections.shuffle(list);
        System.out.println(String.join(" — ", list));
    }
}

Share Button

Christmas 2018


Feliĉan Kristnaskon! — Frohe Weihnachten! — God Jul! — Merry Christmas! — Joyeux Noël! — クリスマスおめでとう ; メリークリスマス — Срећан Божић! — Buon Natale! — Hyvää Joulua! — З Рiздвом Христовим! — ميلاد مجيد — С Рождеством! — Crăciun fericit! — ¡Feliz Navidad! — καλά Χριστούγεννα! — Natale hilare! — God Jul! — Prettige Kerstdagen!

As I said, I am learning some Python, so let’s use it. I created the message above with this program:

#!/usr/bin/python3
import random
arr = [
    "Frohe Weihnachten!",
    "Merry Christmas!",
    "God Jul!",
    "¡Feliz Navidad!",
    "Joyeux Noël!",
    "Natale hilare!",
    "С Рождеством!",
    "ميلاد مجيد",
    "Buon Natale!",
    "God Jul!",
    "Prettige Kerstdagen!",
    "Feliĉan Kristnaskon!",
    "Hyvää Joulua!",
    "クリスマスおめでとう ; メリークリスマス",
    "Срећан Божић!",
    "Crăciun fericit!",
    "З Рiздвом Христовим!",
    "καλά Χριστούγεννα!"
]
random.shuffle(arr)
print(" — ".join(arr))
print("\n")

Share Button

Indexing of Arrays and Lists

We index arrays with integers. Lists also, at least the ones that allow random access. And sizes of collections are also integers.
This allows for 2^{31}-1=2147483647 entries in Java and typical JVM languages, because integers are actually considered to be 32bit. Actually we could think of one more entry, using indices 0..2147483647, but then we would not be able to express the size in an signed integer. This stuff is quite deeply built into the language, so it is not so easy to break out of this. And 2’000’000’000 entries are a lot and take a lot of time to process. At least it was a lot in the first few years of Java. There should have been an unsigned variant of integers, which would in this case allow for 4’000’000’000 entries, when indexing by an uint32, but that would not really solve this problem. C uses 64 bit integers for indexing of arrays on 64 bit systems.

It turns out that we would like to be able to index arrays using long instead of int. Now changing the java arrays in a way that they could be indexed by long instead of int would break a lot of compatibility. I think this is impossible, because Java claims to retain very good backward compatibility and this reliability of both the language and the JVM has been a major advantage. Now a second type of arrays, indexed by long, could be added. This would imply even more complexity for APIs like reflection, that have to deal with all cases for parameters, where it already hurts that the primitives are no objects and arrays are such half-objects. So it will be interesting, what we can find in this area in the future.

For practical use it is a bit easier. We can already be quite happy with a second set of collections, let them be called BigCollections, that have sizes that can only be expressed with long and that are indexed in cases where applicable with longs. Now it is not too hard to program a BigList by internally using an array of arrays or an array of arrays of arrays and doing some arithmetic to calculate the internal indices from the long (int64) index given in the API. Actually we can buy some performance gain when resizing happens, because this structure, if well done, allows for more efficient resizing. Based on this all kinds of big collections could be built.

Share Button