Some thoughts about testing

We all do a bit of testing, if we are in IT. Even if I would by no means call myself a professional test engineer or test manager, I could make some observation about the topic.

I did hear a lot of times, that testing is kind of the easy task in the IT industry, just for those, who cannot do the „real“ stuff. Or from the other perspective the area, where better carriers can be made, because the task by itself is not so demanding… And I have seen test teams that resembled this. But a good test team can add a lot of value and has a lot of hard tasks to solve for this.

Some ideas:

There was a test team that tested ticket vending machines. Mostly the software, but to some extent also the hardware. Out of around ten people one guy was finding about two thirds of the defects. This does not necessarily really mean two thirds, because it might include a lot of trivial errors, while the more deeply lying errors need much more time to uncover. But while usually one person was running tests on one machine, he ran a second set of tests on his laptop on a simulation to make use of the waiting time. And he asked for a better laptop, in order to run three tests at the same time, which was declined by the test manager, because „running two tests at the same time is more than enough“. Please, give good people the resources and empower them to contributing more. By the way, running two tests at the same time is not easy, I tried it myself with tests that had quite significant waiting time and ended up needing help of others, thus defeating the whole idea. Maybe it can be learned to some extent.

In another project, a server application with very demanding logic, algorithmic complexity and no GUI whatsoever was developed. Nobody can test such a software, especially not the people who cannot do the „real“ stuff. But the test engineer in this project was really good. He was part of the team during development and new very well, what the software was supposed to do and which features could already be tested. While we wrote the software, he figured out how he could test it and when we were ready, he was ready too with some extremely intelligent tests that really did a great coverage and with workable ways how to do testing at all in the first place. And this actually worked great in practice.

We all talk about test automation. This is a good thing and should be done. It should be considered wisely. While it is possible to automate end to end, if we really want to go there, this can be hard. I have seen that robots where used for testing ticket vending machines. Payment was simulated through software, but with better robots that might work as well and it is an important part. It is quite possible to automate GUI testing with tools like Selenium, Selenide and others, but this is getting extremely hard, depending on the GUI. And if the UX team want a minor change the tests might become worthless in a second and we have to start almost from scratch. Using IDs instead of positions or more generally a good abstraction and a reasonably good separation of logic and design helps a lot, but still the problem remains. I recommend to write such tests in a late phase, when the GUI is already pretty stable in terms of Design and UX. This is no place for test driven design, unless the UX-guys are very test-automation-affine. And trust me, they are not. They can be really good in creating good designs and that is their job.

I have seen projects, where there was not test team at all and software that was committed to „trunk“ just had to pass the unit test suite and went immediately to production in case of success. At least during regular working hours. Non trivial changes to trunk had to undergo a code review via a pull request before being merged into trunk and the unit test suite was of course really good. I am not saying that I recommend this, but when it is well done, it actually works, at least in some cases.

What is important for the test team is to know what is going on in the development, to know where the documentation is and to contribute to it, to have a feeling for the whole system to understand how good it is. Mathematics on the defect detection rate is not everything, even thought it can help. It is important to understand the people. It is important to understand the system landscape. And the business logic. And to write good test cases. And for many projects it is important to understand multiple languages, if the software should run in a multilingual or international environment. And in the end of the day it is a hard decision to make, if there can be a „go“ or if it needs to be delayed and by how much. The guys from the top management might already be extremely eager to release the software and probably they do have good reasons. So a wise decision on when it is ok to go is really hard to make and valuable.

There is a lot of things that good test teams can contribute to add value to the product. And a lot of „real“ stuff that can and should be done by them.

Share Button

Internetexplorer and Microsoft Edge discontinued

It seems that Microsoft will discontinue development of its Edge Browser, which was meant to be an replacement for Internet Explorer. Instead they will base it on Google Chromium, so they will use the same HTML- and JavaScript-Engine as Google Chrome and the open source variant Chromium. The HTML- and JS-Engines of Edge will not be used any more. If you like, you can of course say, Edge will continue to exist, just undergo some changes. It is of course a matter of perspective. But the new Edge will not have very much in common with Internet Explorer and the old Edge, but very much with the Google browsers. So the way I see it, they discontinue their own browser and their own browser engine.

Is it a good or a bad thing? I think, it is not very important any more, because both Microsoft browsers combined have a combined share of only slightly above 5% of the human web usage.

Links

Share Button

TruffleRuby

The language Ruby is one of the most beautiful languages. A lot of things can be done, it has a good level of abstraction, it has chosen some very good defaults, has provided some great ideas that I have not discovered in any other language that I know well and provides a lot of flexibility. But I could no longer recommend it for projects that might require a good performance. I won’t go into the issue of static typing vs. dynamic here. Ruby is following the dynamic typing path and if you think that is a bad idea at all, then it will never become your favorite. But this is an issue with pros and cons. The big disadvantage of Ruby is that it is not very good in terms of performance. The single threaded performance is somewhat better in many reasonable languages like Java, Python, Scala, Clojure, C, C#, F# and some others .. And it gets worse when we want to use multiple cores, because Ruby does not run them simultaneously, but uses a global lock which ensures that only one thread at a time is running. Or in case of JRuby just crashes or yields wrong results in certain mulithreaded programs that we could write.

One approach is to go for immutability as a default, which allows quite painless multithreading. Scala and Clojure follow this route, for example. It is hard to write good code with this constraint or to make good use of very local mutability without leaking it outside, but under these conditions multiple threads are just working fine without deadlocks, crashes or falsified results. Another approach is to just copy structures and leave its own copy to each thread. There are ways to do a lot on this path, but the copying costs a lot of memory and performance and it is not always a gain.

Now Ruby heavily relies on mutable structures for strings and collections. It is not reasonable to go for a total paradigm change in this aspect. But there are some ways to get good and safe and fast operations on these collections and strings without breaking this. One idea is to work with chunks of collections or strings. For strings, the string that we are working with is described as a concatenation of such strings. Many operations can be made by just concatenating multiple strings together and possibly replacing one of them with a copy that can be made as needed. This is called Rope. A similar approach can be applied to collections. Then a smart locking mechanism is applied to the shorter string or collections when needed, but many operations can avoid locking or block much less of the structure.

Also the compiler can analyze the program and simplify it to a great extent, compile it to the JVM, which in conjunction with hot spot optimizations will make it run really fast. Now this TruffleRuby is much faster than other Rubies, by a factor of about 10. It uses GraalVM and it actually supports a lot of C-extensions for libraries through the feature of GraalVM that they can be eventually compiled to the JVM. It does not work if extensions rely on implementation details of the Ruby structures in C and it often does not work for C-extensions that go to low level OS functionality. The current version of TruffleRuby is not really ready to use in conjunction with Ruby on Rails, which is kind of a no go, because Ruby is usually used in conjunction with Rails. My impression is that it will be possible to use it with Rails in a year or two.

Hearing of this in a talk by Benoit Daloze in the local Rails user group in Zürich was a great and positive surprise. Ruby gets interesting again.

Links

Share Button