There are several approaches, but the most popular ones are Newton’s method and an algorithmic formulation of how roots are taken manually, for those old enough to still have learned it in school. Earlier measurements that I did many years ago showed that the Newton approximation is slower, but it would be worth to do newer measurements.
So we have an equation or and want to find or a well defined approximation of when we know . Mathematically speaking we want to assume that is constant and we want to find an for which or . If we guess such an and then draw the tangent at the curve of the function at the point or , then the intersection point of the tangent can be used as the next approximation. This method converges in the case of these two functions (and some others) and is reasonably fast. Now the tangent has the linear equation
where and is the derivative of . We want to solve this equation for and thus we get
and thus
As an iteration rule
In case of the sqare root we can just start with an estimation by shifting half the length to the right, but avoiding zero, which is important because of the division. Then we get for an appropriate n
The last form is quite intuitive, even without calculus. As I said this converges usefully fast and there is tons of math around to describe the behavior, speed, precision and convergence of the calculations performed in this algorithm. Written in Ruby just for integers, this is quite simple. Convergence is simply discovered by the fact that the result does not change any more, which may fail in some cases, where intermediate results oscillate between two values, but just for the purpose of benchmarking it seems to be sufficient:
def sqrt_newton(x)
if (x == 0) then
return 0
end
y0 = x
u0 = x
while (u0 > 0) do
y0 >>= 1
u0 >>= 2
end
y0 = [1, y0].max
yi = y0
yi_minus_1 = 1
loop do
yi_plus_1 = (yi + x/yi) >> 1;
if (yi_minus_1 == yi_plus_1) then
return [yi, yi_minus_1].min
elsif (yi == yi_plus_1) then
return yi
end
yi_minus_1 = yi
yi = yi_plus_1
end
end
The newton algorithm tends to oscillate between two approximations, so this termination criteria takes into account , and and uses the lower of the two oscillating values. This results in calculating the largest integer such that and .
For the third root we get for an appropriate n
Again this is a useful way and there is math around on when to stop for a desired precision.
Read Wikipedia for the convergence issues.
There is another approach, that people used to know when doing calculations on paper was more important than today. For the decimal system it works like this:
(source Wikipedia)
1. 7 3 2 0 5

/ 3.00 00 00 00 00
/\/ 1 = 20*0*1+1^2

2 00
1 89 = 20*1*7+7^2

11 00
10 29 = 20*17*3+3^2

71 00
69 24 = 20*173*2+2^2

1 76 00
0 = 20*1732*0+0^2

1 76 00 00
1 73 20 25 = 20*17320*5+5^2

2 79 75
We group the digits to the left and to the right of the decimal point in groups of two. The highest possible square of an integral number that is below or equal to the leftmost group (03 in the example above) is used for the first digit of the result (1 in the example above). This square is subtracted and the next group is appended (200 in the example). Assuming that is the result already calculated and is what we have achieved after the subtraction and the appending of the next group, we search for a digit such that . is chosen in such a way that it yields the maximum possible wich is still . Subtracting from and appending the next group allows for the next iteration.
Now this can be turned into an algorithm. The first approach is to just switch from decimal system to binary system. Then for each iteration step we have to deal just with the possible values of and , which greatly simplifies the algorithm. Here is a simple ruby program that would do this:
def split_to_words(x, word_len)
bit_pattern = (1 << word_len)  1
words = []
while (x != 0  words.length == 0) do
w = x & bit_pattern
x = x >> word_len
words.unshift(w)
end
words
end
def sqrt_bin(x)
if (x == 0) then
return 0
end
xwords = split_to_words(x, 2)
xi = xwords[0]  1
yi = 1
1.upto(xwords.length1) do i
xi = (xi << 2) + xwords[i]
d0 = (yi << 2) + 1
r = xi  d0
b = 0
if (r >= 0) then
b = 1
xi = r
end
yi = (yi << 1) + b
end
return yi
end
It seems that the two solutions yield the same results, but the sqrt_newton
outperforms sqrt_bin
by a factor of two.
Now we should reconsider, if base 2 is really the best choice. Actually we can use any power of 2 as a base and efficiently work with that. Apart from the initial first step, which is done by using an extended version of sqrt_bin, the next steps are estimated by division and trying neighboring values to get the exact result. This makes use of the fact that the equation we need to solve
with the maximum fullfilling this equation, where is the base to which we are working, witch was or above and could now be a power of . As soon as has a certain size, the influence of becomes less relevant. We can consider the maximum posible value for z_n, which is and thus solve and , each for the maximum fullfilling the equation. This can be calculated by simple division. If the range between the two solutions is small enough, then each value in the range can be tried to find the actual accurate solution for and this is more efficient than working just bitwise. This method sqrt_word
seems to outperform sqrt_newton
for longer numbers, for example around 60 decimal digits with word_length=16. So the most promising approach seems to be to optimize the implementation and parameters of sqrt_word
. The issue of termination, which has been properly addressed in the newton implementation, is already dealt with in this implementation. For more serious analysis it would be interesting to implement the algorithms in C or even in assembly language. So this is the final result for square roots, with some checks added:
def check_is_nonneg_int(x, name)
raise TypeError, "#{name}=#{x.inspect} must be Integer" unless (x.kind_of? Integer) && x >= 0
end
def check_word_len(word_len, name="word_len")
unless ((word_len.kind_of? Integer) && word_len > 0 && word_len <= 1024)
raise TypeError, "#{name} must be a positive number <= 1024"
end
end
def split_to_words(x, word_len)
check_is_nonneg_int(x, "x")
check_word_len(word_len)
bit_pattern = (1 << word_len)  1
words = []
while (x != 0  words.length == 0) do
w = x & bit_pattern
x = x >> word_len
words.unshift(w)
end
words
end
def sqrt_bin(x)
yy = sqrt_bin_with_remainder(x)
yy[0]
end
def sqrt_bin_with_remainder(x)
check_is_nonneg_int(x, "x")
if (x == 0) then
return [0, 0]
end
xwords = split_to_words(x, 2)
xi = xwords[0]  1
yi = 1
1.upto(xwords.length1) do i
xi = (xi << 2) + xwords[i]
d0 = (yi << 2) + 1
r = xi  d0
b = 0
if (r >= 0) then
b = 1
xi = r
end
yi = (yi << 1) + b
end
return [yi, xi]
end
def sqrt_word(x, n = 16)
check_is_nonneg_int(x, "x")
check_is_nonneg_int(n, "n")
n2 = n << 1
n1 = n+1
check_word_len(n2, "2*n")
if (x == 0) then
return 0
end
xwords = split_to_words(x, n2)
if (xwords.length == 1) then
return sqrt_bin(xwords[0])
end
xi = (xwords[0] << n2) + xwords[1]
a = sqrt_bin_with_remainder(xi)
yi = a[0]
if (xwords.length <= 2) then
return yi
end
xi = a[1]
2.upto(xwords.length1) do i
xi = (xi << n2) + xwords[i]
d0 = (yi << n1)
q = (xi / d0).to_i
j = 10
was_negative = false
while (true) do
d = d0 + q
r = xi  (q * d)
break if (0 <= r && (r < d  was_negative))
if (r < 0) then
was_negative = true
q = q1
else
q = q+1
end
j = 1
if (j <= 0) then
break
end
end
xi = r
yi = (yi << n) + q
end
return yi
end
def sqrt_newton(x)
check_is_nonneg_int(x, "x")
if (x == 0) then
return 0
end
y0 = x
u0 = x
while (u0 > 0) do
y0 >>= 1
u0 >>= 2
end
y0 = [1, y0].max
yi = y0
yi_minus_1 = 1
loop do
yi_plus_1 = (yi + x/yi) >> 1;
if (yi_minus_1 == yi_plus_1) then
return [yi, yi_minus_1].min
elsif (yi == yi_plus_1) then
return yi
end
yi_minus_1 = yi
yi = yi_plus_1
end
end
This is the approach that has been built into the LongDecimal library, ignoring Newton. The examples have been added to github.
The algorithms can be extended to cubic roots or any higher roots. In this case, the th root of is calculated by starting with the maximal integral number with and the subsequently finding numbers z_j fullfilling an equation of the form . This is always easy to handle for base two, by just testing the two possible solutions. For higher bases and it involves solving an quadratic equation, once the numbers are high enough to neglect the term . For it is just possible to take the square root of the square root. For higher values of and bases other than it becomes really difficult to tame this algorithm. So I intend to constrain myself to square roots and cube roots. I have not explored, if it is useful to calculate the cube root with a higher base than and which approach provides the best performance for cube roots. Even the square root calculation can possibly be tuned a bit. Maybe this will be addressed in another article.
]]>Now we can think of using Java’s own Collections.unmodifiableList and likewise. But that is not really ideal. First of all, these collections can still be modified by just working on the inner of the two collections:
List
list.add("a");
list.add("b");
list.add("c");
List
list.add("d");
ulist.get(3)
> "d"
Now list in the example above may already come from a source that we don’t really control or know, so it may be modified there, because it is just „in use“. Unless we actually copy the collection and put it into an unmodifiableXXX after that and only retain the unmodifiable variant of it, this is not really a good guarantee against accidental change and weird effects. Let’s not get into the issue, that this is just controlled at runtime and not at compile time. There are languages or even libraries for Java, that would allow to require an immutable List at compile time. While this is natural in languages like Scala, you have to leave the usual space of interfaces in Java, because it is also really a good idea to have the „standard“ interfaces in the APIs. But at least use immutable implementations.
When we copy and wrap, we still get the weird indirection of working through the methods of two classes and carrying around two instances. Normally that is not an issue, but it can be. I find it more elegant to use Guava in these cases and just copy into a natively immutable collection in one step.
For new projects or projects undergoing a major refactoring, I strongly recommend this approach. But it is of course only a good idea, if we still keep our APIs consistent. Doing one of 100 APIs right is not worth an additional inconsistency. It is a matter of taste, to use standard Java interfaces implemented by Guava or Guava interfaces and classes itself for the API. But it should be consistent.
]]>There is some confusion as to what is meant:
The truth is, that the first Scala Days were organized in 2010 and took place in Lausanne. So it was 9 years ago, but the tenth event. For this reason it was done in Lausanne again, but in a much larger venue.
Just for the completeness:
This is the opening keynote by Martin Odersky. A video will be on youtube soon.
I visited the following talks:
And on the last day I visited the following talks and the diversity panel and the closing panel:
Here is the whole schedule.
Btw., links might expire when the next Scala Days will start using the site in early 2020. Where it will be in 2020 is still unknown, but since 2014 it has been a habit to do the even years in Berlin. I will try to come, if there are Scala Days 2020 in Europe.
]]>hashCode
, equals
and toString
. They are named like this in Java and in many JVM languages or they use similar names. Some languages like Perl and Scala provide decent mechanisms for the language to figure these out itself, which we do most of the time in Java as well by letting the IDE create it for us or by using a library. This solution is not really as good as having it done without polluting our source code and without using mechanisms like reflection, but it is usually the best choice we have in Java. It does have advantages, because it gives us some control over how to define it, if we are willing to exercise this control.
So why should we bother? equals
is an „obvious“ concept that we need all the time by itself. And hashCode
we need, when we put something into HashMaps
or HashSets
or something like that. It is very important to follow the basic contract, that hashCode
and equals
must be compatible, that is
And equals of course needs to be an equivalence relation.
There has been an article in this blog about „Can hashCodes impose a security risk?„, which covers aspects that are not covered here again.
An important observation is that these do not fit together with mutability very well. When we mutate objects, their hashCode and equals methods yield different results than before, but the HashSet and HashMap assume that they remain constant. This is not too bad, because usually we actually use very immutable objects like Strings and wrapped primitive numbers as keys for Maps. But as soon as we actually write hashCode and equals, this implies that we are considering the objects of this type to be members of HashMaps or HashSets as keys and the mutability question arises. One very ugly case is the object that we put into the database using Hibernate or something similar. Usually there is an ID field, which is generated, while we insert into the database using a sequence, for example. It is good to use a sequence from the database, because it provides the most robust and reliable mechanism for creating unique ids. This id becomes then the most plausible basis for hashCode, but it is null in the beginning. I have not yet found any really satisfying solution, other than avoiding Hibernate and JPAx. Seriously, I do think, that plain JDBC or any framework like MyBatis or Slick with less „magic“ is a better approach. But that is just a special case of a more general issue. So for objects that have not yet made the roundtrip to the database, hashCode and equals should be considered dangerous.
Now we have the issue that equality can be optimized for hashing, which would be accomplished by basing it on a minimal unique subset of attributes. Or it could be used to express an equality of all attributes, excluding maybe some kind of volatile caching attributes, if such things apply. When working with large hash tables, it does make a difference, because the comparison needs to look into a lot more attributes, which do not change the actual result at least for each comparison that succeeds. It also makes a difference, in which order the attributes are compared for equality. It is usually good to look into attributes that have a larger chance of yielding inequality, so that in case of inequality only one or only few comparisons are needed.
For the hashCode it is not very wrong to base it on the same attributes that are used for the equalscomparison, with this usual pattern of calculating hash codes of the parts and multiplying them with different powers of the some twodigit prime number before adding them. It is often a wise choice to chose a subset of these attributes that makes a difference most of the time and provides high selectivity. The collisions are rare and the calculation of the hash code is efficient.
Now the third method in the „club“ is usually toString(). I have a habit of defining toString, because it is useful for logging and sometimes even for debugging. I recommend making it short and expressive. So I prefer the format
className(attr1=val1 attr2=val2 att3=val3)
with className the name of the actual class of this object without package, as received by
getClass().getSimpleName()
and only including attributes that are of real interest. Commas are not necessary and should be avoided, they are just useless noise. It does not matter if the parantheses are „()“ or „[]“ or „{}“ or „«»“, but why not make it consistent within the project. If attribute values are strings and contain spaces, it might be a good idea to quote them. If they contain nonprintable characters or quotation marks, maybe escaping is a good idea. For a real complete representation with all attributes a method toLongString() can be defined. Usually log files are already too much cluttered with noise and it is good to keep them consise and avoid noise.
The question is, what we can do about this. First of all it is necessary to understand, where are places that can be used by more or less unknown users to enter data into the system, for example registration of new users or some upload of information of users that are registered already. What would happen in case of such an attack?
In the end of the day we do want to allow legitimate usage of the system. Of course it is possible, to discover and stop abusive usage, but these detectors have a tendency to be accurate and create both „false positives“ and „false negatives“. This is something that a regular security team can understand and address. We need to remember, that maybe even the firewalls itself can be attacked by such an attack. So it is up to the developers to harden it against such an attack, which I hope they do.
From the developer point of view, we should look at another angle. There could be legitimate data that is hard to distinguish from abusive data, so we could just make our application powerful enough to handle this regularly. We need to understand the areas of our software that are vulnerable by this kind of attack. Where do we have external data that needs to be hashed. Now we can create a hashcode as or , where we prepend the string with some „secret“ that is created a startup of the software, then apply the cryptographic hash and in the end apply a function that reduces the sha1 or sha256 or md5 hash to an integer. Since hash maps only need to remain valid during the runtime of the software, it is possible to change the „secret“ at startup time, thus making it reasonably hard for attackers to create entries that result in the same hashcode, even if they know the workings of the software, but do not know the „secret“. A possible way could be to have a special variant of hash map, that uses strings as keys, but uses its own implementation of hashcode instead of String’s .hashCode()method. This would allow creating a random secret at construction time.
I have only become aware of the weakness of predictable hashcodes, but I do not know any established answers to this question, so here you can read what I came up with to address this issue. I think that it might be sufficient to have a simple hashcode function that just uses some secret as an input. Just prepending the string with a secret and then calculating the ordinary .hashCode() will not help, because it will make the hashcode unpredictable, but the same pairs of strings will still result in collisions. So it is necessary to have a hashcode with the input string and the secret such that for each with there exists a with , so the colliding pairs really depend on the choice of the secret and cannot be predicted without knowing the secret.
What do you think about this issue and how it can be addressed from the developer side? Please let me know in the comments section.
]]>We have to look twice, though. A good software project invests about 30 to 50 percent of its development effort into unit tests, at least if we program functionality. For GUIs it is another story, real coverage would be so much more than 50%, so that we usually go for lower coverage and later automated testing, when the software is at least somewhat stable and the tests do not have to be rewritten too often.
For a library or some piece of code that is a bit less trivial, the effort for a good unit test coverage would be more around 60% of the total development, so actually more then the functionality itself.
Also for APIs that are shared across teams, maybe a bit more is a good idea, because complete coverage is more important.
So the unit tests alone would in this case need about 50% of the development effort. Adding documentation and some other stuff around it, it will become even a bit more than 50%.
In practice it will not work out without intensive cooperation, unless the APIs have been in use by tons of other organizations for a long time and proven to be stable and useful and well documented.
So what exactly are we gaining by this approach of letting another team write the code and just providing a good unit test suite?
]]>Such promises should not stop us from being critical and analyzing pros and cons.
But in order to achieve any benefit from micro services it is crucial to understand them well enough and to apply the concept well enough. Badly implemented micro service architectures just add the disadvantages of microservices to what we already have.
It is often heard that microservices should be made so small that it is „trivial“ to rewrite them. That may be hard to achieve and there are some good reasons why.
If we make microservices so small that they are easy to rewrite and we need only a few of them, probably our application is so trivial that we should question if it is at all necessary to split it into microservices or if we should rather build a monolith and structure it internally. On the other hand, if we have a big application, in the best case it can be combined of a huge number of such „trivial“ microservices. And we get a lot of complexity in the combination. Just imagine you are getting a house built. And the construction company just dumps a few truckloads of lego blocks in the construction site. They are all well designed, well tested, high quality and you just need to plug them together. I cannot imagine that this will ever be trivial for a huge number of services and a nontrivial application.
Another problem is that there are typically one or more spots of the application where the real complexity resides. This is not a complexity made by us, but it is there because of the business logic, no matter how well we look at the requirements, understand them, work on them to get something better that is easier to build. Either these microservices tend to become bigger or they are more connected with other parts of the system than would be desirable for a microservice. So we end up with one or a few relatively fat „micro“services and some smaller ones, the smallest ones actually relatively trivial to rewrite. But we can keep everything nonessential out of these central services and make them at least as simple as reasonably possible.
Now we do have issues. How do the services communicate? How do they share data? By the book each service should have its own database schema. And they should not access each other’s DB schemes, because services should be independent from each other and not be using the DB as integration layer. In practice we see this rule applied, but in the end there is one database server that runs all the schemes for all the microservices. But we could split that up and move the database closer to the service.
Now there is some data that needs to be shared between services. There are several ideas how to accomplish this. The most basic principle is that services should be cut in such a way that little data needs to be shared. But there is also the pattern of having a microservice that operates like a daemon and a companion service that can be used for configuring the daemon service. There are some advantages in splitting this up, because the optimizations are totally different, the deployment patterns are different, the availability requirements are different. And it can be a good idea to build a configuration on the config service, test it on a test system and them publish it to the productive service, when it is complete, consistent and tested. Still these two services closely belong together, so sharing between them is well understood and works well. More difficult is it when it comes to data between really different services. We assume that it has been tried to eliminate such access to data from other services and it only remains for a few cases. There can be data that really needs to be in one place and the same for everybody. We can for example think of the accounts to log in. They should be the same throughout the system and without any delays when changes occur. This can be accomplished by having one service with really good availability. In other cases we can consider one service as the owner of the data and then publish the data to other services via mechanisms like Kafka. This is powerful, but setting up a good and reliable Kafka infrastructure for high performance and throughput is not trivial. When it comes to multimaster data and bidirectional synchronization, it gets really hard. We should avoid that. But it can be done when absolutely needed.
When services talk to each other, we should prefer mechanisms like JMS or Kafka over REST or SOAP calls, because it avoids tight coupling and works better, if one service is temporarily not available. Also we should avoid letting one service wait for the response from another service. Sometimes REST calls are of course the only reasonable choice and then they should be used. We need to address issues as to what happens if the service is not available and how to find the service.
When moving to a microservice architecture it is very easy to „prove“ that this approach „does not work“ or has „only disadvantages“ over a monolithic architecture. But microservices are a legitimate way to create large applications and we should give this approach a chance to succeed if we have decided to follow this road. It is important to embrace the new style, not to try to copy what we would have done in a monolith, but to do things in the microservice way. And it is important to find the right balance, the right distribution and the right places for some allowed exceptions like REST instead of JMS or data sharing.
It was interesting. I attended the following talks:
The early internet did contain functionality to communicate between computers, but it did of course not include http, html and the WWW, which were introduced much later in the early nineties.
]]>This is something all sites should do in the next few months.
]]>