Archiv der Kategorie: Linux

LF and CR LF in git and svn

Veröffentlicht am von
Antworten

Typically we observe that line endings of text file turn out to be „linefeed“ (LF or „\n“ or 0x0a or Ctrl-J) on Linux and Unix-like systems (including MacOSX), while they are „carriage return and linefeed“ (CR LF or „\r\n“ or 0x0d 0x0a or Ctrl-M Ctrl-J) on MS-Windows. See the little obstacles of interoperability.

This can become annoying and there is little reason for this. Most tools for editing, compiling and working with these files just understand both variants. It does become annoying when diffs are created between different files and even more so when scripts are turning out to have the „CR LF“ ending and the script interpreter given in the first line is not found, because the system tries to find one that has the otherwise invisible „CR“ in its file name. It also becomes kind of messy, when multiple CR-characters are present. CR-characters are annoying even on MS-Windows itself as soon as we use cygwin. Since in most cases the target system is a Linux system anyway and we just waste space with the unnecessary CR-character, it is actually in most cases a good idea to agree on not having these CR-characters at all in certain kinds of files.

The easiest way is to just set up git or subversion to change files from CR LF to LF on commit. So the repository only contains „clean“ files, at least from that moment onward.

This is accomplished in subversion by applying the following command on the files that we want to be kept on „LF“ only:

svn pset svn:eol LF

and then committing.

Git has a way to achieve this using the .gitattributes configuration file. So the .gitattributes file can contain something like this:

* text=auto eol=lf

Remark

Of course I recommend to use git instead of svn for new projects and to consider migrating existing projects from svn to git. But for this special aspect svn provides a slightly more powerful and more intuitive tool than git.

Links

Share Button

The little obstacles of interoperability

Veröffentlicht am von
2

Deutsch

A lot of things in today’s IT landscape have been unified and interoperability is much better than 20 years ago.

Some examples:

  • Networking: Today networking is TCP/IP. Even the physical cables with RJ45/Ethernet and the wireless networks have been standardized and all kinds of devices can use the same networks. In the old days there were tons of mutually incompatible proprietary network technologies like BitNet (IBM), NetBios (MS), DecNet (DEC), IPX (Novell),….
  • Character Sets: Today we have Unicode and a few standard encodings. At least for Web and Emails we have ways to provide meta information about the actual encoding. This area is not totally free of problems, but on a very good way. In earlier days we had to deal with different EBCDIC-encodings or with character sets that only fully supported English langauge and a few languages using the same alphabet without additional letters like „ä“, „ö“, „ü“, „ø“, „ñ“,… So there we have encountered a great progress.
  • Numbers: For floating point numbers and integers a relatively small set of standardized numerical types has been established, that are used more or less everywhere and behave almost in the same way. The issue of integer overflow remains problematic, though.
  • Software: In the old days software was written for a specific machine, one CPU architecture and one OS. Today we have common platforms on different kind of hardware: Linux runs on almost any physical and virtual hardware from mobile phones and routers to super computers with practically the same kernel. Java, Ruby, Perl, Scala and some other programming languages are available on a range of platforms and provide some kind of abstract platform. And the web is often a good way to develop applications once for a large range of devices.
  • File system: At least we now have a common understanding what a file system looks like. There are some OS specific specialties, but the general idea is still the same. It is possible to share file systems between different operating systems by using technologies like Samba.
  • GNU-Tools: The GNU-Tools (bash, ls, cp, mv,……..) have become the standard on Linux boxes and they are way superior to the traditional Unix tools of the same role and name, that we can still find in Solaris, for example. You can (and should) install them on any Unix and even via cygwin on MS-Windows.

Interoperability is today for many of us interoperability between Linux (and possibly some other rare Posix/Unix-like systems) and Win32/Win64 (MS-Windows).

Experienced Linux users are used to having the forward slash („/“) as file path separator. The backslash is used for escaping special characters. In the MS-Windows-world we often see the backslash („\“) as file path separator. That is enforced in the CMD-window, because it does not pass through the forward slash. My experience with low level Win32/Win64 libraries shows that they understand both variants equally well. Anyway do Ruby, Perl, Java, Cygwin and others support the forward slash. So there is hardly ever any need to check the OS and use backslash or forward slash depending on that, other than for cmd/bat-scripts, but who wants them for more then five lines? I strongly recommend just using the forward slash also under MS-Windows when writing programs in Java, Perl, Scala, Ruby, … It makes life also easier, because the backslash often has to be doubled and it is sometimes hard to keep track of how many backslashes need to be written and to read it.

The line ending is a bit tricky. Linux and Unix use just a „Linefeed“ („\n“=Ctrl-J). For MS-DOS and MS-Windows the combination „Carriage-Return+Linefeed“ („\r\n“=Ctrl-M Ctrl-J) has become the default. Most of today’s programs do not care and understand both variants more or less equally well. Only Notepad does get confused with Linefeed-only files, but notepad is a bad choice anyway. Better Editors (gvim, emacs, ultraedit, scite, …) exist. On the other hand we get problems with the MS-Windows-line-termination in the case of executable scripts. Usually the contain a first line like „#/usr/bin/ruby“. The OS uses that as hint on how to execute them, in this case by calling /usr/bin/ruby. If the line ends with Ctrl-M Ctrl-J, then the OS tries to find a program „/usr/bin/ruby^M“ (^M = Ctrl-M = „\r“), which of course does not exist and we just get an obscure error message.

It is easy to do the conversion:

$ perl -i~ -p -e ’s/\r//g;‘ script

Or the other direction:

$ perl -i~ -p -e ’s/\n/\r\n/g;‘ textfile

For those who use subversion there are ways to enforce a certain way of line endings. Even git supports this.

Share Button

Encryption of Disks

Veröffentlicht am von
Antworten

Today we should use encryption of disks for many situations.

I recommend at least encrypting disks of portable computers that contain the home directory and portable USB disks. They can easily get stolen or lost and it is better if the thief does not have easy access to the content. We should even consider encrypting swap partitions.

There are many ways to do this on different operating systems and actually I only know how to do it for Linux. A possible approach for Windows is to run MS-Windows in a virtual box inside Linux and just profit from the Linux-based encryption. That is what I do, but I do not use my MS-Windows very much. About Apple computers I have no knowledge, please go to the site of somebody else for encryption of disks for them. I know that there is an option available for this, but I do not know how to use it and how good it is.

I prefer to rely on open source solutions for security related issues, because it is harder (but not impossible) to put in malicious components into the software and it is easier to find and to fix them. This is a general point that serious security specialists tend to make that it is better to rely on good and well maintained open source software for security than on closed software of which we do not know the wanted and unwanted backdoors and vulnerabilities.

The way it works in Linux is that we encrypt a disk partition. It can then be accessed after providing a password. It is possible to provide several alternative passwords. The tools to use are dm-crypt, a kernel module, LUKS, cryptsetup, and cryptmount.

It can be done like this (example session) for an external drive that appears as /dev/sdc. Please be extremely careful not to erase any data that you still need or hide data behind a password that you do not know…


# check the partitions
$ fdisk -l /dev/sdc

# encrypt the partition and provide a password:
$ cryptsetup luksFormat /dev/sdc1

# access the partition
$ cryptsetup luksOpen /dev/sdc1 encrypted-external-drive-1

# format it with whatever file system you want to use
$ mkfs.ext4 /dev/mapper/sdc1 encrypted-external-drive-1
# or
$ mkfs.btrfs /dev/mapper/sdc1 encrypted-external-drive-1
# or whatever you prefer..

Now each time the disk is mounted, the password needs to be provided.

The issue which file system is best might be worth writing about in the future, it is not in this article.

Links

Share Button

Microsoft stops Windows Mobile

Veröffentlicht am von
1

It seems that Microsoft is ending the development of Windows Mobile. After having tried with some effort to get into the mobile operating system business, it seems that the market share is now less than 1%, with Android being >85% and iOS close to 15% according to IDC. Because the market is so large, it would be possible to run a profitable business even with a low market share, but this is probably hard for a big company and it seems more attractive to concentrate on other areas. It is good to have some good mobile apps available for the platform and that is an area where Android and iOS shine, while doing a third app for Windows phone is a bit unusual. A funny detail is that a retired guy, who was once the founder of Microsoft and who is still associated with that company by some people uses an Android phone for himself. But he is retired, so that is no longer too important.

The story is a bit weird, though… In 2010 Stephen Elop became the boss of Nokia. At this time Nokia had a market share of around 50% in mobile phones covering a wide range from tiny „non-smart“ phone to high end smart phones. They were mostly using Symbian as OS, but the transition to Maemo and MeeGo, like Android Linux variants, was on a good way and it would have been worth seeing where this might go. At this time it was already quite clear that MS Windows phone/Windows mobile was a failure. All efforts concerning Linux-based systems were stopped and Symbian was announced as being a dead end and the strategy was to move to MS-Windows phone only. Most likely this was done because Stephen Elop had more loyalty to Microsoft than to the company that he was running. To my knowledge this never became a case for the courts, but one might assume some criminal energy behind this. And some stupidity of the stock holders, who selected this person as CEO. Some time later the mobile phone branch of Nokia went down and was bought for very little money by Microsoft. After that acquisition it was further downsized and will probably go to zero soon, because Microsoft does not have interest to develop new hardware.

As it seems, there are mobile phones with the brand „Nokia“ again. HMD, a company in Finland designs them, pays to the company Nokia some money to use their brand. And of course they use Android.

It seems that the issue of having to write a mobile app twice because of Android and iOS has become a bit less pressing. Firstly, currently with Android and iOS, there are two dominating platforms that kind of allow neglecting all the others. And besides mobile web applications that can pretty much behave as rich clients through modern JavaScript frameworks, there are „fake apps“ that are actually pretty much browsers without a visible URL bar and programmed to only surf their home site. And even native apps are now increasingly being developed in Swift for iOS and Kotlin for Android, which seem to be quite similar, at least at a conceptional level. There are very interesting mobile operating systems like Sailfish OS, that are worth looking into, but for a market share of about 85% supporting Android only is sufficient and for a market share of 99% Android and iOS together are sufficient in many cases.

Share Button

Shell Scripts

Veröffentlicht am von
Antworten

Shell scripts can be useful for writing small stuff like combining a few commands to pipes or doing a bit of „back ticking“. Even simple loops and if-conditions are possible. And if we want, it is almost a full programming language. A bit hard to tame, maybe, but quite a lot of stuff is possible. Those who like to know more about it may look into startup scripts of typical java software. Often a .bat and a .sh file are provided, where the right jvm is found, the classpath and the execution path and maybe some other environment are put together. In the end the .sh-file is quite a long and unreadable horror story and the .bat file is even much worse, because the cmd-language is just a lot more primitive and less capable and requires even worse hacks.

There are ways to make shell scripts more readable, which by themselves are truly admirable, but I think that route is wrong. We can learn all the Shell functionalities and understand bit by bit even more complex shell scripts, but I think for non trivial shell scripts it is time to switch to real programming languages instead. Scripting languages, of course, for example Perl, Ruby, Python or Lua. We may still execute „shell commands“, that are actually programs in /bin, /usr/bin or /usr/local/bin where they are powerful and more concise than writing purely in that programming language. But a magic for putting together a classpath is much cleaner in the Perl programming language than in pure bash (or worse cmd/bat).

This is of course another example of the Golden Hammer anti pattern. We should balance our tool box. Not add specific tools for making any minor task a bit easier on the expense of supporting one more tool, but keep a broad range of tools that in conjunction are very powerful. For example I would retire awk and sed and use either Perl or Ruby instead. We only have to keep them around because a lot of system tools that are just there still rely on them, but for a team I would deprecate awk and sed for new scripts or even for enhancing existing scripts. Bash would be ok only for small scripts, you can invent a line number or a maximum complexity, but for very short scripts I think bash is a legitimate tool.

Switch to Perl, Perl6, Ruby or… when you encounter any of the following:

  • The scripts is getting kind of long (>= 100 lines)
  • You find yourself modularizing it with functions
  • You find yourself using non trivial perl, ruby, sed or awk within the script, for example regex-stuff
  • The script need interaction
  • The scripts needs arrays, numbers or other types
  • More than one or two trivial if-statements or loop-statements are needed
  • Database access is done by the script (SQL or NoSQL)
  • String encoding becomes relevant
  • Quoting levels become an issue

This post was inspired by a similar post on the Isoblog by Kris. And the Shell Style Guide of Google is quite good especially in limiting the area where shell scripts are acceptable.

Share Button

Bring your own Device

Veröffentlicht am von
Antworten

This issue is quite controversial and it applies to laptops, tablets and smart phones.
Usually the „bringing“ is not really an issue, you can have anything in your bags and connect it via the mobile phone network as long as it does not absorb the working time.
But usually this implies a bit more.
There are some advantages in having company emails and calendar on a smart phone. This is convenient and useful. But there are some security concerns that should be taken serious. How is the calendar and the emails accessed? How confidential are the emails? Do they pass through servers that we do not trust? What happens, if a phone gets lost?
This is an area, where security concerns are often not taken too serious, because it is cool for top manager to have such devices. And they can just override any worries and concerns, if they like.
This can be compensated by being more restrictive in other areas. 😉
Anyway, the questions should be answered. In addition, the personal preferences for a certain type of phone are very strong. So the phone provided by the company might not be the one that the employee prefers, so there is a big desire to use the own phone or one that is similar to the own phone, which depends on the question of who pays the bills, how much of private telephony is allowed on the company phone and if there are work related calls to abusive times.
Generally the desirable path is to accept this and to find ways to make this secure.

The other issue is about the computer we work with. For some kind of jobs it is clear that the computer of the company is used, for example when selling railroad tickets or working in the post office or in a bank serving customers.

It shows that more creative people and more IT-oriented people like to have more control on the computer they work with.
We like to have hardware that is powerful enough to do the job. We like to be able to install software that helps us do our job. We like to use the OS and the software that we are skilled with. Sometimes it is already useful to be able to install this on the company computer or in a virtual computer within the company computer. Does the company allow this? It should, with some reasonable guidelines.

Some companies allow their employees to use their own laptops instead. They might give some money to pay for this and expect a certain level of equipment for that. Or just allow the employees to buy a laptop with their own money and use it instead of the company computer. They will do so and happily spend the money, even though it is wrong and the company should pay it. But the pain of spending some of the own money is for many people less than the pain of having to use crappy company equipment.

This rises the question of the network drive Q:, the outlook, MS-Word, MS-Excel,…
Actually this is not so much an issue, at least for the group we are talking here. Or becoming less of an issue.
Drive Q: can quite well be accessed from Linux, if the company policies allow it. But actually modern working patterns do not need this any more.
We can use a Wiki, like MediaWiki or Confluence for documentation. This is actually a bit better in many cases and I would see a trend in this direction, at least for IT-oriented teams.
Office-Formats and Email are more and more providing Web-Applications that can be used to work with them on Linux, for example. And MS-Office is already available for Linux, at least for Android, which is a Linux Variant. It might or might not come for Desktop Linux. LibreOffice is most of the time a useful replacement. Maybe better, maybe almost as good, depending on perspective… And there is always the possibility to have a virtual computer running MS-Windows for the absolutely mandatory MS-Windows-programs, if they actually exist. Such an image could be provided and maintained by the company instead of a company computer.

It is better to let the people work. To allow them to use useful tools. To pay them for bringing their own laptop or to allow them to install what they want on the company laptop. I have seen people who quit their job because of issues like this. The whole expensive MS-Windows-oriented universe that has been built in companies for a lot of money proves to be obsolete in some areas. A Wiki, a source code repository, … these things can be accessed over the internet using ssh or https. They can be hosted by third parties, if we trust the third party. Or they can be hosted by the company itself. Some companies work with distributed teams…

It is of course important to figure out a good security policy that allows working with „own“ devices and still provide a sufficient level of security. Maybe we just have to get used to other ways of working and to learn how to solve the problems that they bring us. In the end of the day we will see which companies are more successful. It depends on many factors, but the ability to provide a innovative and powerful IT and to have good people working there and actually getting stuff done is often an important factor.

Share Button

tmp-directories

Veröffentlicht am von
Antworten

On all computers we have some concept of a tmp-directory. Typically it is /tmp on Linux- and Unix-systems and something like C:/TEMP plus some subdirectory in each users home directory on MS-Windows.

In terms of software development this tends to be some dark area. Programs like to create some files there, store some stuff there and then maybe remove it, maybe not. And we do not know for sure, when we can delete these files and we actually do not want to care. Linux and Unix-Systems sometimes clear their tmp-directories on reboot, while providing an additional /var/tmp-directory, that survives reboot. Sometimes the tmp-directory is deducted from shared memory, so it is kind of a RAM-disk, but usually stored in the swap partition (or swap file) of our OS. Now this cleanup on reboot does not help too much, when we want to keep our system running for a long time.

These days most computers are somehow dedicated. Either they are virtual computers that run exactly one server application or a set of closely related server applications. Or it is a mobile phone, tablet or desktop computer that is typically used by only one person. But still we should not forget that the system should allow being used by several applications and by several users. So sharing the same tmp-directory for everyone can cause some conflicts. The Unix- and Linux-family has a way of setting file permissions for the tmp-directory itself and for its entries that stop users from reading, changing or deleting each others files, but still there is some concurrency about using the namespace of this one directory, which is usually quite elegantly bypassed by each software by using smart naming or by having the OS create unique names. But I would not consider it ideal. On the other hand, sometimes we might actually want to use the tmp-directory to share something between users or between processes, where this one tmp-directory might come in handy.

The approach of having a separate tmp-directory in each home directory and in a sub directory of each server application’s installation is tempting, because it separates name spaces, allows to disallow reading the directory entries by others and does not mix totally unrelated stuff in one directory. There is a drawback to this. We usually have different storage technologies. Some are optimized for reading, maybe even avoiding redundancy, because the system can be reinstalled. Some use sporadic writing, some are strictly read-only. And some use a lot of reading and writing. Some data is transient, some can be easily restored and some data needs to be stored redundantly to be safe. Depending on that we should aim to put it on Flash disks, or on a different RAID setup of hard disks. This is getting harder with virtualization, but eventually we can get to the point where virtual computers have disks of different characteristics, that are mapped to the appropriate hardware.

So there is no real good answer to this question, but I think that a tmp-directory that is separate from the home directory, but specific to each user, would be the best approach. Will this change? Probably not so easily. But maybe in some distant future.

Share Button

Tablet Computers

Veröffentlicht am von
Antworten

The idea of tablet computers is actually quite old and it has been tried a couple of times, at least up to prototypes. Probably a certain level of hardware and software was needed to make them both useful and affordable for enough people to become a mass product. This is actually a quite common thing. Some person, group or company has invented something really good, but they were not able to provide a sufficiently reliable, useful and affordable product to the market or just were not able to leave their home market efficiently. There are just a few examples for this, that I have observed.

  • Tilting trains have been tried in Germany, UK, Italy, Spain, Sweden Switzerland, Canada, France and Japan, in some countries several times. Many efforts become dead ends because the technology was not easily built in an affordable and reliable and maintainable way, so the mechanism was disabled or the trains were put out of service way too early. Italy actually made this technology work, but some of the train sets suffered serious deficiencies in quality, reliability and maintenance. Spain did the Talgo, which is less ambitious, because it uses gravity instead of an active mechanism and provides for a weaker effect. Sweden developed the X2000 trains, which seemed to work more or less well, but were quite expensive. But finally it seems that companies are able to produce good trains with this technology, like the relatively new Swiss ICN-trains.
  • A British company had produced trailer bikes for children already in the 1930s. They have one wheel and are attached to a parent’s bike. These were hard to get and they were almost unknown, even though the idea is great. In the 1990s German companies started to adopt the concept and actually produce them in good quality and sell them internationally, which was off course easier than 60 years earlier. They are now a common concept.
  • In the 1970s many bicycles had three speed hub gears. Derailleur gears already existed, but they were hard to use and fragile. For steeper roads it was possible to use a larger sprocket and to be able to climb slopes at the expense of lacking higher gears for flat sections. A British company produced a 5 speed hub gear, but it was extremely difficult to get and the quality was so poor that it would be almost half of the time in repair for a more active cyclist. Today we see mature hub gears with more than ten gears, but the derailleur technology has also become mature enough for the main stream.

So there are several requirements to success.

Another interesting aspect is that the actual usage might become different than anticipated. I understand that the tablet computers where sold as a „better replacement“ for PCs and Laptops in certain areas. I do not think that this is reasonable. Having a keyboard and a larger screen is usually better and it makes sense to transport a small or even a larger laptop. I have often had an external keyboard on top of the laptop, when I could afford to transport it and anticipated a heavy use. The netbook was so small that it did not hurt to have it in the luggage, but it was eventually hard to expand the memory and to get a replacement. A relatively small laptop still serves the purpose when a real computer is needed, but luggage is constrained.

The tablet computer does have some features that make it worth having one on top of a good phone and different sizes of Laptops. I am using an Android tablet, which is the most common OS for tablets, but there are off course some others, which I do not know well enough to write about them.

It is easier to switch between keyboard types. I am using the Cyrillic keyboard a lot and the computer with which I am writing this text has two external keyboards attached. I can switch with a key sequence, but this approach has its limitations. Probably buying a Laptop in Russia and just knowing the German keyboard without relying on the symbols on the keys would work for me. But the tablet makes this work with very little setup, while buying a physical Cyrillic keyboard in Switzerland is a bit harder, but still easy and buying a Laptop with Cyrillic keyboard layout does need some effort.

When doing small stuff, mostly reading or even some smaller emails, this is much better than the phone, but it can be used in the train, in the park, anywhere, where it is possible to sit. A laptop requires some kind of a table to be reasonably useful. There are seats with tables in the train, but that is a matter of luck to get one.

Finally we currently have a lot of Android Apps. They could be written for „normal“ desktop Linux as well or as web applications. Maybe that will happen. But currently some of them are available for Android, but not or not in a useful way for desktop Linux. This may change and it heavily depends on what we are actually using. But in my case it is true and it proved to be helpful to have the larger screen than on the Android phone.

Concerning the SIM card, I actually went the extra mile in terms of higher price and more effort for buying it in order to get a SIM card slot. I have not used it very much, because the extra SIM card is kind of expensive, moving SIM cards between devices is inconveniant and using the Android phone as a WiFi-Router seems to work well enough. But maybe this is useful when travelling a lot with SIM-cards from many countries to use just all the slots in older and newer phones and tablets and to use the device with the currently preferred SIM card as the WiFi router for all the others.

And finally it can be said that we can now buy fairly affordable good tablet computers. What I am missing is that tools from desktop Linux are usually not available on Android or only in a limited version. But the most common applications, a web browser and an email client are off course working on both…

Share Button

Running a Large Number of Servers

Veröffentlicht am von
Antworten

These days we often have to run a large number of servers, and the times where we could afford to manually log into each one to do system administration tasks are mostly over.

It turns out that there are always different approaches to deal with this. In most cases we are talking about virtual hosts, so we have a layer between via the visualization that can help us. We can have a number of master images and create virtual images from those even on demand in a matter of a minute. In case of MS-Windows it is an issue that they have some internal UUID as host-id which should be unique and which is heavily spread throughout the image, but this issue can be ignored if we do not worry about windows domains. Usually we do and I leave dealing with these issues to MS-Windows-experts.

Talking about Linux, we only need to make sure that the network interface is unique, which it is if we use hardware and do not mess around with it, but it is not necessarily if we use visualization and virtual network devices. This issue needs to be addressed, but it is well supported by common visualization tools. Another point is the host name. This is not too hard, because we only need to change it in one or two places, which can easily be done by a script. We can mount the image and do the change. Now the image can contain a start-up script that discovers on boot that it is a fresh copy and uses its host-name to retrieve further setup from some server. And we just have to maintain there which host has which setup. These can be automated to a very high extent. Then we can for example request a certain number of servers with certain software and configuration via an web interface. This creates new host-names, stores the setup with these host-names in its setup table, creates the virtual images, deploys them on any available hardware server and once they have stared they retrieve their setup from the server. We can also have master images that already contain certain predefined setups so that this second step is only needed for minor adjustments. We have to assume that these exist. Yes, this is called cloud technology.

If we keep the data somewhere else, these servers can be discarded and new ones can be created, so there is no need to do too complex stuff on them. Of course we want to run our software on them. So the day long procedure to install our software is not attractive any more. We need mechanisms that can be automated.

Running real hardware is a bit more demanding and for larger servers that might even be justifiable, because they do a lot of work for us. Quite often it is possible to actually do mechanisms quite similar to the virtual world even on real hardware. It is possible to boot the machine from an USB-stick which copies fetches an image and copies it on disk. Again only the host-name needs to be provided and then the rest can be automated. Another approach is to initially boot via the network, which is an option that most of us rarely use, but which is supported by the hardware. For running a large server farm such a hardware and bios setting can just be initially the default and from there machines can install and reconfigure themselves. In this case we probably need to use the Ethernet-address of the network device as a key to our setup table and we need to know what Ethernet addresses are in use. It is a big deal to set up such an environment, but once it is running, it is tremendously efficient. Homogenous hardware is of course essential, maybe an small number of hardware setups, but not a new model with each delivery. It is not enough that the new hardware is named the same as the old one, it needs to be able to run the same images without manual customization. It is possible to have a small number of images, but having to supply already different images for different server setups multiplying there number with the number of the hardware setups can grow out of control, if one of the numbers or both become too large.

Now we also have ways to actually access oure servers. there have been tools to run a shell just simultaneously on n hosts to do exactly the same at once. This is fine if they are exactly the same, but this is something we need to enforce or we need to accept that servers deviate. There are tools around to deal with these issues, but it is actually quite reasonable to do a script based approach. What we do is using ssh-key-exchange to make sure that we can log into the servers from some admin server without password. We can then define a subset of the set of our servers, which can be one, a couple, a large fraction, all or all with a few exceptions, for example. Then we distribute a script with scp to all the target machines in a loop. We run this on each target machine using ssh and parse the outputs to see which have been successful and which not. Here it is usually a good idea to have a farm of test servers to try this out first and then start on a small number of servers before running it on all of them.

The big bang philosophy of applying a change twice a year on the whole server landscape is not really a good idea here, because we can loose all our servers if we make a mistake and this can be hard to recover, although still have the same tools and scripts even for that, unless we really screw things up. So in these scenarios software that supports the interfaces of the previous version for its communication partners is useful, because it allows to do a smooth migration.

Just to give you a few hints: During some coffee break I suggested that Google has around a million servers. Even though there is no hard evidence for this, because this number seems to be confidential and only known to Google employees, I would say that this is a reasonable number. For sure they cannot afford a million system administrators. The whole processes needs to be very stream-lined. Or take the hosting provider where this site is running on. It is possible to have virtual web-hosts, in this case it is multiple sites running on the same virtual or physical machine sharing the same Apache instance with just different directories attached to different URL-patterns. This is available for very little money, again suggesting that they are tremendously efficient.

Share Button

Microsoft SQL Server will be available for Linux in 2017

Veröffentlicht am von
4

Deutsch

Microsoft has officially announced that their database MS SQL Server will become available for Linux in 2017.

I think the time has come for this. Since the departure of Steve Ballmer Microsoft has become a little bit less religous and more pragmatic. There are good reasons to be skeptical about companies like Microsoft and Oracle, but having more competition and more choice is a good thing. Maybe the database product from Oracle is slightly better than MS SQL Server, but there are very few projects where this difference really matters. So now we have five important relational DB products: DB2, Oracle, MS-SQL-Server, PostgreSQL and MariaDB (the successor of mySQL). When starting a new project with no specific constraints about the DB I would usually look at PostgreSQL first, because it is a feature rich and powerful open source database. Since database products are usually something that cannot reasonably be changed within one software system for decades this is a good thing, because we never know what the big companies want to do in such a long time scale. If the migration to another DB product is easy, then the software does not really make use of the power and the features of the DB. And it will not be easy anyway.

There are a lot of cases where the combination of MS-SQL-Server with Linux will make a lot of sense. Since there are software systems that make use of this DB product, it gives the flexibility to run the DB on Linux servers. And maybe avoid an expensive migration to another DB product. As I already said it gives one more choice. In development environments where MS-products are commonly used, it gives one more combination. And eventually it will encourage Oracle and IBM a little bit to refrain from excessive price increases.

Share Button