Blog posts in this series

  • Evaluation of our existing automation tool
  • Evaluation of new automation tools (planned)
  • Inventory migration (planned)
  • Rex Module migration (planned)
  • Rex Wrapper (planned)
  • Testing (planned)
  • Bring it into production (planned)
  • Experiences (planned)

Ansible Migration: Evaluation of our existing automation tool

In Adjust we are (or maybe have been, by the time you are reading this blog posting) using Rex as an automation tool. It served us well, and we grew the company and the infrastructure based on it. But over time we discovered that the more complex the infrastructure becomes, the more complex the solutions - and workarounds - in Rex become.

Rex is defining infrastructure as code written in Perl, a programming language which is notorious as write-only language, and the usage is declining among developers. While quite a few people in the company understand Perl, it is hard to find new people who can maintain the code base, or can dive into the complexity. Rex does not provide idempotence, which makes writing large and complex tasks a challenge. And increasingly we found ourselves just running single tasks instead of whole operations and deployments on a server, both for speed and for the uncertainty that all previously defined tasks will work. Also deploying a new server using Rex occasionally fails, and small changes in the code base are required.

Nevertheless before deciding on such a big step like changing the automation tool for everyone, we decided to evaluate if Rex is serving us well. For that, we came up with a set of questions we want to answer for Rex, and also use the same questions as template for future research should we decide that a change is necessary.

Must Haves

A number of key features an automation tool must provide. We knew that at this point Rex might not provide all of them, but we did not know how many Yes or No we would end up with. In [square brackets] the answers we found for Rex, a future blog post will evaluate other tools.

  • Is the tool supported / being actively developed [No, the latest updates for Rex are quite old, and only one person seems to work on the code]
  • Is there an active community? Things like meet-ups, help in forums and change in code base? [No, nothing to be found, and the mailing list only has a few emails per year]
  • Is the project > 3 years old? [Yes]
  • Is the project used by other companies? [Yes]
  • Is the project written in a language we can and want support? [Yes, although it is written in Perl - few people in the company can do Perl and even fewer people like to hack Perl]
  • Does the project support all used operating systems at Adjust? Gentoo, Ubuntu, Mac as controller ect? [Yes, but it’s complicated to get it running]
  • Does it use Push or Pull? [Push]
  • Does it require setting up servers right from the beginning, or can it be rolled out gradually? [Can be rolled out gradually]
  • Can credentials be abstracted from regular users? [No]
  • Client on target system required? [No]
  • Can partial steps be run by a user? [Yes]
  • Is the result idempotent? [No]
  • Does it support multiple cloud environments? [No]
  • Can it re-run failed deployments? [Not guaranteed]
  • Is there a cost associated, and if yes what is the cost structure? [No]
  • Can it manage infrastructure (servers, switches, ect), operating systems (packages, network configuration, users ect) and applications (deploy new websites, upload builds, manage service * configurations, ect)? [Theoretically, if someone writes Perl code]
  • Requires an agent? [No]
  • SSH Based? [Yes]
  • Is it centralized, or distributed? [No]
  • Does it require another tool to make it autonomous? [Not possible]
  • Declarative manifests? [No]
  • Does it keep the state or is it stateless? [Stateless]
  • In case of down times/disaster happening to the tool, how long to fix/ temporary solution for deployment? [Anyone can run it]
  • How many of the team are already experienced with the tooling? [No Rex experts, only Rex survivors]

Nice to Haves

Which features are nice to have, both for Rex and for another tool we might need to evaluate.

  • Is the project backed by a company and can we get enterprise support? [No]
  • Does the project have a UI? [No]
  • Is there a known migration path from Rex to the project? [Yes, obviously]
  • How many points at Stack Overflow? [Popularity contest - around 0.1% in 2022]
  • Does it integrate well with known CI systems? [No]
  • How many job postings for this software? (Not answered)

Conclusion

After we looked at the result, and found more No than Yes answers, it became apparent that our initial thoughts are confirmed by hard evidence. Rex served us well, but the company outgrew the options and possibilities it provides.

It’s time to find something new. Stay tuned.