In this blog I intend to introduce a new open source project that I have been working on recently. However, before I get into the detail I’d like to describe my motivations for this work. TLDR, gitHub.com/desdrury/k8sdc.
Over many years I’ve had lots of different and interesting roles in the IT industry. And although I’ve predominately been a systems administrator I’ve also had a number of roles as a developer. Combining both has always been a love of mine. Basically doing DevOps long before the term was invented and became fashionable.
Unfortunately over this period I’ve also had to suffer a great deal of frustration and disappointment as I try to deliver value to the people I work with. IT in general can be a difficult industry to work within. It is overly complex and burdened with silo-ed technical teams that do not collaborate well together. The thing that I enjoy most is helping people to achieve their goals in the simplest way possible. And this has been the case with the majority of people that I have had the pleasure to work with. However, IT organisations are generally structured in such a way that doing the simplest thing and working well together is often not possible. There are many reasons for this but I feel that one of the major ones is simply that IT systems are very complex. This leads to fragmented knowledge, bad solutions and cumbersome processes, which in turn leads to stress and finger pointing when things don’t work out. People in the IT industry often feel powerless to make a positive change and after attempting fruitlessly for the n’th time many simply give up or leave the profession.
However, I believe that recent technical advances are finally offering the possibility to tame this complexity. Open source software has been around for some time now and It has reached a point where there are multiple mature products that cover all aspects of an IT system. Often these products are better than equivalent vendor based solutions. Certainly they are cheaper to start using. And with the advent of configuration management tools and cloud infrastructures IT systems can now be built programmatically, allowing rapid and consistent provisioning. But the thing that I feel will have the biggest impact in reducing the complexity of IT systems is the emergence of products that can orchestrate container based workloads.
Recently a proliferation of container orchestration products have come to market. But, in my opinion, Kubernetes is the absolute best. Offering a very open, scaleable and performant architecture. With Kubernetes you declare the components of a dynamic system. Including how the components find each other, communicate with each other and respond to external events. Capabilities such as naming, service discovery and lifecycle management are automatically taken care of for you, and an administrator can easily manipulate or gain insight into the state of the system. Systems become much more organic in nature with Kubernetes. In concrete terms some of the benefits of Kubernetes are.
- Not tied into to running any particular technology stack or methodology, such as Java or 12-factor apps.
- Extremely well suited to running microservices architectures.
- Any number of environments, created in seconds and running with consistency. Including on developer workstations.
- Simplified and easy workflow to push changed components through environments. Allowing the release cadence to be increased.
- Independent from the base IaaS, bare metal or OS technology.
- Separation of concerns with Dev and Ops. Allowing developers much greater freedom to experiment and operations the ability to control managed environments.
- Intelligent schedulers can automatically modify the system based on any number of inputs.
- Applications can become far more resilient, scaleable and performant.
Another important product that I use is Ansible. Once again, in my opinion, I feel that Ansible is the best of all the configuration management tools, although it is so much more than just a CM tool. It is also an excellent orchestrator of actions over large sets of endpoints. Ansible has a simple YAML based DSL and is easily extended. With dynamic inventories being a particularly powerful feature. The fact that agents do not need to be installed on the endpoints makes Ansible very lightweight. And by using a PEX file to wrap Ansible and its dependancies it becomes trivial to deploy it wherever it is required. For these reasons I use Ansible to build the base Kubernetes infrastructure.
So, finally, I would like to introduce my project. It is called k8sdc, pronounced Kubernetes DC, aka Kubernetes Data Center, and it is a project to create an entire data centers worth of infrastructure sat atop Kubernetes. This includes all of the software needed for CI/CD, source control, code review, artefact storage, centralised logging and alerting, user registry, mail server, load balancing, blogs, wiki, productivity tools, etc.
The k8sdc platform will be cloud provider agnostic. It may make use of some cloud provider capabilities, such as external load balancers, but this will be as an alternative to core capabilities. The intention is to give small / medium businesses and non-profit organisations the ability to stand up an infrastructure that rivals the capabilities of an enterprise at little to no cost and within hours.
There will be simple user guides that describe each of the capabilities and how to use them in combination. This will allow the development and delivery of applications in a simple, rapid and robust manner, allowing small organisations to deliver digital services to market far more quickly and professionally.
Presently k8sdc does not do anything whiz-bang. Only laying down the foundations of a Kubernetes install. And although there are other ways to install Kubernetes I wanted to create a way that worked how I wanted. For one, k8sdc retrieves all products from GitHub or get.docker.com. This means that the latest versions can be used without waiting for them to become available in a particular Linux distribution. Secondly, a single node is used as a download server, which means that only this node needs access to the Internet, allowing for a more secure environment. This also improves the install performance of all the other nodes as they are downloading packages over a local network, as well as many other aspects that I included to tailor things to my liking.
k8sdc is a long term project and as such it will take some time to develop, especially if the quality is to be kept high. To see what I will be working on next you can check out the project roadmap, it details both my high priority items and those that will come afterwards. However, please take it only as rough guidance as things may change.
I believe that right now we are at the start of a revolution in the IT industry. We now have the ability to do everything using open source and in ways that are so much better and faster than has ever been the case before. In fact locking yourself into a proprietary vendor product or cloud offering is likely to reduce an organisations long term effectiveness and agility. Small organisations will be able to compete with enterprises on an even playing field in the technology space. In all my years of working in the IT industry right now is the most exciting. Although the number of operational staff is likely to be significantly reduced in the future I feel that those that remain will have a much more enjoyable and satisfying work life.
The views expressed in this blog are purely my own and do not reflect those of any employer I may be currently working for.