Add documentation

AUTHORS

@@ -1,4 +1,4 @@
-# This is the official list of K8PSH authors for licensing purposes.
+# This is the official list of k8psh authors for licensing purposes.
 # If you agree to license any contributions in accordance with the license,
 # you may add your name and email here (as specified in the commit log)
 # in sorted order as part of your commit.

LICENSE.md

@@ -11,7 +11,7 @@ are not considered public domain are governed by the license below.
 
 # MIT License
 
-Copyright (c) 2021 The K8PSH Authors
+Copyright (c) 2021 The k8psh authors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -1,2 +1,85 @@
-# Kubernetes Pod Shell (K8PSH)
-K8PSH (/kə-ˈpēsh/) is a simple shell that allows one container to execute a process inside another container in the same Kubernetes Pod.
+# Kubernetes Pod Shell (k8psh)
+k8psh (/kə-ˈpēsh/) is a minimal shell allowing a Pod container to run an executable inside another container.
+
+## Building
+The project can be built using cmake:
+```shell
+$ cmake -B build -DCMAKE_BUILD_TYPE=MinSizeRel -G Ninja .
+$ cmake --build build
+```
+
+For testing, use the "Debug" build type. Debugging output can be enabled by individual files (`K8PSH_DEBUG="Process, Main"`) or globally (`K8PSH_DEBUG=All`) by setting the environment variable `K8PSH_DEBUG`.
+```shell
+$ cmake -B build -DCMAKE_BUILD_TYPE=Debug -G Ninja .
+$ cmake --build build --target check -v
+```
+
+The project is compiled into a statically-linked binary. This allows better portability between different operating systems and containers by eliminating dependencies on system libraries.
+
+## Running
+The project is divided into two components: a client, `k8psh`, and a server, `k8pshd`. Both components are contained in the same binary. `k8pshd` is typically hardlinked or symlinked to `k8psh`, and the name of the executable is used to determine whether to run in client or server mode. Running `k8pshd` is equivalent to running `k8psh --server`.
+
+In practice, the client is only used with the `--install` or `--server` options. It is invoked implicitly by any stub executables, but this is not visible to the end user. It is also used for interactive testing with a running server, since the configuration file and client command can be changed on the command line.
+
+A configuration file is used to configure the clients and servers. (See the example configuration file for more details.) It can be set on the command line with the `--config` option or using the environment variable `K8PSH_CONFIG`. Specifying the option will override any value in the environment variable.
+
+## Installing
+The project can be installed in multiple ways:
+ 1. Using the cmake `install` target.
+ 2. Copying the binaries into a directory in `$PATH`.
+ 3. Runing `k8psh --install [path/to/file]`.
+
+The preferred way to install into a container is using the `--install` option. Configuring the k8psh container as a Pod init container allows `k8psh` to be installed into a shared volume before other containers run. (An alternative is to already have `k8psh` installed in a shared persistent volume.) Other containers then run the executable in the shared volume. An example is given in `test/k8s/test.yaml`. The relevant parts are shown below:
+```yaml
+spec:
+  initContainers:
+  - name: k8psh
+    image: 76eddge/k8psh
+    args: [ --install, /k8psh/bin/k8pshd ]
+    volumeMounts:
+    - mountPath: /k8psh/bin
+      name: k8psh-bin
+  containers:
+  - name: server
+    image: centos:6
+    command: [ /bin/sh, -c, '/k8psh/bin/k8pshd --config=/workspace/test/config/k8s.conf --name=server' ]
+    volumeMounts:
+    - mountPath: /k8psh/bin
+      name: k8psh-bin
+      readOnly: true
+  - name: client
+    image: ubuntu
+    tty: true
+    command: [ /bin/sh, -c, 'k8pshd --executable-directory=/k8psh && g++ ...' ] # Runs g++ on the centos:6 container
+    env:
+    - name: K8PSH_CONFIG
+      value: /workspace/test/config/k8s.conf
+    - name: PATH
+      value: '/k8psh:/bin:/usr/bin:/k8psh/bin'
+    volumeMounts:
+    - mountPath: /k8psh/bin
+      name: k8psh-bin
+      readOnly: true
+  volumes:
+  - name: k8psh-bin
+    emptyDir:
+      medium: Memory
+```
+
+The ideal way to install `k8psh` would be to use the k8psh container as a data-only container and mount it as a shared volume. However, this is not possible at the present time due to limitations with Kubernetes.
+
+## What does it solve?
+One of the goals of the project is to ease docker image maintenance on highly matrixed workloads. For example, a Java/C++/JNI project has 5 different build environments/images: Alpine, Ubuntu - GCC 5.0+, Ubuntu - GCC 4.X, ARM v7, ARM v8. The project requires gradle, gcc, JDK 8, cmake, and ninja to build.
+
+The build tools could be installed on each individual docker image. However, this can result in different versions of each tool on the different images and can result in a lot of maintenance if tools need to be regularly updated for each image. Image scanning may also be needed on any updated images which may require justifications for using older tools or dependencies that contain vulnerabilities. Additionally, some docker images may come from an upstream source and cannot easily be modified (or may need to be modified every time a new version is received).
+
+Using k8psh, all the existing images can be used as-is, without any changes or updates. An additional image (for example `ubuntu:latest`) is added containing all the common build tools. (In this case gradle, JDK 8, cmake, and ninja.) The k8psh server is configured to run on each container exposing the GCC family of executables. These commands can then be run from the build tools image simply by running `alpine-gcc`, `arm-linux-gnueabihf-gcc`, or even just `gcc`.
+
+The builds now all use the same versions of the build tools, and only one image (the build tools image) needs to be maintained when newer versions of build tools are released. The other images can either be frozen or stripped down into bare or distroless images with only executables and dependent libraries. The k8psh server has no dependencies and can even run on scratch images.
+
+## How it works
+One or more k8psh servers are started on different containers in a Pod, each exposing different executables for other containers to run. Each server's exposed executables are listed in a configuration file. When `k8pshd` starts, it generates a set of stub executables (see `--executable-directory`) that can be called just like any other executable. If the server determines that it must expose executables, it will listen for requests from other containers.
+
+When one of the stub executables is called on a client, `k8psh` connects to the appropriate server (as found in the configuration file), requests to start the command on the server, and then transfers data (stdin, stdout, stderr, exit code) as the executable runs. All communication is done via a TCP socket. On the client container, running the stub will appear no different than running the actual program on the client itself.
+
+Multiple servers can be configured, each with different exposed executables. Every server listens on a different port that is determined by the configuration file. The configuration file is typically shared by all containers in a pod.

src/Main.cxx

@@ -109,7 +109,7 @@ static void mainClient(int argc, const char *argv[])
 {
 	std::string commandName = k8psh::Utilities::getExecutableBasename(argv[0]);
 	k8psh::OptionalString config;
-	std::string copyFilename;
+	std::string installFilename;
 	std::size_t i = 1;
 
 	// Parse command line arguments
@@ -122,26 +122,44 @@ static void mainClient(int argc, const char *argv[])
 
 			if (parseOption(arg, "-c", "--config", "[config]", i, argc, argv, config))
 				config.exists();
-			else if (parseOption(arg, "", "--copy-to", "[file]", i, argc, argv, copyFilename))
+			else if (arg == "-h" || arg == "--help")
+			{
+				std::cout << "Usage: " << clientName << " [-s | --server] [options] command..." << std::endl;
+				std::cout << "  Executes a " << clientName << " client command" << std::endl;
+				std::cout << std::endl;
+				std::cout << "Options:" << std::endl;
+				std::cout << "  -c, --config [file]" << std::endl;
+				std::cout << "      The configuration file loaded by " << clientName << ". Defaults to $" << environmentPrefix << "CONFIG." << std::endl;
+				std::cout << "  -h, --help" << std::endl;
+				std::cout << "      Displays usage and exits." << std::endl;
+				std::cout << "  --install [file]" << std::endl;
+				std::cout << "      Installs " << clientName << " to the specified file, overwriting any existing file, and then exits." << std::endl;
+				std::cout << "  -s, --server" << std::endl;
+				std::cout << "      Runs the server, with all options passed to the server." << std::endl;
+				std::cout << "  -v, --version" << std::endl;
+				std::cout << "      Prints the version and exits." << std::endl;
+				std::exit(0);
+			}
+			else if (parseOption(arg, "", "--install", "[file]", i, argc, argv, installFilename))
 			{
 				std::string clientCommand = k8psh::Utilities::getExecutablePath();
 
 #ifdef _WIN32
-				if (CreateHardLinkA(copyFilename.c_str(), clientCommand.c_str(), NULL) == 0 && CopyFileA(clientCommand.c_str(), copyFilename.c_str(), FALSE) == 0)
-					LOG_ERROR << "Failed to copy " << clientCommand << " to " << copyFilename << ": " << GetLastError();
+				if (CreateHardLinkA(installFilename.c_str(), clientCommand.c_str(), NULL) == 0 && CopyFileA(clientCommand.c_str(), installFilename.c_str(), FALSE) == 0)
+					LOG_ERROR << "Failed to install " << clientCommand << " to " << installFilename << ": " << GetLastError();
 #else
-				if (link(clientCommand.c_str(), copyFilename.c_str()) != 0)
+				if (link(clientCommand.c_str(), installFilename.c_str()) != 0)
 				{
 					char buffer[8192];
 					int source = open(clientCommand.c_str(), O_RDONLY, 0);
 
 					if (source < 0)
 						LOG_ERROR << "Failed to open " << clientCommand << ": " << errno;
 
-					int dest = open(copyFilename.c_str(), O_WRONLY | O_CREAT, 0755);
+					int dest = open(installFilename.c_str(), O_WRONLY | O_CREAT, 0755);
 
 					if (dest < 0)
-						LOG_ERROR << "Failed to open " << copyFilename << ": " << errno;
+						LOG_ERROR << "Failed to open " << installFilename << ": " << errno;
 
 					for (;;)
 					{
@@ -156,15 +174,15 @@ static void mainClient(int argc, const char *argv[])
 								if (wrote == size)
 									break;
 								else if (wrote >= 0)
-									LOG_ERROR << "Failed to write data to " << copyFilename << " while copying data from " << clientCommand;
+									LOG_ERROR << "Failed to write data to " << installFilename << " while installing from " << clientCommand;
 								else if (errno != EINTR)
-									LOG_ERROR << "Failed to write data to " << copyFilename << " while copying data from " << clientCommand << ": " << errno;
+									LOG_ERROR << "Failed to write data to " << installFilename << " while installing from " << clientCommand << ": " << errno;
 							}
 						}
 						else if (size == 0)
 							break;
 						else if (errno != EINTR)
-							LOG_ERROR << "Failed to read data from " << clientCommand << " to be copied into " << copyFilename << ": " << errno;
+							LOG_ERROR << "Failed to read data from " << clientCommand << " to be installed into " << installFilename << ": " << errno;
 					}
 
 					(void)close(source);
@@ -174,24 +192,6 @@ static void mainClient(int argc, const char *argv[])
 
 				std::exit(0);
 			}
-			else if (arg == "-h" || arg == "--help")
-			{
-				std::cout << "Usage: " << clientName << " [-s | --server] [options] command..." << std::endl;
-				std::cout << "  Executes a " << clientName << " client command" << std::endl;
-				std::cout << std::endl;
-				std::cout << "Options:" << std::endl;
-				std::cout << "  -c, --config [file]" << std::endl;
-				std::cout << "      The configuration file loaded by " << clientName << ". Defaults to $" << environmentPrefix << "CONFIG." << std::endl;
-				std::cout << "  --copy-to [file]" << std::endl;
-				std::cout << "      Copies this binary to the specified file, overwriting existing files, and then exits." << std::endl;
-				std::cout << "  -h, --help" << std::endl;
-				std::cout << "      Displays usage and exits." << std::endl;
-				std::cout << "  -s, --server" << std::endl;
-				std::cout << "      Runs the server, with all options passed to the server." << std::endl;
-				std::cout << "  -v, --version" << std::endl;
-				std::cout << "      Prints the version and exits." << std::endl;
-				std::exit(0);
-			}
 			else if (arg == "-s" || arg == "--server")
 				mainServer(argc, argv);
 			else if (arg == "-v" || arg == "--version")

src/Socket.hxx

@@ -30,15 +30,13 @@ public:
 #ifdef _WIN32
 	typedef SOCKET Handle;
 	typedef HANDLE Event;
-
-	static constexpr Handle INVALID_HANDLE = INVALID_SOCKET;
 #else
 	typedef int Handle;
 	typedef int Event;
+#endif
 
 	static constexpr Handle INVALID_HANDLE = Handle(-1);
-#endif
-	static const unsigned short RANDOM_PORT = 0;
+	static constexpr unsigned short RANDOM_PORT = 0;
 
 private:
 	Handle _handle;

test/k8s/test.yaml

@@ -10,7 +10,7 @@ spec:
       - name: k8psh
         image: 76eddge/k8psh
         imagePullPolicy: Never
-        args: [ --copy-to, /k8psh/bin/k8pshd ]
+        args: [ --install, /k8psh/bin/k8pshd ]
         env:
         - name: K8PSH_DEBUG # To debug, modify the Dockerfile to have cmake create a debug build
           value: 'Process, Main, Configuration'