This section introduces Corus basics.
What is Corus?
What is Corus?
Corus is an infrastructure that allows centrally controlling application processes in distributed environments. In a short list, here are Corus' features:
- Centralized, remote, replicated application deployment/undeployment across multiple hosts.
- Centralized, remote, replicated execution/termination of application processes across multiple hosts.
- Monitoring of running processes to detect crashes (and restart crashed processes automatically).
- A REST API (http://www.sapia-oss.org/projects/corus/rest.html) allowing for fine-grained control from automation frameworks and scripts.
- Repository functionality: automatic synchronization between Corus nodes so that newly appearing nodes become exact copies of existing ones.
- Possibility for processes to publish status information at the application level, through the Corus infrastructure. Status information can then be monitored centrally.
- Possibility for processes to trigger their own restart.
- Replicated, JNDI-compliant naming service: based on Sapia's Ubik distributed computing framework (guaranteeing fail-over and load-balancing), the naming service can be used by deployed Java applications.
- Container support (seamless integration with Docker), allowing distributed control of non-Java apps.
- Allows centralized distribution of «standard» Java applications (Java classes with a «main» method), and remote control of corresponding JVMs.
- Allows multiple Java applications per-JVM, through the use of Magnet (http://www.sapia-oss.org/projects/magnet).
- Support for gracefull deployment across a cluster of machines (allows for avoiding service degradation when performing application upgrades or performing server restarts).
- Free and open source with mixed Apache 2.0 and GPL v3 licenses – see the Licensing section (next) for more details.
This section gives a high-level view of the Corus architecture.
From the user's point of view, Corus comes as three components:
- A lightweight Java server that is installed on a host in order to allow remote deployment and remote execution of processes on that host.
- A Java command-line interface (CLI) that gives centralized control over multiple Corus servers (note that each Corus server also publishes REST APIs that provide functionality equivalent to the CLI).
- A Java monitoring tool that helps users/administrators see what is happening in a remote Corus server at runtime.
A Corus server is typically installed on a given host (such hosts are identified in the diagram above by the larger green circles with the “Corus" label in them). Corus servers are grouped by domains (or clusters). A cluster of Corus servers can be centrally managed through the command-line interface (illustrated by the figure with the “CLI" label, in the diagram below). Each Corus server monitors/controls processes on the host on which it is installed.
Corus servers discover each other through a broadcast mechanism (either IP multicast, or relying on a TCP-based multicast mechanism - as explained in the Discovery section). The following describes how Corus servers are grouped into domains:
- At startup, Corus servers are given a domain name through the command-line, or through their configuration file.
- When a new Corus server appears in the network, it broadcasts an event stating the domain it is part of.
- Existing Corus servers will trap the event. If their domain name corresponds to the domain name that is part of the received event, they then in turn broadcast an event stating their existence.
The newly started Corus server will in turn trap the events coming from existing servers. It will add these servers (or, rather, their corresponding addresses) in its internal sibling list. A typical Corus topology is illustrated below:
As explained throughout the documentation, Corus is all about controlling processes in a distributed manner.
The Command-Line Interface
Corus offers a powerful command-line interface (named "CLI"). Although it a pure devops way, it is preferrable to use the API, explaining the CLI and showing some examples of commands provides the best illustration.
The command-line interface is used to:
- Deploy application distributions into Corus servers that are part of a given domain - when targeted at more than one Corus server, a deployment is replicated (the distribution is simultaneously deployed to all Corus servers).
- Start processes from the given distributions. Process execution can be replicated also: processes corresponding to given deployed distributions can be started on multiple hosts simultaneously.
- Undeploy applications from a single Corus server, or from multiple Corus servers in a domain.
- Stop running processes at a single Corus server, or at multiple Corus servers in a domain.
- Obtain status information from processes in a domain.
- View what distributions have been deployed.
- View what processes are running.
- Deploy shell scripts and execute them remotely, on the Corus nodes.
To ease its use, the CLI emulates Unix commands (ls lists deployed distributions, ps and kill respectively list and stop running processes, etc.).
The CLI in addition supports pattern matching operations. The following examples illustrate typical use cases:
|deploy dist/*.zip||Deploys all distribution archives ending with the .zip extension, under the dist directory.|
|exec -d myapp -v 1.* -n echoServer -p test||Starts the echoServer process corresponding to all 1.xx versions of the myapp distribution under the test profile|
|exec -d * -v * -n * -p test -cluster||Starts the processes corresponding to all versions of all distributions under the test profile, on all Corus hosts in the cluster.|
|kill -d myapp -v 1.* -n echoServer -p test||Kill all echoServer processes corresponding to all 1.xx versions of the myapp distribution under the test profile.|
|kill -d myapp -v 1.* -n * -p test||Kills all processes corresponding to all 1.xx versions of the myapp distribution under the test profile.|
|kill -d * -v * -n * -p test -cluster||Kills all processes under the test profile, on all Corus hosts in the cluster.|
|kill all -p test -cluster||A shortcut that amounts to the same thing as the above.|
Help on the available commands can be obtained through the man command in the CLI. Typing man will display all available commands; typing man <command_name> will display help on the specified command.
Process execution occurs upon request of Corus administrators. Concretely: the exec command is typed at the CLI (with required command-line arguments), and a corresponding command object is sent to the Corus server to which the user is connected. If performed in clustered mode (if the -cluster switch has been entered at the command-line), the command is replicated to other Corus servers.
Upon receiving the command, the Corus server will start the specified process. From then on, processes executed by Corus servers are required to poll their respective Corus server, according to the protocol explained by the Corus Interoperability Specification .
All communications between Corus-controlled processes and their server indeed follow that specification, which details a SOAP/HTTP-based protocol (where processes are in fact clients). The following diagram illustrates a Corus server and the processes it controls:
The interoperability protocol specifies a series of commands, corresponding to the life-cycle of processes started by a Corus server. That life-cycle can be summed up as follows:
- Processes are started following an exec command.
- After startup, processes poll their Corus server at a predefined interval to signal that they are up and running. At a predefined interval also, processes send status information to their server (application-code within the process can take part in status generation).
- When polling, processes eventually get back their list of pending events (the server keeps an object representation of each process, and within these objects, a queue of pending commands targeted at the «real» process).
- Processes are eventually killed using the kill command (not the one of the OS, but the one emulated by the CLI). The object corresponding to the kill command is sent to the desired Corus servers. Internally, the servers introspect the kill command to figure out what processes must be stopped; the objects representing the processes that must be killed have a «kill» event queued up in them. At the next poll, that signal is «recuperated» (sent as part of a SOAP response) to polling processes.
- Processes that are shutting down following a «kill» event must confirm their shutdown to their Corus server. As a matter of precaution, processes that fail doing so are eventually killed using an «OS» kill by their Corus server.
- Processes that do not poll their Corus server within the predefined interval are deemed crashed or unstable, and thus automatically killed (and optionally restarted, depending configuration information provided at deployment).
- The fact that interoperability between Corus servers and their corresponding processes is implemented using SOAP over HTTP means that even if Corus is implemented in Java, potentially any type of application processes can be controlled with it (C++, Python, Perl, etc.). As was mentioned, currently though, the only implementation of the Corus Interoperability Specification is in Java.
We advise that you look at the wiki for the latest instructions. Nevertheless, here is a summary:
- Download the latest Corus distribution from Mediafire. The distribution comes as a .gzip or .zip file, depending on your target platform.
- On the host where Corus will be installed, create a directory where the distribution will be “unzipped". Make sure that the full path to the directory you have chose does not contain spaces.
- On Unix/Linux, create a user (by convention, it is named corus) that will own the above-created directory.
- Create the $CORUS_HOME environment variable on the Corus host; the variable should correspond to the full path of the directory that has been created in the previous step (under Linux/Unix, you can also define that variable in a file that is loaded when users log in – such as in /etc/profile on some Linux distributions).
- Add the bin directory under $CORUS_HOME to your PATH environment variable. That's it. You can start corus by typing corus in a terminal.
Installing as a Service
The Corus wiki contains instructions describing how to install Corus as a service on your target platform. See the documentation on the old Google Code wiki.
Also, you may look at the Corus Configuration section for more information about how to configure Corus. The corus.properties file under Corus' config directory also contains instructive comments.
All of the links below appear on the Corus wiki, and are provide as shortcuts to relevant sections of the wiki:
If your Linux flavor does not appear below, you may base yourself on a distribution that supports Corus as a service through init.d (such as Fedora).
We're supporting Mac through launchd. See: http://code.google.com/p/sapia/wiki/CorusInstallationProcedure_SnowLeopard .
Corus Network Address
As mentioned in other sections of this document, Corus is built upon Sapia's Ubik distributed computing framework. A special Ubik property may be used to specify on which network address Corus should listen, That address will also be returned as part of the remote stubs that Corus generates in the course of communicating with clients (which is the case in the context of JNDI lookups for example, or when using the CLI). The property to set (in the corus.properties file) is as follows:
The value of the property is expected to be a regexp that will be matched against the network interfaces that Corus detects. Note the double-backslashes; the single-backslash is an escape character in Java, but in this case we want it to be interpreted literally so that it is processed by the regexp evaluation.
Testing the Server
You are ready to start a Corus server. To test your installation, go to the command-line of your OS and type:
Typing the above with no arguments should display the Corus server help. Read it for further information. Typically, to start a Corus server, you provide the name of the domain “under" which the server should start, and the port on which it should listen (by default, the port is 33000, and the domain is simply “default"). Hence:
corus -d mercury -p 33100
Will start the server in the foreground under the “mercury" domain, on port 33100.
Testing the Client
To connect to the above server using the command-line interface, type:
coruscli -h localhost -p 33100
As you might have guessed, the arguments passed at the command-line specify the host of the server we wish to connect to, as well as the port of that server (for help, type the command without any arguments). As for the server, the -h and -p options are optional and will default to localhost and 33000 respectively.
Once you are in the CLI, you can see the list of available commands by typing:
Yes, you are right: just like Unix. You should see the list of available commands then. To read help about one of them, type:
And off you go: help about the specified command is displayed.
To exist the CLI, type: