Advanced Management

The goal of this section is to given more in-depth tips about how to use Corus more efficiently.

Process Dependencies

In the world of distributed applications, interdependencies are common. Let's say application A depends on application B, and that both of these applications run into their own processes. What if at the startup of A, B is not available ? In an ideal world, application A would implement a provision with regards to that issue (using JINI-like dynamic discovery of services, for example). But it's not always possible, either for lack of resources, time, development expertise, or simply because of legacy code that does not support dynamic discovery in the first place.

As a workaround, Corus has built-in support for handling process dependencies. The mechanism is as follows:

  1. In the Corus descriptor, declare which processes depend on which other processes.
  2. At the start up of a given process, Corus checks on which other processes it depends. If there are indeed such dependencies, the corresponding processes are started first - that logic is followed recursively.
The above means that if your start a dependent process, you do not need to start the ones on which that process depends, since Corus does it automatically.

Let's see a concrete example: building on a previous tutorial, we're introducing a Jetty container to which we plug a custom SessionManager implementation that depends on a remote session cache to provide distributed session management.

More precisely, the distributed session caches (built on EHCache) are deployed in their own processes, an replicate session data to one another in order to provide failover. This way, if a Jetty server crashes, requests can be load-balanced to another server instance, which will retrieve the corresponding session data from one of the distributed caches. In the same manner, since the caches are redundant, session data can be retrieved from an alternate node in case of failure at one instance.

The diagram below illustrates the architecture described above:

The implementation is somewhat naive, but that's not the point. The goal is only to demonstrate a concrete scenario where process dependencies make sense.

It is clear from the above that the Jetty servers depend on the session caches. This is materialized in the Corus descriptor:

<distribution name="scalable-sample-jetty" version="1.0" xmlns="http://www.sapia-oss.org/xsd/corus/distribution-5.0.xsd">

  <process  name="server" 
            maxKillRetry="3" 
            shutdownTimeout="30000" 
            invoke="true">
    <port name="jetty-server" />        
    <java mainClass="org.sapia.corus.sample.jetty.ScalableJettyServer"
          profile="dev" vmType="server">
      <xoption  name="ms" value="16M" />
      <dependency distribution="scalable-sample-jetty" process="sessions" />
    </java>
    <java mainClass="org.sapia.corus.sample.jetty.ScalableJettyServer"
          profile="prod" vmType="server">
      <xoption  name="ms" value="128M" />
      <dependency distribution="scalable-sample-jetty" version="1.0" process="sessions" />
    </java>
  </process> 
  
  <process  name="sessions" 
            maxKillRetry="3" 
            shutdownTimeout="30000" 
            invoke="true">
    <java mainClass="org.sapia.corus.sample.jetty.session.SessionServer"
          profile="dev" vmType="server">
      <xoption  name="ms" value="16M" />
    </java>
    <java mainClass="org.sapia.corus.sample.jetty.session.SessionServer"
          profile="prod" vmType="server">
      <xoption  name="ms" value="128M" />
    </java>
  </process> 
</distribution>

As you can see, we're declaring a dependency element for the ScalableJettyServer application - the element has been introduced at this level to allow changing dependencies on a per-profile basis.

The dependency element takes the following attributes:

  • distribution (optional): the distribution of the process on which the application depends. Defaults to the same distribution as the application.
  • version (optional): the distribution version of the process on which the application depends. Defaults to the same version as the application
  • process (mandatory): the name of the process on which the application depends.

We're seeing in the descriptor that the Jetty server has a dependency on the "sessions" process, which incidentally corresponds to our SessionServer application. Let's try it out. Start a Corus instance, and (using the CLI) deploy the distribution:

deploy target/*-scalable.zip

And now start the Jetty server:

exec -d * -v * -n server -p dev

You can look at the terminal in which Corus runs: you should see the output from the Corus server indicating that it indeed starts the process corresponding to the session server first. The CLI's output also indicates the startup order:

Scheduling execution of process: [dist=scalable-sample-jetty, version=1.0, process=sessions]
Scheduling execution of process: [dist=scalable-sample-jetty, version=1.0, process=server]

There's a delay between startups (which can be configured). After a certain amount of time, you should see that all processes are running by typing ps.

Now kill all processes (kill -d * -v * -n *) and jump to the next item in this tutorial.

Execution Configurations

Let's say that even using dependencies, you end up with multiple completely independent processes that you have to start manually. You could declare fake dependencies amongst independent processes, in order to be able to start only the first one in the chain. But that would not be ideal.

There's a more elegant solution to this problem: execution configurations. An execution configuration is just an XML file containing predefined markup that indicates to Corus which processes should be started. Such a configuration has a unique name within a Corus instance, and you use the exec command in conjunction with the name of an execution configuration to trigger the startup of its related processes.

Building on the previous example, we've defined the following execution configuration (see the exec.xml file under the corus_sample_jetty module):

<exec name="jetty-server" startOnBoot="true" xmlns="http://www.sapia-oss.org/xsd/corus/exec-3.0.xsd">
  <process dist="scalable-sample-jetty" version="1.0" name="server" profile="dev" />
</exec>

As you can see, the exec element is the root, and has two attributes:

  • name: indicates the name of the configuration
  • startOnBoot: indicates if the processes corresponding to each process element in the configuration should be started upon startup of the Corus instance itself.
Using startOnBoot is a safeguard against the processes becoming unavailable after a machine reboot.

Nested within the exec element are potentially many process elements, each referring to a process that should be started.

Let's see how the whole thing works: deploy the execution configuration into Corus:

deploy etc/exec.xml

To view the currently deployed execution configurations, type the following:

ls -e

Make sure all currently running processes are killed ( kill -d * -v * ) and type the following:

exec -e jetty-server

Check the Corus console: you should see in the logging output what is happening: the process that's indicated is eventually started (the dependency startup chain is respected even in that case). After a certain amount of time, all processes should be up (do a ps to confirm it).

Now we'll test the "start on boot" feature: first kill all the processes (kill -d * -v * -w). After their termination, shutdown Corus, and start it again.

You will now see that the processes will be automatically restarted, because of the startOnBoot flag: restart Corus and watch the output. After a while (it takes about 45 seconds), you will see that the processes are gradually restarted.

In order to prepare for the last item in this tutorial, leave everything as is, for we're going to demonstrate the ultimate productivity feature: Corus scripts.

Scripts

You now should have the ScalableJettyServer distribution running in Corus - since we've not undeployed it. Let's imagine now that we want to do a full redeploy of the distribution (because we've fixed a bug, for example).

In theory we have to kill the currently running processes, undeploy the distribution, redeploy the new one... When you have multiple environments to maintain (Dev, QA, Beta, Prod...), these steps have to be performed repetitively, and can consume quite a lot of your time.

To work around this hassle, Corus provides a script command. The command takes a file that itself contains the list of commands that you want to perform, as if you'd time them sequentially in the CLI. In our case, here's the script we're going to use (you can find it under the etc directory, in the script.txt file):

kill -d * -v * -w
undeploy -d * -v *
deploy target/*-scalable.zip
deploy etc/exec.xml
exec -e jetty-server

To launch the script, type the following in the CLI:

script etc/script.txt

Watch the output in the CLI: all commands are processed one by one, in a single batch, with only one interaction on your end.

Note that the commands in script files can also be clustered using the -cluster switch, as usual. Imagine what this means: being able to execute multiple commands in one shot across a whole domain.

In addition, the CLI supports executing a script right away, without entering prompt mode, using the -s switch:

corucsli -s etc/script.txt

In this case the CLI executes the script and then exits immediately.

Stored Properties

Corus allows storing properties, which are passed to processes through -D options (thus, as JVM or system properties).

To store process properties in Corus, you use the conf command. Here's an example:

conf -p myapp.xms=32M

You can pass multiple properties as such:

conf -p myapp.xms=32M;myapp.env.type=dev

If you have many properties, it becomes cumbersome to specify them all at the command-line. As a convenience, the command allows specifying the path to a Java properties file, whose properties will be loaded and sent to Corus:

conf -p etc/conf/myprops.properties

The command will check the value of the -p option and interpret it as specifying a properties file if it detects a .properties extension.

You can also list properties and delete them (wildcard matching is supported for deletion):

conf ls
conf del -p myapp.*

Of course, the command also supports the -cluster option, allowing you to manage the storage of properties across a whole cluster.

It is not recommended to store all your properties in Corus: the feature has not been developed with that practice in mind. Rather, you should store in Corus the properties that are environment-specific and are used in the context of your application's startup, for determining certain states. For example, you could store a property identifying the environment itself (dev, QA, UAT, prod, etc.) and load your application configuration based on that single environment-determining property.

HTTP Extensions

Corus makes available bits of functionality "under" HTTP. These are grouped by categories that have been named "extensions". These extensions allow accessing information using a web browser (rather than using the CLI). Typically, extensions make data available through XML.

Note that this functionality precedes Corus's REST API, which should be considered better suited for the purposes described in this section.

Too see the list of available extensions, just type a URL of the form: http://<corus_host>:<corus_port>/. To try it out, start the corus server under the "samples" domain (as you've been doing in the context of the tutorials):

corus -d samples

And now, open your web browser to the following URL: http://localhost:33000/. This URL corresponds to the Corus server home page, where you can see the list of available extensions.

To view the list of available distributions, type the following: http://localhost:33000/deployer/ls. The above URL corresponds to the "deployer" extension. Each extension in Corus is published under the Corus server root URL. The deployer extension corresponds to the server's internal Deployer module, which handles the distributions. It's the same module that's called when using the ls and deploy/undeploy command in the CLI.

In fact, as you can see, the last segment in the URL is indeed ls. The whole URL therefore maps one-to-one to the CLI's ls command. It also takes the same options (d, v, cluster...). For security reasons, only the Deployer's ls command has been made available as an HTTP extension.

Command options are passed with HTTP parameters. Thus, the following would be valid: http://localhost:33000/deployer/ls?d=*&v=*.

For its part, the "processor" extension provides information about currently running processes, making the following commands available through HTTP: ps, status.

One neat thing about the processor extension is that its output can be integrated as part of a monitoring infrastructure. Indeed, through the status command, application data can be recuperated and analyzed. For example, open the following in your browser: http://localhost:33000/processor/status?cluster=true.

Conclusion

In this tutorial, you've seen productivity features that, when used in conjunction, allow performing application deployments in just a few steps. The features greatly help making deployments a breeze and minimize downtime. You've also seen Corus' so-called "HTTP extensions", which are typically intended for publishing runtime status data using XML. That data can conveniently be integrated as part of a monitoring infrastructure, for example.