Home -> VM -> TP1.1.1 -> Admin Guide

Workspace Admin Guide for TP1.1.1 (deprecated)

1. Introduction

This guide contains configuration information for system administrators working with Workspaces. It provides references to information on procedures typically performed by system administrators, including installing, configuring, deploying, and testing the installation.

Important

This information is in addition to the basic Globus Toolkit prerequisite, overview, installation, security configuration instructions in the GT 4.0 System Administrator's Guide. Please read through this guide before continuing.

In the future we will provide detailed instructions for configuring GT4 with the bare necessities for the Workspace Services. The short version of these instructions is to install the Java GT4 core with common database tools:

make wsjava common globus_proxy_utils globus_simple_ca \
globus_simple_ca_setup globus_database_common postinstall

(if you have proxies and proxy tools from another GT[2-4] installation, you can leave out these middle targets: "common globus_proxy_utils globus_simple_ca globus_simple_ca_setup")

2. Building and Installing

Download the service here.

After expanding the tarball, navigate to the top level directory created which we will refer to as $WORKSPACE_HOME.

$WORKSPACE_HOME contains two directories, workspace_service contains the GT4 services and workspace_control contains the program that it is necessary to install on a physical node running the Xen hypervisor. In this version, these will be installed on the same node.

2.1. Installing the Workspace Service.

Currently, only the source is distributed. The root build file for the Workspace Services is $WORKSPACE_HOME/workspace_service/build.xml

This will install the client and service:

export GLOBUS_LOCATION=/path/to/GL
cd $WORKSPACE_HOME/workspace_service
ant deploy

Before testing the service, you need to configure it and install and configure the backend program.

2.2. Installing the backend program.

The program is implemented in Python, located under the $WORKSPACE_HOME/workspace_control directory. There are two ways to install the backend program: use the supplied setup.py script that will install the Python packages to the regular OS Python location or use a supplied wrapper script that sets up the PYTHONPATH environment variable to correctly point at the program's libraries.

For Xen3 and for one method of passing the workspace image configuration information, the sudo program must be installed on the same node as the backend program. See this section.

2.2.1. Option A: setup.py

The setup.py script is a regular Python distutils script that will install the workspace backend program worksp and its libraries to the regular Python program location on your system.

The distutils script inherits functionality besides "install," including RPM generation. For help with commands, there are two built-in help messages:

python setup.py --help
python setup.py --help-commands

We will use the install subcommand. Help with this command can be obtained by running:

python setup.py --help install

As root, run

python setup.py install

This will install the worksp program to a location on a regular PATH (typically /usr/bin), its libraries to a Python site-packages directory (typically under /usr/lib), and the worksp configuration file to /etc/worksp.conf.example

You should now be able to see this message when running worksp

arguments?
for help use --help

Copy /etc/worksp.conf.example to /etc/worksp.conf

After a successful installation, proceed to configure the backend program by editing /etc/worksp.conf

2.2.2. Option B: wrapper script

If you do not use the setup.py program, you need to configure a shell script to properly setup the PYTHONPATH to call the worksp program. This is like many of the scripts in $GLOBUS_LOCATION/bin that simply set up the Java classpath.

An example called workspace_control is installed in $GLOBUS_LOCATION/bin when the Workspace Service is installed. The script looks for an environment variable called WORKSPACE_HOME to allow it to easily point to the proper Python libraries.

The best approach if you are tied to option B is to simply edit the (very simple) shell script $GLOBUS_LOCATION/bin/workspace_control to point to an absolute path instead, to alleviate the need for this environment variable.

Important

If you do not edit this to use an absolute path as described above, a value for WORKSPACE_HOME would need to be set in the environment of the container running the Workspace Service.

You should now be able to see this message when running $GLOBUS_LOCATION/bin/workspace_control

arguments?
for help use --help

Copy $WORKSPACE_HOME/worksp.conf.example to /etc/worksp.conf

After a successful installation, proceed to configure the backend program by editing /etc/worksp.conf

3. Configuring

3.1. Workspace Service configuration

After installing the Workspace Service, files that alter its configuration are stored in the $GLOBUS_LOCATION/etc/workspace_service/ directory.

3.1.1. Factory grid-mapfile

To edit the list of authorized users of the factory service, add DNs to $GLOBUS_LOCATION/etc/workspace_service/workspace-grid-mapfile

Important

The factory service grid-mapfile overrides the container wide grid-mapfile (typically this is /etc/grid-security/grid-mapfile). The service is distributed with zero entries, which disables access to workspace creation entirely.

Add a DN to the list in normal Globus fashion (if the DN has spaces in it, use quotes around it). The gridmap authorization module requires a username mapping, but it is irrelevant to this service, so using e.g. "fakeuser" for each DN entry is OK.

Other authorization methods are now available (documentation will be updated shortly).

After a workspace is created, only the creator may manage the WSRF resource. This is configured programmatically for each workspace, so no grid-mapfile for the workspace service is needed.

3.1.2. Backend path

To edit the path to the workspace backend program that was installed, edit $GLOBUS_LOCATION/etc/workspace_service/jndi-config.xml

In jndi-config.xml, find the workspLocation entry under the WorkspaceService section. If the program you want will not be on the container environment's PATH (or you are using sudo), then use an absolute path to the program. The service is distributed with the proper configuration for when the backend program is installed with option A.

In the same section is the sudo configuration ("true" or "false" to prefix commands to the backend program with sudo), for more explanation on what it is for, see below

3.1.3. Factory policies

Currently, the factory can be configured with three simple policies concerning workspace deployment. Each value is listed in the WorkspaceFactoryService section of $GLOBUS_LOCATION/etc/workspace_service/jndi-config.xml

The defaultRunningTimeMinutes value is used if the DeploymentTime element of the deployment request is not specified.

The maxRunningTimeMinutes value can not be exceed by the value of the DeploymentTime element of the deployment request.

The WSRFResouceMinutesPastRunningTime value configures the default termination time of the WSRF resource representing the workspace. When a workspace is shutdown, its representation is not terminated (one reason being that it can be started again). This value is added on to the running time and is currently only configurable at this location.

The optional architecture value can be one of the JSDL ProcessorArchitectureEnumeration values such as "x86". If you are running Xen on other architectures, this should be changed appropriately.

The optional vmm value can be either Xen or VMware, but this will currently be Xen.

The optional vmmVersions value can be a comma separated list of one-word strings (deciding on values to use is a deployment issue within VOs).

The optional associationFileLocation value is a directory location that specifies where to find the networking pool information for the Allocate networking method. If this is a relative path, it is evaluated with respect to $GLOBUS_LOCATION. For more information on configuring associations, see below.

For more information about the deployment request, see the TP1.1.1 interfaces documentation.

3.1.4. Optional: Networking assocations

The workspace service can allocate IP addresses from pools of addresses (called associations) for the Allocate networking method. The name of the file is the association name, and the directory to find all such files must be configured in the workspace factory JNDI config (see above).

The service is packaged with two sample association files public and private. These files can be found in the example associations directory: $GLOBUS_LOCATION/etc/workspace_service/associations. The files contain the syntax explanation as comments. Currently there is no script to auto-generate such files from higher level logic, each entry must be specified manually.

Side note: The service takes these files and inserts them into its database (uniqueness is judged by the combination of association name and IP address). This happens during the container boot process in the factory service initialization. The files themselves are therefore not the authoritative source of association information, but are for administrator convenience. Note that currently there is no support for assessing if an administrator has removed any entries between boots of the GT container. If you have added any entries, it will add them, but there is no support for removing database entries besides resetting the entire database (and losing all persistent state) by using "ant resetDB" from $WORKSPACE_HOME. To implement such a feature would require a much longer container boot process as well as shutting all workspace down that used the removed address.

3.1.5. Optional: Resource implementation

To edit the plugin used by the workspace service to instantiate virtual machines, edit $GLOBUS_LOCATION/etc/workspace_service/jndi-config.xml

In jndi-config.xml, find the resourceClass element in the WorkspaceService section. This value controls what plugin is used, and will not take effect unless the container is restarted with a new configuration.

Important

Only the XenLocal plugin is supported in this version.

Another value to use is "XenDummy" in place of "XenLocal". XenDummy will do everything the XenLocal plugin does, except that it will not actually call on the backend program. This is only useful for service testing and development.

3.2. Workspace backend program configuration

All configurations are specified in /etc/worksp.conf.

There are a lot of possible configurations for the backend program, but only a few that would be typically different than the default values. Documentation for each option is included in the worksp.conf.example file distributed under $WORKSPACE_HOME/workspace_control.

3.2.1. sudo

For Xen3 and for one method of passing the workspace image configuration information, the sudo program must be installed on the same node as the backend program. For more information about the mentioned networking configuration, see below.

As root with the visudo command, add this line to the /etc/sudoers file that reflects the user running the GT container (the first word) and the correct full path to the worksp program:

globus ALL=(ALL) NOPASSWD: /usr/bin/worksp

Note that you should us an absolute path and that it must match the exact path configured in a) the workspace service settings if you are using the backend program setup.py installation method ("install option A") or b) the exact path that the wrapper script call when using that less 'invasive' installation procedure ("installation option B"). It's of course recommended that secure file permissions are used for the program and its image/persistence directories.

In order for the workspace service to prefix calls to the backend program with the sudo invocation, you must configure that in the jndi-config.xml file as discussed above.

3.2.2. Image cache

A local image cache directory should be created. This directory is used to resolve relative filenames given to the workspace service for the kernel and VM image files. This setting can be located in the [images] section, it is called localdir.

3.2.3. Authorized kernels

This version of the workspace service does not support unauthorized client kernels. As such, the guestkernels configuration under the [images] section lists the kernels workspaces are allowed to be instantiated with. A client can choose from these kernels in the metadata, but they must already exist at the hypervisor node and must be in the guestkernels list.

3.2.4. Networking

With Xen, for each NIC on the hypervisor node, it is typical to create a corresponding bridge such that guest virtual machines may be bridged to each NIC.

Support for multiple physical and virtual NICs is present in this version, expressed in terms of a networking association string. A client will specify the required association in the metadata, assuming the deployment nodes support that association (otherwise, a creation fault will be thrown). An association is created by configuring an entry for it in the /etc/worksp.conf file. This should match the service association configurations.

For example, the node could have two physical NICs, one bound to a private LAN and one bound to the Internet. To accomodate virtual networking cards being bound to both NICs, a site admin could create two bridges, xen-br0 and xen-br1. Then an association would be configured for each bridge, e.g., one called "public" and one called "private" for example.

In the sample /etc/worksp.conf file, find the association_ settings in the [networking] section. If xen-br0 is the 'public' interface, add this line:

association_0: public,xen-br0; AA; 140.221.0.0/16

This specifies that there will be an association called 'public' with bridge 'xen-br0' with MAC address prefixes of 'AA' given to virtual NICs bound to this bridge and with an authorized IP range of 140.221.0.0 to 140.221.255.255. Setting up the 'private' bridge would be analogous.

Important

The MAC address cannot have a truly random prefix. The sample works.conf contains an error by using "none" for the MAC adress prefix.

Quoting the current entry at http://wiki.xensource.com/xenwiki/XenNetworking

When choosing MAC addresses to use, ensure you choose a unicast address.
That is, one with the low bit of the first octet set to zero. For example, an
address starting aa: is OK but ab: is not. It is best to keep to the range of
addresses declared to be "locally assigned" (rather than allocated globally to
hardware vendors). These have the second lowest bit set to one in the first
octet. For example, aa: is OK, a8: isn't.

In summary, an address of the following form should be OK:

XY:XX:XX:XX:XX:XX

where X is any hexadecimal digit, and Y is one of 2, 6, A or E

Note: if you only have one bridge for virtual network cards, a simple configuration would be to list one association like so:

association_0: default; AA; 1.0.0.0/16

Note that no bridge name is given, the default bridge as determined by Xen will be used. Note the IP range is fake; if you want to not keep track of workspace IP assignments (to prevent conflicts) or to not check if a requested IP setting is valid for the bridge, set the check_ip_ranges, track_MAC_assignments, and track_IP_assignments settings to 'false'.

3.2.5. In-VM Networking

To actually enact networking changes (using the Allocate networking method for example), the VM must set its own L3 networking information (IP address, default gateway, etc) from inside the VM. Currently we have a simple script that runs instead of an OS distribution's networking script. This script receives a configuration message from the workspace backend program, parses it, and enacts the changes by using the ip or ifconfig programs (it looks for ifconfig first) and the route program. (sh, sed, cat, and echo are also required).

Find this script at $WORKSPACE_HOME/workspace_control/00vm_boot.sh, it should be run as soon as possible (after the filesystem becomes read/write though) in your OS's init script system (found in /etc/). Support for hostname, DNS, and gateway will be added very soon.

To test compatibility with your images and to test what would happen, there is a DRYRUN configuration at the top of the script that can be set to "true" in order for it to not actually enact any changes.

The workspace backend program has two options for sending this script configuration information such that it is present while the VM is booting. It first looks for parameters passed via the kernel command line extra arguments. If those are not present it looks for a file at >/.workspaceconfig. This file is placed onto the virtual machine's root disk by the workspace backend program via the mount command right before starting the VM. If this fails (for example, if sudo is not configured to allow the mount command to succeed), the command line approach will be used. We are not supporting the command line only because there is a very limiting buffer size in the Linux kernel commandline code that will not prevent one or two NICs from being configured but will prevent sending a lot of other information as well (which we will soon be supporting). Besides requireing sudo, a directory /tmp/mnt is assumed to be set aside for the workspace backend program.

4. Testing

To test the workspace service, you can use the sample metadata and deployment files and a test image.

4.1. Using the workspace client to test the service

Before running the client, stage an image to the node running the workspace service and the hypervisor (in this version they need to be the same).

The default service URL it uses is listed in the help message:

$GLOBUS_LOCATION/bin/workspace --help

If this does not correspond to the URL of the WorkspaceFactoryService that prints when you start the container, you should change that by using the "--service" flag.

The --metadata flag is used to point to a metadata instance on the filesystem of the client.

Inspect $GLOBUS_LOCATION/share/workspace_client/sample-workspace.xml and $GLOBUS_LOCATION/share/workspace_client/sample-deployment-request.xml

One of the most important fields to note, it is distributed with the <path> URI = file://ttylinux-xen

That signals that it is a relative path from the local image cache on the Xen node. It could be an absolute path as well, e.g., file:///home/client/image

In the sample metadata and deployment request XML instances, there are embedded comments for other fields. Also, consult the interfaces section online starting here.

Run the client (adding --deploy is unecessary, it is the default subcommand):

$GLOBUS_LOCATION/bin/workspace --file workspace.epr  \
      --metadata $GLOBUS_LOCATION/share/workspace_client/sample-workspace.xml \
      --request  $GLOBUS_LOCATION/share/workspace_client/sample-deployment-request.xml

You may need to also specify the service endpoint, e.g.:

-s https://192.168.12.1:8443/wsrf/services/WorkspaceFactoryService

After deployment, the client will subscribe for notifications.

In another terminal, use the workspace program with EPR file to manage it. The EPR file was written out to the path specified by the --file flag in the --deploy operation.

$GLOBUS_LOCATION/bin/workspace -e workspace.epr --start
$GLOBUS_LOCATION/bin/workspace -e workspace.epr --shutdown
$GLOBUS_LOCATION/bin/workspace -e workspace.epr --pause
$GLOBUS_LOCATION/bin/workspace -e workspace.epr --destroy

As of TP1.1, you do not need to specify an XML file for the deployment request's three most common (currently) parameters: the workspace memory, duration, and requested deployment state. These can now be specified via the client command line. If the XML instance is not specified with the --request parameter, the three parameters "--deploy-duration", "--deploy-mem", and "--deploy-state" with respective arguments are required. Also, you may additionally provide the "--dryrun" parameter which will just print out what the deployment request would have looked like in its XML form (and then the program exits without actually making a grid service call).

Also new in TP1.1 is the ability to query the factory resource properties with the "--factoryrp" command (no arguments to it, just that and a non-default factory service URL if needed, see above for an example of that).

5. Troubleshooting

  • Problem: This error is reported:

    FATAL persistence.PersistenceAdapter [main,initialize:82] javax.naming.NamingException:
    Could not create resource factory, ClassNotFoundException:org.apache.commons.dbcp.BasicDataSourceFactory
    Failed to start container: Failed to initialize 'WorkspaceFactoryService' service
    [Caused by: ; nested exception is: javax.naming.NamingException:
       Bean initialization failed [Root exception is java.lang.RuntimeException: ]]

    Solution: Add the globus_database_common libraries to $GLOBUS_LOCATION/lib -- this can be accomplished in two ways:

    1. If you used the Globus installer to install Java core, run "make globus_database_common postinstall" in the installation directory.
    2. If you are using the Java core standalone tarball, you need to obtain commons-dbcp.jar and commons-pool.jar from the Apache project or alternatively from Globus' CVS (in database/java/common/source/lib) or the Globus installer (in source-trees/database/java/common/source/lib). Place these jars in $GLOBUS_LOCATION/lib

  • Problem: Sometimes, from the start of the workspace's deployment, one of the VM's NICs is unreachable. (specifically, the ARP protocol does not resolve the IP address to a MAC address)

    Solution: Make sure the MAC address prefix is valid. See the backend networking configuration section for more details and the setting to use.