|
|
||||
|
|
||||
|
|
||||
Table of Contents
This guide contains configuration information for system administrators working with Workspaces. It provides references to information on procedures typically performed by system administrators, including installing, configuring, deploying, and testing the installation.
This information is in addition to the basic Globus Toolkit prerequisite, overview, installation, security configuration instructions in the GT 4.0 System Administrator's Guide. Please read through this guide before continuing.
In the future we will provide detailed instructions for configuring GT4 with the bare necessities for the Workspace Services. The short version of these instructions is to install the Java GT4 core with common database tools:
make wsjava common globus_proxy_utils globus_simple_ca \ globus_simple_ca_setup globus_database_common postinstall
(if you have proxies and proxy tools from another GT[2-4] installation, you can leave out these middle targets: "common globus_proxy_utils globus_simple_ca globus_simple_ca_setup")
Download the service here.
After expanding the tarball, navigate to the top level
directory created which we will refer to as
$WORKSPACE_HOME.
$WORKSPACE_HOME contains two directories,
workspace_service contains
the GT4
services and workspace_control
contains the program that it is necessary to
install
on a physical node running the Xen hypervisor. In this
version, these will be installed on the same
node.
Currently, only the source is distributed.
The root build file for the Workspace Services is
$WORKSPACE_HOME/workspace_service/build.xml
This will install the client and service:
export GLOBUS_LOCATION=/path/to/GL cd $WORKSPACE_HOME/workspace_service ant deploy
Before testing the service, you need to configure it and install and configure the backend program.
The program is implemented in Python, located under the
$WORKSPACE_HOME/workspace_control
directory. There are two ways to install the backend
program: use the supplied setup.py
script that will install the Python packages to the
regular OS Python location or use a supplied wrapper
script that sets up the PYTHONPATH
environment variable to correctly point at the program's
libraries.
For Xen3 and for one method of passing the workspace image configuration information, the sudo program must be installed on the same node as the backend program. See this section.
The setup.py script is a
regular Python
distutils
script that will install the workspace backend program
worksp and its libraries to the
regular Python program location on your system.
The distutils script inherits functionality besides "install," including RPM generation. For help with commands, there are two built-in help messages:
python setup.py --help python setup.py --help-commands
We will use the install subcommand. Help with this command can be obtained by running:
python setup.py --help install
As root, run
python setup.py install
This will install the worksp
program to a location on a regular PATH (typically
/usr/bin), its libraries to a
Python site-packages directory
(typically under /usr/lib), and
the worksp configuration file to
/etc/worksp.conf.example
You should now be able to see this message when running
worksp
arguments? for help use --help
Copy /etc/worksp.conf.example to
/etc/worksp.conf
After a successful installation, proceed to
configure
the backend program by editing
/etc/worksp.conf
If you do not use the setup.py program, you need to
configure a shell script to properly setup the
PYTHONPATH to call the worksp
program. This is like many of the scripts in
$GLOBUS_LOCATION/bin that
simply set up the Java classpath.
An example called
workspace_control is installed in
$GLOBUS_LOCATION/bin when
the Workspace Service is installed.
The script looks for an environment variable called
WORKSPACE_HOME to allow it to easily
point to the proper Python libraries.
The best approach if you are tied to option B is to
simply edit the (very simple) shell script
$GLOBUS_LOCATION/bin/workspace_control
to point to an absolute path instead, to alleviate the need
for this environment variable.
WORKSPACE_HOME
would need to be set in the environment
of the container running the Workspace
Service.
You should now be able to see this message when running
$GLOBUS_LOCATION/bin/workspace_control
arguments? for help use --help
Copy
$WORKSPACE_HOME/worksp.conf.example
to
/etc/worksp.conf
After a successful installation, proceed to
configure
the backend program by editing
/etc/worksp.conf
After installing the Workspace Service, files that alter
its configuration are stored in the
$GLOBUS_LOCATION/etc/workspace_service/
directory.
To edit the list of authorized users of the factory
service, add DNs to
$GLOBUS_LOCATION/etc/workspace_service/workspace-grid-mapfile
/etc/grid-security/grid-mapfile).
The service is distributed with zero entries, which
disables access to workspace creation entirely.
Add a DN to the list in normal Globus fashion (if the DN has spaces in it, use quotes around it). The gridmap authorization module requires a username mapping, but it is irrelevant to this service, so using e.g. "fakeuser" for each DN entry is OK.
Other authorization methods are now available (documentation will be updated shortly).
After a workspace is created, only the creator may manage the WSRF resource. This is configured programmatically for each workspace, so no grid-mapfile for the workspace service is needed.
To edit the path to the workspace backend program that
was installed, edit
$GLOBUS_LOCATION/etc/workspace_service/jndi-config.xml
In jndi-config.xml, find the
workspLocation entry under the
WorkspaceService section. If
the program you want will not be on the container environment's
PATH (or you are using sudo), then use
an absolute path to
the program. The service is distributed with the proper
configuration for when the backend program is installed
with option
A.
In the same section is the sudo configuration ("true" or "false" to prefix commands to the backend program with sudo), for more explanation on what it is for, see below
Currently, the factory can be configured with three simple
policies concerning workspace deployment. Each value
is listed in the WorkspaceFactoryService
section of
$GLOBUS_LOCATION/etc/workspace_service/jndi-config.xml
The defaultRunningTimeMinutes value is
used if the DeploymentTime element of
the deployment request is not specified.
The maxRunningTimeMinutes value can
not be exceed by the value of the
DeploymentTime element of the deployment
request.
The WSRFResouceMinutesPastRunningTime value
configures the default termination time of the WSRF resource
representing the workspace. When a workspace is shutdown,
its representation is not terminated (one reason being that
it can be started again). This value is added on to the
running time and is currently only configurable at this
location.
The optional architecture value can
be one of the JSDL ProcessorArchitectureEnumeration values such
as "x86". If you are running Xen on other architectures, this
should be changed appropriately.
The optional vmm value can
be either Xen or VMware, but this will currently be Xen.
The optional vmmVersions value can
be a comma separated list of one-word strings (deciding on
values to use is a deployment issue within VOs).
The optional associationFileLocation
value is a directory location that specifies where to find the
networking pool information for the Allocate networking method.
If this is a relative path, it is evaluated with respect to
$GLOBUS_LOCATION. For more information
on configuring associations, see
below.
For more information about the deployment request, see the TP1.1.1 interfaces documentation.
The workspace service can allocate IP addresses from pools of addresses (called associations) for the Allocate networking method. The name of the file is the association name, and the directory to find all such files must be configured in the workspace factory JNDI config (see above).
The service is packaged with two sample association files
public and private.
These files can be found in the example associations directory:
$GLOBUS_LOCATION/etc/workspace_service/associations.
The files contain the syntax explanation as comments. Currently
there is no script to auto-generate such files from higher level
logic, each entry must be specified manually.
Side note: The service takes these files and inserts them into its database (uniqueness is judged by the combination of association name and IP address). This happens during the container boot process in the factory service initialization. The files themselves are therefore not the authoritative source of association information, but are for administrator convenience. Note that currently there is no support for assessing if an administrator has removed any entries between boots of the GT container. If you have added any entries, it will add them, but there is no support for removing database entries besides resetting the entire database (and losing all persistent state) by using "ant resetDB" from $WORKSPACE_HOME. To implement such a feature would require a much longer container boot process as well as shutting all workspace down that used the removed address.
To edit the plugin used by the workspace service to
instantiate virtual machines, edit
$GLOBUS_LOCATION/etc/workspace_service/jndi-config.xml
In jndi-config.xml, find the
resourceClass element in the
WorkspaceService section. This value
controls what plugin is used, and will not take effect
unless the container is restarted with a new configuration.
Another value to use is "XenDummy" in place of "XenLocal". XenDummy will do everything the XenLocal plugin does, except that it will not actually call on the backend program. This is only useful for service testing and development.
All configurations are specified in
/etc/worksp.conf.
There are a lot of possible configurations for the backend
program, but only a few that would be typically different
than the default values. Documentation for each option
is included in the worksp.conf.example file
distributed under
$WORKSPACE_HOME/workspace_control.
For Xen3 and for one method of passing the workspace image configuration information, the sudo program must be installed on the same node as the backend program. For more information about the mentioned networking configuration, see below.
As root with the visudo command, add this
line to the /etc/sudoers file that reflects
the user running the GT container (the first word) and the
correct full path to the worksp program:
globus ALL=(ALL) NOPASSWD: /usr/bin/worksp
Note that you should us an absolute path and that it must match the exact path configured in a) the workspace service settings if you are using the backend program setup.py installation method ("install option A") or b) the exact path that the wrapper script call when using that less 'invasive' installation procedure ("installation option B"). It's of course recommended that secure file permissions are used for the program and its image/persistence directories.
In order for the workspace service to prefix calls to the backend program with the sudo invocation, you must configure that in the jndi-config.xml file as discussed above.
A local image cache directory should be created. This directory
is used to resolve relative filenames given to the workspace
service for the kernel and VM image files. This setting
can be located in the [images] section,
it is called localdir.
This version of the workspace service does not support
unauthorized client kernels. As such, the
guestkernels configuration under
the [images] section lists the
kernels workspaces are allowed to be instantiated with.
A client can choose from these kernels in the metadata,
but they must already exist at the hypervisor node and
must be in the guestkernels list.
With Xen, for each NIC on the hypervisor node, it is typical to create a corresponding bridge such that guest virtual machines may be bridged to each NIC.
Support for multiple physical and virtual NICs is present in
this version, expressed in terms of a networking
association string. A client will
specify the required association in the metadata, assuming the
deployment nodes support that association (otherwise, a creation
fault will be thrown). An association is created by
configuring an entry for it in the
/etc/worksp.conf file. This should match
the service association configurations.
For example, the node could have two physical NICs, one bound
to a private LAN and one bound to the Internet. To accomodate
virtual networking cards being bound to both NICs, a site
admin could create two bridges, xen-br0
and xen-br1. Then an association
would be configured for each bridge, e.g., one called "public"
and one called "private" for example.
In the sample /etc/worksp.conf file,
find the association_ settings in the
[networking] section. If
xen-br0 is the 'public' interface, add
this line:
association_0: public,xen-br0; AA; 140.221.0.0/16
This specifies that there will be an association called 'public' with bridge 'xen-br0' with MAC address prefixes of 'AA' given to virtual NICs bound to this bridge and with an authorized IP range of 140.221.0.0 to 140.221.255.255. Setting up the 'private' bridge would be analogous.
The MAC address cannot have a truly random prefix. The sample works.conf contains an error by using "none" for the MAC adress prefix.
Quoting the current entry at http://wiki.xensource.com/xenwiki/XenNetworking
When choosing MAC addresses to use, ensure you choose a unicast address. That is, one with the low bit of the first octet set to zero. For example, an address starting aa: is OK but ab: is not. It is best to keep to the range of addresses declared to be "locally assigned" (rather than allocated globally to hardware vendors). These have the second lowest bit set to one in the first octet. For example, aa: is OK, a8: isn't. In summary, an address of the following form should be OK: XY:XX:XX:XX:XX:XX where X is any hexadecimal digit, and Y is one of 2, 6, A or E
Note: if you only have one bridge for virtual network cards, a simple configuration would be to list one association like so:
association_0: default; AA; 1.0.0.0/16
Note that no bridge name is given, the default bridge as
determined by Xen will be used. Note the IP range is fake;
if you want to not keep track of workspace IP assignments (to
prevent conflicts) or to not check if a requested IP setting
is valid for the bridge, set the
check_ip_ranges,
track_MAC_assignments, and
track_IP_assignments settings to 'false'.
To actually enact networking changes (using the Allocate
networking method for example), the VM must set its own
L3 networking information (IP address, default gateway, etc)
from inside the VM. Currently we have a
simple script that runs instead of an OS distribution's
networking script. This script receives a configuration message
from the workspace backend program, parses it, and enacts the
changes by using the ip or
ifconfig programs (it looks for
ifconfig first) and the
route program. (sh, sed, cat, and echo
are also required).
Find this script at
$WORKSPACE_HOME/workspace_control/00vm_boot.sh,
it should be run as soon as possible (after the filesystem
becomes read/write though) in your OS's init script system
(found in /etc/). Support for hostname, DNS, and gateway will
be added very soon.
To test compatibility with your images and to test what would happen, there is a DRYRUN configuration at the top of the script that can be set to "true" in order for it to not actually enact any changes.
The workspace backend program has two options for sending this
script configuration information such that it is present while
the VM is booting. It first looks for parameters passed via
the kernel command line extra arguments. If those are not present
it looks for a file at >/.workspaceconfig.
This file is placed onto the virtual machine's root disk by
the workspace backend program via the mount command right before
starting the VM. If this fails (for example, if sudo is not
configured to allow the mount command to succeed), the command
line approach will be used. We are not supporting the command
line only because there is a very limiting buffer size in the
Linux kernel commandline code that will not prevent one or two
NICs from being configured but will prevent sending a lot of
other information as well (which we will soon be supporting).
Besides requireing sudo, a directory /tmp/mnt
is assumed to be set aside for the workspace backend program.
To test the workspace service, you can use the sample metadata and deployment files and a test image.
Before running the client, stage an image to the node running the workspace service and the hypervisor (in this version they need to be the same).
The default service URL it uses is listed in the help message:
$GLOBUS_LOCATION/bin/workspace --help
If this does not correspond to the URL of the WorkspaceFactoryService that prints when you start the container, you should change that by using the "--service" flag.
The --metadata flag is used to point to a metadata instance on the filesystem of the client.
Inspect
$GLOBUS_LOCATION/share/workspace_client/sample-workspace.xml
and
$GLOBUS_LOCATION/share/workspace_client/sample-deployment-request.xml
One of the most important fields to note, it is
distributed with the <path> URI =
file://ttylinux-xen
That signals that it is a relative path from the local
image cache on the Xen node. It could be an absolute path
as well, e.g., file:///home/client/image
In the sample metadata and deployment request XML instances, there are embedded comments for other fields. Also, consult the interfaces section online starting here.
Run the client (adding --deploy is unecessary, it is the default subcommand):
$GLOBUS_LOCATION/bin/workspace --file workspace.epr \
--metadata $GLOBUS_LOCATION/share/workspace_client/sample-workspace.xml \
--request $GLOBUS_LOCATION/share/workspace_client/sample-deployment-request.xml
You may need to also specify the service endpoint, e.g.:
-s https://192.168.12.1:8443/wsrf/services/WorkspaceFactoryService
After deployment, the client will subscribe for notifications.
In another terminal, use the workspace program with EPR file to manage it. The EPR file was written out to the path specified by the --file flag in the --deploy operation.
$GLOBUS_LOCATION/bin/workspace -e workspace.epr --start $GLOBUS_LOCATION/bin/workspace -e workspace.epr --shutdown $GLOBUS_LOCATION/bin/workspace -e workspace.epr --pause $GLOBUS_LOCATION/bin/workspace -e workspace.epr --destroy
As of TP1.1, you do not need to specify an XML file for the deployment request's three most common (currently) parameters: the workspace memory, duration, and requested deployment state. These can now be specified via the client command line. If the XML instance is not specified with the --request parameter, the three parameters "--deploy-duration", "--deploy-mem", and "--deploy-state" with respective arguments are required. Also, you may additionally provide the "--dryrun" parameter which will just print out what the deployment request would have looked like in its XML form (and then the program exits without actually making a grid service call).
Also new in TP1.1 is the ability to query the factory resource properties with the "--factoryrp" command (no arguments to it, just that and a non-default factory service URL if needed, see above for an example of that).
Problem: This error is reported:
FATAL persistence.PersistenceAdapter [main,initialize:82] javax.naming.NamingException: Could not create resource factory, ClassNotFoundException:org.apache.commons.dbcp.BasicDataSourceFactory Failed to start container: Failed to initialize 'WorkspaceFactoryService' service [Caused by: ; nested exception is: javax.naming.NamingException: Bean initialization failed [Root exception is java.lang.RuntimeException: ]]
Solution: Add the globus_database_common libraries to $GLOBUS_LOCATION/lib -- this can be accomplished in two ways:
If you are using the Java core standalone tarball,
you need to obtain
commons-dbcp.jar and
commons-pool.jar from the
Apache project or alternatively from Globus' CVS
(in database/java/common/source/lib) or the
Globus installer
(in source-trees/database/java/common/source/lib).
Place these jars in
$GLOBUS_LOCATION/lib
Problem: Sometimes, from the start of the workspace's deployment, one of the VM's NICs is unreachable. (specifically, the ARP protocol does not resolve the IP address to a MAC address)
Solution: Make sure the MAC address prefix is valid. See the backend networking configuration section for more details and the setting to use.