|
|
||||
|
|
||||
|
|
||||
Table of Contents
This guide contains configuration information for system administrators working with Workspaces. It provides references to information on procedures typically performed by system administrators, including installing, configuring, deploying, and testing the installation.
This information is in addition to the basic Globus Toolkit prerequisite, overview, installation, security configuration instructions in the GT 4.0 System Administrator's Guide. Please read through this guide before continuing.
Configuring GT4 with the bare necessities for the Workspace Service requires you to install the Java GT4 core with security tools:
make wsjava common globus_proxy_utils globus_simple_ca \ globus_simple_ca_setup postinstall
If you can obtain certificates and proxy tools from some other location, all that is required is Java GT4 core which is available as a binary tarball from the Globus download page.
If you only need to install the workspace client see the client only section of this manual before proceeding. Some of the client-only installation options do not require you to install GT4 first.
There are different ways to install a workspace client environment:
Once installed, see the Workspace User Guide.
Download the Xen based client image, listed on the Workspace download page.
Untar the archive and edit the enclosed config file to point at the correct kernel and image.
Log in as root (password is 'root'). As root, run this:
su - globus
to log in to the Globus user environment.
For more information about the client environment on this VM (and in general), see the Workspace User Guide.
The VM is set up to DHCP, you can change this by editing the
/etc/conf.d/net file. The file contains
a commented out example of a static IP assignment. Then run
/etc/init.d/net.eth0 restart
Download the VMware based client image, listed on the Workspace download page.
Untar the archive and point the VMware GUI to the enclosed vmx file (File -> Open).
Log in as root (password is 'root'). As root, run this:
su - globus
to log in to the Globus user environment.
For more information about the environment on this VM (and in general about the client environment), see the Workspace User Guide.
The VM is set up to DHCP, you can change this by editing the
/etc/conf.d/net file. The file contains
a commented out example of a static IP assignment. Then run
/etc/init.d/net.eth0 restart
. The Workspace User Guide discusses the exact steps to take to accomplish this in the VM Networking section.
The basic Globus Java client environment almost entirely overlaps with the server environment. We provide an option to download a tarball of this environment with the Workspace sample client already installed into it.
Installing the client this way assumes you have a working Java JRE (1.4+) as well as externally obtained certificates.
The proxy generation tools included with this tarball are not the recommended tools for use with Linux. There is a problem with Java's interaction with Linux terminals that results in typed characters possibly being seen by an onlooker.
Download the "Binary, Globus Java core plus client" tarball from the Workspace download page.
Expand the tarball and set your
GLOBUS_LOCATION environment variable
to the created directory.
Now see the Workspace User Guide.
The GAR format is a portable, binary archive of everything needed to deploy the client into a pre-existing container.
Installing the client via GAR files assumes you have a working GT4 Java core installation. A Java JRE (1.4+) and Ant (1.6+ and 1.6.1+ if using Java 1.5) is assumed.
Configuring GT4 with the bare necessities for the Workspace reference client requires you to install the Java GT4 core with security tools:
make wsjava common globus_proxy_utils globus_simple_ca \ globus_simple_ca_setup postinstall
If you can obtain certificates and proxy tools from some other location, all that is required is Java GT4 core which is available as a binary tarball from the Globus Toolkit download page.
Download the "Client GAR files" tarball from the Workspace download page and expand it to a random directory..
Set the GLOBUS_LOCATION environment
variable as your environment requires (bourne shell shown):
export GLOBUS_LOCATION=/path/to/globus
Set the ANT_HOME environment
variable as your environment requires (bourne shell shown):
export ANT_HOME=/path/to/ant/home
Deploy the three GAR files:
cd $GLOBUS_LOCATION ./bin/globus-deploy-gar workspace_schemas.gar ./bin/globus-deploy-gar workspace_common.gar ./bin/globus-deploy-gar workspace_client.gar
More detailed instructions are available for globus-deploy-gar and globus-undeploy-gar if necessary.
Now see the Workspace User Guide.
Installing the client from source assumes you have a working GT4 Java core installation. A Java JDK (1.4+) and Ant (1.6+ and 1.6.1+ if using Java 1.5) is assumed.
Configuring GT4 with the bare necessities for the Workspace reference client requires you to install the Java GT4 core with security tools:
make wsjava common globus_proxy_utils globus_simple_ca \ globus_simple_ca_setup postinstall
If you can obtain certificates and proxy tools from some other location, all that is required is Java GT4 core which is available as a binary tarball from the Globus Toolkit download page.
Download the "Grid Service" source tarball from the Workspace download page.
Set the GLOBUS_LOCATION environment
variable as your environment requires (bourne shell shown):
export GLOBUS_LOCATION=/path/to/globus
Set the ANT_HOME environment
variable as your environment requires (bourne shell shown):
export ANT_HOME=/path/to/ant/home
The root build file for the Workspace Service is
workspace-service/build.xml and it
contains a special build target to just build the client.
Build and install the client:
cd workspace-service ant client
Now see the Workspace User Guide.
Download the grid service and workspace-control from the download page.
One tarball contains
the GT4
services and the other contains the
workspace-control
program which is required to be
installed
on a physical node running the Xen hypervisor. The service and
workspace-control can be installed on the
same node or on
different nodes. And further, you can install the
workspace-control program to multiple hypervisors and the service
will manage them as a pool of resources.
Currently, only the source is distributed for the service.
The root build file for the Workspace Service is
workspace-service/build.xml
This will install the client and service:
export GLOBUS_LOCATION=/path/to/GL cd workspace-service ant deploy
Before testing the service, you need to configure it and install and configure the workspace-control program.
As of this TP1.2.2 version of the workspace service you also need to install DHCP and ebtables on each hypervisor node. For more information on why this is now necessary and how it will not interfere with a site's pre-existing DHCP server, see the DHCP overview and configuration section.
The program is implemented in Python and comes with its own installer. It requires Python 2.3+
If you are planning on installing workspace-control to multiple hypervisor nodes so that the service can manage the nodes as a resource pool, you should install to one first. Once you have edited the configuration file as you need to, it can all be replicated across your resource pool by using the automatic installer.
For Xen3, network information delivery to workspaces, and for any functionality requiring workspace image editing (currently only certificate staging), the sudo program must be installed on the same node as the backend program. See the configuring sudo section for more information.
For workspace-control to support networking information delivery to VMs, you are required to install DHCP and ebtables on each hypervisor node. When networking information in a workspace needs to change at its startup (which is typical upon deployment), workspace-control will make a call via sudo to a program that adds a MAC address to IP mapping into the local DHCP server for each of the workspace's NICs that need to be configured. It will also adjust ebtables rules for each of the workspace's NICs that need to be configured: these make sure the NICs are using the proper MAC and IP address as well as directing DHCP requests to the local DHCP server only. For more information on why this is now necessary and how it will not interfere with a site's pre-existing DHCP server, see the DHCP overview and configuration section.
The install.py program has a help switch (-h) that will explain its options, but briefly there are three main modes of the installer, excert from the -h output:
-n, --noninteractive Don't ask the user anything (for automated install with
a well-known conf file).
-o, --onlyverify Just run the setup tests and print what would have
happened in --noninteractive mode. Good option to try
first since it will do nothing to the filesystem.
-i, --install Install the program, make needed directories, and set
them with default permissions. Will block to ask you
questions if necessary.
The --noninteractive mode is a good, fast choice if you have no /opt/workspace directories on your system and just want to use the default /opt/workspace hierarcy that comes in the sample configuration file. The installer makes adding hypervisor nodes to your resource pool very easy, once you have decided on some of the configuration choices.
For more on running the installer, see the configuration section.
For an overview of what these tools are used for and how they are situated in terms of your site's network setup, see the DHCP overview and configuration section.
For a lot of hypervisor nodes this may not be necessary as the packages are already present. Check your distribution tools for ISC's DHCP server and ebtables. Or simply try to run the command "/sbin/ebtables" and look for the "/etc/dhcp/dhcpd.conf" file and accompanying "/etc/init.d/dhcpd" script. The ISC DHCP server is the most standard and widely supported DHCP server so it is likely that is what you have installed.
All major Linux distributions include these two packages, you should be able to easily install them with your package management system. For example, "rpm -ihv dhcp-*.rpm", "apt-get install dhcp", "emerge dhcp", etc. And the same for ebtables.
ebtables requires kernel support in dom0, the default Xen kernel includes this support. If your dom0 kernel does not include these for some reason, the options to enable are under Networking :: Networking options :: Network packet filtering :: Bridge Netfilter Configuration :: Ethernet Bridge Tables
Any recent version of each package should be compatible, the distributed scripts that automate the configurations were tested with ISC DHCP 3.0.3 and ebtables 2.0.6 userspace tools.
The scripts that interact with the packages are straightforward and can be swapped with alternate implementations if you need to; also feel free to ask questions about the implementation on workspace-user@globus.org.
After installing the Workspace Service, files that alter
its configuration are stored in the
$GLOBUS_LOCATION/etc/workspace_service/
directory.
To edit the list of authorized users of the factory
service, add DNs to
$GLOBUS_LOCATION/etc/workspace_service/workspace-grid-mapfile
/etc/grid-security/grid-mapfile).
The service is distributed with zero entries, which
disables access to workspace creation entirely.
Add a DN to the list in normal Globus fashion (if the DN has spaces in it, use quotes around it). The gridmap authorization module requires a username mapping, but it is irrelevant to this service, so using e.g. "fakeuser" for each DN entry is OK. We support grid-mapfiles instead of simple access control lists for maximum compatibility with predeployed installations.
Other (attribute based) authorization methods are now available via the plugin system.
After a workspace is created, only the creator may manage the WSRF resource. This is configured programmatically for each workspace, so no grid-mapfile for the workspace service is needed, just the factory.
There are currently two service implementations, xenlocal and xenSSH.
xenlocal does not handle the resource pool model, it is only for the cases where you want to run one workspace service on the same node as the (sole) hypervisor it manages.
xenSSH on the other hand will enable you to manage a pool of resources from one running service. The service keeps track of the what resource are available at the current time, adding and subtracting the resources used, keeping a view of what is available at the different hypervisor nodes. In this version of the service, the first introduction of the resource pool model, this only tracks the RAM available for VMs at the node. To configure the resource pools, see here.
To edit the path to the workspace backend program(s) that
was (were) installed, edit
$GLOBUS_LOCATION/etc/workspace_service/jndi-config.xml
In jndi-config.xml, find the
backendPath entry under the
WorkspaceService section. Use
an absolute path to the program. The service is distributed
with the proper configuration for when the backend program is
installed with the workspace-control default configurations
and installer.
Note: if you are using the resource pool
setup and the XenSSH implementation, this
backendPath entry refers to the program's
location on the VMM nodes.
Currently, the factory can be configured with three simple
policies concerning workspace deployment. Each value
is listed in the WorkspaceFactoryService
section of
$GLOBUS_LOCATION/etc/workspace_service/jndi-config.xml
The defaultRunningTimeMinutes value is
used if the DeploymentTime element of
the deployment request is not specified.
The maxRunningTimeMinutes value can
not be exceed by the value of the
DeploymentTime element of the deployment
request.
The WSRFResouceMinutesPastRunningTime value
configures the default termination time of the WSRF resource
representing the workspace. When a workspace is shutdown,
its representation is not terminated (one reason being that
it can be started again). This value is added on to the
running time and is currently only configurable at this
location.
The optional architecture value can
be one of the JSDL ProcessorArchitectureEnumeration values such
as "x86". If you are running Xen on other architectures, this
should be changed appropriately.
The optional vmm value will currently be Xen.
The optional vmmVersions value can
be a comma separated list of one-word strings (deciding on
values to use is a deployment issue within VOs).
The optional associationFileLocation
value is a directory location that specifies where to find the
networking pool information for the Allocate networking method.
If this is a relative path, it is evaluated with respect to
$GLOBUS_LOCATION. For more information
on configuring associations, see
below.
The optional certificateFileLocation
value is a directory location that specifies where to find the
certificate and keyfiles that an association may optionally
refer to.
The optional resourcepoolFileLocation
value is a directory location that specifies where to find the
resource pool information for the xenSSH implementation.
If this is a relative path, it is evaluated with respect to
$GLOBUS_LOCATION. For more information
on configuring a resource pool, see
below.
For more information about the deployment request, see the TP1.2.2 interfaces documentation.
This is optional because you may be using "local" mode where the service is being hosted on the only hypervisor. But in most cases, the resource pool mode of operation (where you deploy one grid service that manages many hypervisor nodes) is recommended. See Service Implementation.
As noted above, the resourcepoolFileLocation directory can contain files that represent resource pools. The file format is currently very simple: for each node in the pool, list the hostname and the amount of RAM it can spare for running guest VMs.
These configurations are currently under the same constraint as the association entries with respect to removing entries (see DB side note).
The workspace service can allocate IP addresses from pools of addresses (called associations) for the Allocate networking method. The name of the file is the association name, and the directory to find all such files must be configured in the workspace factory JNDI config (see above).
The service is packaged with two sample association files
public and private.
These files can be found in the example associations directory:
$GLOBUS_LOCATION/etc/workspace_service/associations.
The files contain the syntax explanation as comments. Currently
there is no script to auto-generate such files from higher level
logic, each entry must be specified manually or generated via script.
Side note: The service takes these files and inserts them into its database (uniqueness is judged by the combination of association name and IP address). This happens during the container boot process in the factory service initialization. The files themselves are therefore not the authoritative source of association information, but are for administrator convenience. Note that currently there is no support for assessing if an administrator has removed any entries between boots of the GT container. If you have added any entries, it will add them, but there is no support for removing database entries besides manually deleting them via SQL or resetting the entire database (and losing all persistent state) by using "ant resetDB" from the service source tree. To implement such a feature would require a longer container boot process as well as shutting all workspace down that used the removed address (if there is user demand for this convenience, we can take the time to add it).
After untar-ing the package on the VMM node, move the sample configuration file to a permanent place and then edit it. There are many configurations for directory paths in the file and they almost all point to a hierarchy under /opt/workspace. These directories are for isolating and mounting the guest images as well as giving the program places for persistence, tmp files, and an image cache (where you can put the default kernel for example).
Every directory path is editable even when using the automatic
installer because you will pass the configuration file to the
installer. If this is your first time installing workspace-control
it is probably best if you stick with the default
/opt/workspace setup.
One of the directories is bin. The program
is placed here by default, but importantly
also tools that the program can call via sudo to
make configuration changes before deployment. Because of this,
the directory these tools are in and all of its parents must be
owned by root.
(Elevating an account's privileges to root in order to call
a particular program over sudo when the program is editable by
non-root defeats the purpose of using sudo.)
Note: the workspace-control installer can check all of the related permissions for you, create missing directories, and then adjust their permissions to be correct. There is also a detailed permissions note available in the workspace-control configuration file.
One of the necessary configurations to make is to point the program at the default kernel it should use for guests, this is one of the labelled options in the workspace-control configuration file. Another necessary configuration is to make sure its association (network pool) configurations match the workspace service's, this section is also labelled in the default configuration file.
You need to have a privileged user and unix group that will manage the VMs since we do not allow the whole program to be run as root. You can just pass the user and group information to the installer so it can know how to chown the directories.
For example, if you accept the default /opt/workspace hierarchy and your account and group are globus and globus, you could run these commands (roughly):
cd workspace-control cp worksp.conf.example /opt/workspace/worksp.conf ./install.py -c /opt/workspace/worksp.conf -a globus -g globus -n mount /boot cp /boot/vmlinuz-2.6.16-xen /opt/workspace/images/ umount /boot
(now edit the /opt/workspace/worksp.conf file to point to that kernel for guest use; also, adjust the networking information)
Just before exiting, the installer will print out the sudo policies you need to configure (see the next section).
For Xen3 control, editing the networking configurations, and for one feature where the image itself is modified (certificate staging), the sudo program must be installed on the same node as the backend program. For more information about the mentioned networking configuration, see below.
As root with the visudo command, add these
lines to the /etc/sudoers file that reflects
the user running the GT container (the first word) and the
correct full paths to the dhcp-config.sh tool, the xm tool, and
the xend daemon (needed in the case where the workspace-control
program is permitted to reboot the daemon if it has fallen).
Note: the installer will print out the exact sudo policy you need that matches the configurations it read in from the workspace-control conf file. e.g.:
You must manually add these policies to your sudo config (use visudo): globus ALL=(root) NOPASSWD: /opt/workspace/bin/dhcp-config.sh globus ALL=(root) NOPASSWD: /usr/sbin/xm globus ALL=(root) NOPASSWD: /usr/sbin/xend
This version of the workspace service does not support
unauthorized client kernels. As such, the
guestkernels configuration under
the [images] section lists the
kernels workspaces are allowed to be instantiated with.
A client can choose from these kernels in the metadata,
but they must already exist at the hypervisor node and
must be in the guestkernels list.
With Xen, for each NIC on the hypervisor node, it is typical to create a corresponding bridge such that guest virtual machines may be bridged to each NIC. This allows workspaces to be on different networks or allows spreading out of the networking load (in the case where the hypervisor's multiple NICs are on the same network).
Support for multiple physical and virtual NICs is present in this version, mapped in the grid context in terms of a networking association string. A client will specify the required association in the metadata, assuming the deployment nodes support that association (otherwise, a creation fault will be thrown). An association is created by configuring an entry for it in the workspace-control configuration file. This should match the service association configurations.
For example, the node could have two physical NICs, one bound
to a private LAN and one bound to the Internet. To accomodate
virtual networking cards being bound to both NICs, a site
admin could create two bridges, xenbr0
and xenbr1. Then an association
would be configured for each bridge, e.g., one called "public"
and one called "private" for example (a typical configuration).
In the sample workspace-control configuration file,
find the association_ settings in the
[networking] section. If
xenbr0 is the 'public' interface, add
this line:
association_0: private; xenbr0; vif0.0 ; AA:01; 192.168.0.0/24
This specifies that there will be an association called 'private' with bridge 'xenbr0' with MAC address prefixes of 'AA:01' given to virtual NICs bound to this bridge and with an authorized IP range of 192.168.0.0 to 192.168.0.255.
The third field lists the interface of the NIC that is listening for DHCP requests. This is the interface that DHCP requests originating from workspaces should be allowed to broadcast to. Directing the request to a specific interface prevents DHCP requests from being broadcast to other workspaces as well as to the real network. You can doublecheck what vif is on what bridge by running "brctl show". Normally you'd run the DHCP server in dom0 and the defaults are like so:
xenbr0 - vif0.0 xenbr1 - vif0.1 xenbr2 - vif0.2
You are not required to run the DHCP server in dom0 as the workspaces' DHCP request broadcast can be directed to any interface, but currently to run it in another domain would require altering the dhcp-config.sh callout to work remotely (this would not be hard for you to do, grep for "dhcpconfigpath" in the xen_v2.py module and prepend ssh parameters, for example).
The MAC address cannot have a truly random prefix.
Quoting the current entry at http://wiki.xensource.com/xenwiki/XenNetworking
When choosing MAC addresses to use, ensure you choose a unicast address. That is, one with the low bit of the first octet set to zero. For example, an address starting aa: is OK but ab: is not. It is best to keep to the range of addresses declared to be "locally assigned" (rather than allocated globally to hardware vendors). These have the second lowest bit set to one in the first octet. For example, aa: is OK, a8: isn't. In summary, an address of the following form should be OK: XY:XX:XX:XX:XX:XX where X is any hexadecimal digit, and Y is one of 2, 6, A or E
Note: if you only have one bridge for virtual network cards, a simple configuration would be to list one association like so:
association_0: default; AA; 1.0.0.0/16
Note that no bridge name is given, the default bridge as
determined by Xen will be used. Note the IP range is fake;
if you want to not keep track of workspace IP assignments (to
prevent conflicts) or to not check if a requested IP setting
is valid for the bridge, set the
check_ip_ranges,
track_MAC_assignments, and
track_IP_assignments settings to 'false'.
To actually enact networking changes (using the Allocate networking method for example), the VM must set its own L3 networking information (IP address, default gateway, etc) from inside the VM. Currently we only support delivery of the information via DHCP. Booting into DHCP client mode is well supported in virtually every operating system in existence. Previously we passed information via kernel parameters which required a special understanding inside the VM. The result of using DHCP is that workspace images are easier to create and easier to maintain.
A DHCP server is required to run on each hypervisor node. The purpose of this server is to respond to broadcast requests from workspace's that are booting locally. Before starting a VM, if any of its NICs need to be configured via DHCP, workspace-control will call out to "dhcp-config.sh" via sudo, passing it a specific MAC address to IP mapping (as well as other information to be passed to the VM such as hostname, dns servers, broadcast address, default gateway, etc).
In addition to a DHCP server, we also insert custom ebtables rules when the workspace is deployed. These rules accomplish three things:
Broadcasted DHCP requests are only permitted to be bridged to the correct virtual interface (configured in workspace-control's configuration file, see the workspace-control networking configuration section.
A draft of the workspace DHCP design document is available here.
Configuring the DHCP server consists of copying the example DHCP file "dhcp.conf.example" to "/etc/dhcp/dhcpd.conf" and editing it to include the proper subnet lines (see the contents of the example file). The subnet lines are necessary to get the DHCP server to listen on the interface in the first place. No lease configurations, available ranges, etc. should be added: these are added dynamically to the file after the token at the bottom.
In most cases it is unecessary, but if you have a non-standard DHCP configuration you may need to look at the "dhcp-config.sh" script in the protected workspace bin directory and look at the "adjust as necessary" section. The assumptions made are as follows:
To test the workspace service, you can use the sample metadata and deployment files and a test image.
For a walkthrough and explanations, see the Workspace User Guide.
Any questions can be posted to the workspace-user mailing list and will likely be answered promptly by a member of the community. For instructions on how to subscribe and post messages to this list, see this web page.
Problem: Sometimes, from the start of the workspace's deployment, one of the VM's NICs is unreachable. (specifically, the ARP protocol does not resolve the IP address to a MAC address)
Solution: Make sure the MAC address prefix is valid. See the backend networking configuration section for more details and the right setting to use.