Virtual Workspaces - VWS

Changelog

TP1.3.3.1

• Introduction of support for contextualization with virtual clusters. See the clouds page and the new one-click clusters page to see the various new features in action.

• New ensemble service report operation allows efficient queries about a large number of workspaces.

• Support for storing images at the repository in gzip format and retrieving them from the repository in gzip format. This can save a lot of time in cluster situations.

• Support for pegging the number of vcpus clients receive.

• Various client enhancements including internal organization, cleaner output, and new commandline options. Embedded security tools (like grid-proxy-init) work more out of the box now.

• No configuration migrations are necessary for moving to this version from TP1.3.2. Some configuration additions will be necessary if you'd like to take advantage of features.

• There was a WSDL update: additions, changes and new namespaces. The base namespace for workspace schemas is now http://www.globus.org/2008/06/workspace/

• Some bug fixes.

• Integration with context broker.

• New ensemble service report operation allows efficient queries about a large number of workspaces. Can retrieve status and error messages about entire ensemble at once.

• Fixed scheduler backout to correctly handle situation where ensemble wasn't launched yet but ensemble-destroy was invoked.

• Fixed bug where IP address updates were not passing through cache layer to DB correctly causing a possible inconsistency if container restarted in certain circumstances. NOTE: this bugfix was not present in TP1.3.3 but is present in TP1.3.3.1.

• Various internal changes (see CVS log)

• No configuration changes are necessary for moving to this version from TP1.3.2. But to enable the context broker, you need to configure paths to a credential for it in the jndi-config file and make sure the WSDD file lists the context broker as in the source file "deploy-server.wsdd" (which becomes server-config.wsdd)

• Added cloud-client cluster and contextualization support. Includes new "--cluster" flag (see cloud-client CHANGES.txt for full changes there).

  See the clouds page and the new clusters page.

• The regular commandline client has new flags for ensemble and context broker support. See "-h" output.

• Support for gzip via filename-sense. See cloud notes on image compression/decompression. This can save a lot of time in cluster launch situations since the gzip/gunzip happens on the VMMs simultaneously, cutting transfer times (where there is contention) considerably.

• Local-locked the control of dhcpd start and stop: now works for situations where multiple workspaces are deployed on a VMM simultaneously (such as one VM per core and launching as part of a cluster). The DHCP adjustment was being excercised simultaneously, revealing the race.

• There is no need to change the workspace-control configuration file from a TP1.3.2 compatible one. There is a new configuration if you want to use it, though. The "[behavior] --> num_cpu_per_vm" configuration allows you to peg the number of vcpus that are assigned to every workspace.

  You can not upgrade workspace-control at all if you don't want the features listed here.

• No changes.

TP1.3.2

• Introduction of the cloud configuration and cloud client for user friendly client access to the workspace service.

• Introduction of the "groupauthz" authorization plugin for typical configurations including the cloud setup.

• Clients may now send customization tasks with request, files on the image will be replaced with the content. The cloud client, for example, is set up by default to send a customization request that sets up the workspace's "/root/.ssh/authorized_keys" file.

• Clients can request an alternate unpropagation target to save a template VM into a new personal copy. This new URL may be requested both at creation time and on the fly in a unpropagate request.

• Centralization of MAC address allocations to the central workspace service. This allows all backend configurations files to be identical. Older/advanced configurations are still possible but not recommended unless necessary.

• Hard disk images are now supported (client may bring a matching kernel along).

• Various client enhancements including internal organization, cleaner output, and new commandline options.

• A few bug fixes.

• There was a WSDL update: additions, changes and new namespaces. The base namespace for workspace schemas is now http://www.globus.org/2008/03/workspace/

• See the Cloud Guide for an overview of a new set of configurations/conventions that allow for clients to get up and running in minutes even from laptops on NATs. Currently this comes at the cost of obscuring some features like group deployments and multiple NICs.

• Centralized MAC address allocations to the workspace service. This allows all backend configurations files to be identical. Older/advanced configurations are still possible but not recommended unless necessary.

  There is a new configuration in the jndi-config.xml file that allows the administrator to define the valid prefix for MAC address selection. See WorkspaceFactoryService -> NetworkAdapter -> macPrefix

  Once an IP is assigned a MAC address (during service initialization) it remains with that IP as long as it is configured as part of the network pools. This ensures that local network devices can cache MAC/IP bindings without needing to be manually cleared (no need for unsolicited ARP reply to guarantee connectivity).

• Introduction of the "groupauthz" plugin. This comes directly with the workspace service (no separate plugin installation is necessary) but it is not enabled by default. This authorization plugin supports different policies for different group members which you organize by inserting identities into different group files.

  The plugin can enforce the following policies. The request data to check is determined on a per-request, per-client basis. The limits are defined on a per group basis (every caller identity must be a part of a group).

  • Maximum currently reserved minutes at one point in time. If the caller has two other workspaces with 10 hours scheduled for each, the value being checked against this policy would be 20 hours plus whatever time the current request is.
  • Maximum elapsed and currently reserved minutes at one point in time. If the caller has one other workspace with 10 hours scheduled and 80 hours of recorded past usage, the value being checked against this policy would be 90 hours plus whatever time the current request is. This is the all-time maximum usage cap.
  • Maximum number of running workspaces at one point in time.
  • Maximum number of workspaces per request (the largest group request possible).
  • The image node that must be specified.
  • The image node base directory that must be specified.
  • Support for identity-hash based image subdirectories (see the cloud setup documentation to understand this convention).

  Each policy can be set to disabled/infinite for specific groups if you desire.

• Arbitrary file customization tasks may be sent with the workspace creation request. The image is mounted on the VMM and the contents of the task are placed into the specified file.

  This requires mount-alter.sh support on the backend which expects the mount -o loop construct to work without specific filesystem selection. i.e., this will not support workspaces with filesystems that the VMM kernels do not support.

  This requires three new jndi-config.xml configurations:

  • WorkspaceService -> home -> localTempDirectory
  • WorkspaceService -> home -> scpPath
  • WorkspaceService -> home -> backendTempDirectory

• Inclusion of alternate unpropagation URL. This allows the client to specify the target URL for where the workspace is unpropagated. It can be specified as part of the creation request or overriden after deployment. If the default shutdown mechanism was to destroy the workspace, this can still be used (with shutdown-save) to cause unpropagation to the given URL.

• Authorization enhancement to support late-specified alternate unpropagation URL. An operation to check the contents of a post-deployment alternate propagation URL request was added to the authorization callout interface.

  This can be used to filter out invalid requests. For example, the groupauthz plugin discussed above will use the same logic here for image repository policy checking that it does at create time. Previously, the authorization callout had only one operation which was called at creation time only.

• Fault information can now be stored as part of the Corrupted state (for both RP queries and asynchronous state notifications). This will help the remote client debug issues that can arise after a successful factory creation, such as "the file you specified to propagate does not exist at the image repository" etc.

• Various internal changes (see CVS log)

• See the end of the administrator guide for notes on configuration migration to this version from older workspace releases.

• Introduction of cloud-client system. This consists of a wrapper program run from a specific directory setup that contains an embedded globus client installation among other things.

  For more information on the client and setting up a configuration to support it, see the Cloud Guide. To see some examples of end-user commands, see the clouds page.

• The main client's help system was reorganized. For help on options that are specific to an action, use "--help --<name of action>". See the main "--help" output to get started.

• The main client has a new "--exit-state" option that causes modes with subscriptions (in either poll or async mode) to wait for the specified state before exiting with success. If the workspace moves to a terminal state (Corrupted etc.) then this is considered an error. This is aimed at making scripts that wrap the client more effective.

• The main client has a new "--save-target" option whose argument is an override to any previous unpropagation URL. You can use this before or after deployment has succeeded (although it could fail because of authorization issues). See the client's "-h --shutdown-save" output for more information.

• Arbitrary customization tasks are possible by defining them in an optional parameters file. But the main client now also includes a shortcut for the very common task of inserting your SSH public key as the desired contents of the /root/.ssh/authorized_keys file on the VM. See the client's "-h --deploy" output for more information on this new "--sshfile" option.

• Support for post-deployment error printing (faults can now be included as part of Corrupted notifications).

• Status client allows for a bulk query ("in one remote operation, show me a short update of all workspaces I manage at this service").

• Introduction of a base client API which abstracts operations out from the webservices implementation and provides common subscription tools, utility methods, etc. (the main workspace client was internally reorganized to use this API: if you are a client developer you could examine this code for a lot of concrete usage samples).

• (re-)inclusion of mount-alter for file customization tasks. Using this requires an additional sudo rule.

• Fix for a bug where certain NIC bridging problems with a workspace that had more than one NIC would not trip a backout.

• Fix for a bug where the lack of a gateway specification would cause a problem when inserting a workspace's DHCP policy. Lack of a default gateway is legal (and sometimes necessary).

• When DHCP configuration file cannot be found, a more helpful error is printed.

• Files on VMM were not being deleted in one unpropagate situation where they should have been.

• The VM name prefix sent to the VMM has been shortened from "workspace" to "wrksp". String length limits for NIC names were being reached too early ("wrksp" should accomodate workspace IDs in the millions).

• We are including a "foreign-subnet" script that allows VMMs to deliver IP information over DHCP to workspaces even if the VMM itself does not have a presence on the target IP's subnet. This is an advanced configuration, you should read through the script's leading comments and make sure to clear up any questions before using.

  This is particularly useful for hosting workspaces with public IPs where the VMMs themselves do not have public IPs. This is because it does not require a unique interface alias for each VMM (public IPs are often scarce resources).

• Added support for booting hard disk images (pygrub). Resolves enhancement request #5423. Client must specify mountpoint like "hda" instead of "hda1" for this to trigger.

• See the end of the administrator guide for notes on configuration migration to this version from older workspace releases.

• In some situations the sleep() system call that the pilot makes during an unexpected backout situation was returning too early. This syscall been replaced by an alternate implementation that will not fail in those situations.

TP1.3.1

• Added support for workspace pilot resource management. The pilot is a program the service will submit to a local site resource manager in order to obtain time on the VMM nodes. When not allocated to the workspace service, these nodes will be used for jobs as normal (the jobs run in normal system accounts in Xen domain 0 with no guest VMs running). See below.

• Added functionality to ensure multiple workspaces (including groups of workspaces) are co-scheduled. See below.

• Various client enhancements including ensemble service support, cleaner output, and new commandline options.

• Various bug fixes.

• There was a WSDL update: additions, changes and new namespaces.

• Added support for workspace pilot resource management. The pilot is a program the service will submit to a local site resource manager in order to obtain time on the VMM nodes. When not allocated to the workspace service, these nodes will be used for jobs as normal (the jobs run in normal system accounts in Xen domain 0 with no guest VMs running).

  Several extra safeguards have been added to make sure the node is returned from VM hosting mode at the proper time, including support for:

  • the workspace service being down or malfunctioning
  • LRM preemption (including deliberate LRM job cancellation)
  • node reboot/shutdown

  Also included is a one-command "kill 9" facility for administrators as a "worst case scenario" contingency.

  Using the pilot is optional. By default the service does not operate with it, the service instead directly manages the nodes it is configured to manage.

• Added functionality to ensure multiple workspaces (including groups of workspaces) are co-scheduled. This includes the introduction of the Workspace Ensemble Service. This functionality allows complex virtual clusters to have all its component workspaces be scheduled to run at once if that is necessary. This works with both the default and pilot-based resource managers.

• All remote interfaces (WSDLs/schemas) have been updated with at least new namespaces. You can examine them directly online at the WSDL and XSD files page (or read the descriptions on the Interfaces section). The main difference is an extension to the factory create/deploy operation and the addition of the ensemble service.

• SSH based workspace-control invocations may now be configured with an alternate private key.

• SSH based workspace-control invocations now use options to ensure easier identification of misconfigurations (no password entry hang is possible now).

• If using the pilot mechanisms, a new configuration section in the service configuration file needs to be uncommented for pilot specific configurations (see the configuration comments there).

• If using the pilot mechanisms, a client may now not submit a flag to the factory that requests the workspace be unpropagated after the running time has elapsed. Instead, unpropagation must be triggered manually by a client before this deadline is reached.

• If using the pilot mechanisms, a shared secret must be configured in etc/workspace_service/pilot/users.properties for HTTP digest access authentication based notifications from the pilot. Use the included shared-secret-suggestion.py script. (alternatively SSH may be used for notifications but it is slower)

• New dependencies (these are distributed with the service):

  • backport-util-concurrent
  • jetty - only necessary if using the pilot with the faster, default HTTP digest access authentication based notifications.

• Some platforms+JVMs have buffer size issues which caused some workspace-control invocations to fail. This problem is addressed.

• DHCP based network delivery to the VMs now requires unique hostnames for each allocatable address (even if they do not resolve to an IP). This addresses Bug #5738.

• A new client workspace-ensemble allows you to destroy all workspaces in a running ensemble as well as trigger the workspaces in the ensemble to be co-scheduled and (afterwards) allowed to launch. This trigger is also available in the last workspace deployment of the ensemble, if desirable (this will save a web services operation).

• Enhancement Bug #5795 is addressed, this allows an early unpropagate request to be sent. The new workspace action is "--shutdown-save" and requires a single or group workspace EPR.

• The workspace program includes a new flag "--trash-at-shutdown" which allows callers to include a request that the service simply discards the VM after use (instead of unpropagating it). This is typical behavior for virtual cluster compute nodes, for example. The functionality itself is not new in this release, just this flag. It allows you to include the flag when using commandline based resource requests as well as override a given resource request file with a trash-at-shutdown flag.

• The workspace program has improved output, especially in the cases where you are launching groups and ensembles.

• Note: a previously used TP1.2.3 or TP1.3 configuration file for workspace-control will still work because of the nature of these changes. See this migration section of the administrator's guide for details.

• A bug with failed propagations has been addressed: Bug #5681.

• Will now support older ISC DHCP versions (v2 servers). See Bug #5470.

• The defaults paths for ebtables and the dhcpd.conf file are now the more common occurences:

  • /sbin/ebtables is now /usr/sbin/ebtables
  • /etc/dhcp/dhcpd.conf is now /etc/dhcpd.conf

• This is a new tarball on the download page and is only necessary when using pilot based resource management.

TP1.3

• There was a WSDL update, changes and new namespaces.

• Functionality to start multiple workspaces in one request was added, including introduction of the Workspace Group Service.

• Optional accounting functionality was added, including introduction of the Workspace Status Service.

• Configuration enhancements to make service administration easier.

• Various client enhancements including group and status service support, reorganized help output, and new commandline options.

• Various bug fixes.

• All remote interfaces, WSDLs/schemas, have been updated and also have new namespaces. You can examine them directly online at the WSDL and XSD files page (or read the descriptions on the Interfaces section).

• The Workspace Factory Service was extended to support starting a homogenous group of workspaces in one deployment request. A global maximum group size can be specified natively (without needing to use an authorization callout).

• The Workspace Group Service was added to manage groups after deployment. See the group overview on the main interfaces page.

• Hooks for accounting modules were added. These plugins allow you to track clients' used or reserved running time. There are separate reader and writer interfaces for flexibility. A default database backed implementation is provided and enabled by default. By default this implementation includes a periodic write to log files on the system (one for current reservations, another for major events). See Bug 5443.

• The Workspace Status Service was added, it allows a Grid client to consult the usage statistics that the service has tracked about it. See Bug 5444.

• Some configurations have been added, changed name or changed location in the JNDI configuration file, see this migration section of the administrator's guide for details.

• Resource selection now favors VMMs not in use. The previous selection process accepted the first VMM with enough memory which could result in a situation where e.g. two workspaces are running on one VMM but no workspaces are running on another.

• Resource pool configurations can now be adjusted without resetting the database, see this migration section of the administrator's guide for details.

• Networking address pool configurations can now be adjusted without resetting the database, see this migration section of the administrator's guide for details.

• Resolved Bug 5441: Add functionality for late network binding to client and service.

• Resolved Bug 5442: Move persistence information to its own subdirectory. All information is not stored under $GLOBUS_LOCATION/var/workspace_service/ instead of various subdirectories of $GLOBUS_LOCATION/var itself.

• Host certificate transfer functionality was removed. The association configuration and WSDL has changed accordingly.

• Resolved Bug 5415: WorkspacePersistenceDB not updated after workspace --shutdown

• Resolved Bug 5345: resource not destroyed correctly when time expires and shutdown method is "trash"

• Asynchronous notifications from workspace-control (propagation events) are handled more reliably.

• The toplevel build file includes many new convenience targets, including more control over what is deployed/undeployed and more control over the different kinds of persistence information.

• The build files now do not proceed if your JDK is an earlier version than 1.4.

• The help system was organized, run the client with "-h" to see the definitive list and explanation of features old and new.

• The client can subscribe and listen to many workspaces at a time after deploying a group. As this can be quite verbose for large groups, there are two new options to control subscription output verbosity. See the "-h" text.

• There is a numnodes argument that will control how many workspaces will be requested during the create operation. If there is a NodeNumber element in a given deployment request file, this argument will override that. For more about group support, see the Interfaces section.

• The client can now run management commands using both regular and group workspace EPRs (it looks at which it is dealing with).

• Resolved Bug 5441: Add functionality for late network binding to client and service. In the default case where subscriptions are desired, the client will notice if networking is missing and requery for it when the workspace(s) move to the Running state.

• Resolved Bug 5445: various reference client improvements.

• There is a new workspace-status client for querying accounting information. See Bug 5444.

• The sample XML (metadata, resource request, etc) files included with the client have been updated and more samples have been added.

• The client build now checks that the sample XML (metadata, resource request, etc) files validate against their respective schemas. If your ant installation does not include the xmlvalidate task, these checks are skipped.

• Note: a previously used TP1.2.3 configuration file for workspace-control will still work because of the nature of these changes. See this migration section of the administrator's guide for details.

• Resolved Bug 5360: destroy log shows dhcp/ebtables backout problem

• install.py handles user groups better and has an improved --onlyverify mode

• Removed unecessary configurations from sample worksp.conf file.

• ebtables-config.sh rule backout handles an additional corner case

• JNDI class discovery is done differently, this may affect you if you have alternate implementations of any module or plugin interface. A new workspace Initializable interface can be used. See the org.globus.workspace.Locator class.

• Message intake and initial validation support is now implemented as a plugin, see the org.globus.workspace.service.binding.BindingAdapter interface.

• The default scheduler's "node picking" support is now implemented as a plugin, see the org.globus.workspace.scheduler.defaults.SlotManagement interface.

• AllocateAndConfigure (association) support is now implemented as a plugin, see the org.globus.workspace.network.AssociationAdapter interface.

• New optional AccountingEventAdapter and AccountingReaderAdapter plugins, see the org.globus.workspace.accounting package.

• The optional creation-time authorization callout interface was altered to include group requests as well as the caller's accrued used and reserved running minutes (if an accounting reader is running).

TP1.2.3

• Significant documentation updates including the addition of a guided User Quickstart and the Workspace Marketplace.

• Added the ability to specify multiple partitions for one VM. There is a restriction in this version that only one partition file may be used with the propagation mechanisms, the other partitions must be cached or on a shared filesystem. (Bug 5216)

• Added the ability to create blank partitions on the fly if the client specifies to do so by sending a storage request (the MB of blank space needed) in the resource requirements.

  Currently this hardcodes the filesystem to create on the blank partition (the default is ext2), in the future this may be specifiable by the client. (Bug 5215)

• Added an HTTP transfer adapter for pre- and post-deployment staging. Included is the ability to provide checksums that will be checked after the transfer as well as decompression functionality. For more details, see the Optional parameters documentation. (Bug 5219)

• Added the ability to choose hypervisors in the resource pool based on what networking associations they support. For example, a request may arrive for a workspace to have NICs on two separate networks: the pool node selection algorithm will use the requirement to support both of these networks in its search. (Bug 5214)

• The workspace types schema, workspace_types.xsd, has a new namespace: the "2006/08" part of it is now "2007/03".

• Resolved Bug 5211: networking allocations were not backed out (returned to pool) under all error conditions during initial request processing.

• Resolved Bug 5212: queries on the Workspace Factory resource properties gave incorrect asssocaition information after a container restart.

• Resolved Bug 5213: the Advisory IP acquisition method was being incorrectly validated.

• Resolved Bug 5217: the workspace-control program was not backing out DHCP policy additions under all error conditions.

TP1.2.2

• Added support for DHCP delivery of networking information. See the administrator guide DHCP overview and configuration section which also includes a link to a design document.

• Added unit tests under "workspace-service/service/java/test/".

• Streamlined the logistics section of metadata, see the logistics section of the interfaces guide for more information.

• Small bugfixes in StateTransition.

• Internal refactoring to better accomodate unit tests.

TP1.2.1

• Resolved Bug 4792 (propagation via globus-url-copy adds extra file URL scheme)

• Resolved Bug 4793 (xenlocal arg parsing error)

• Resolved Bug 4879 (issue with database jars that were already installed)

• Resolved Bug 4880 (extra semicolons being sent in network information)

• Fixed client build invocation (WS stubs weren't deployed by default)

• Minor internal refactoring

TP1.2

• Added support for a resource pool model that allows one grid service to manage a large group of VMMs, sending incoming workspace deployment requests to appropriate VMM nodes for instantiation.

• To support the resource pool model, managed file propagation support was added to move files associated with workspaces to and from the resource pool nodes and storage nodes. The current choices are GridFTP and SCP.

• An optional RFT staging plugin is available to allow a deployment request to include a stage in and/or stage out directive. This is to manage client file movement in the grid context as opposed to the managed, inter-site propagation functionality.

• To support host based authorization (which include a reverse IP check in its algorithm), IP pool entries may now optionally include matching certificate and key pairs that are moved on to the VM when it is allocated a particular networking address.

• New functionality is supported: create-paused, reboot. A choice of default shutdown method when the maximum running time has been reached: normal, trashed.

• Logging choices for both the grid service and workspace-control program have been significantly enhanced.

• The VMM workspace-control program has a new installer that will install the executable and create all of its necessary work directories and will review all directory and file permissions for safety (and correct problems if instructed to).

• The VMM workspace-control program now employs a sudo callout to do its privileged work.

• The VMM workspace-control program has been enhanced to isolate user files from each other and is set up with a safe environment for image altering. A new /opt/workspace hierarchy is the default installation option but it allows for flexible choices.

• The grid service portion has been significantly improved internally for asynchronous event handling, scalability and the ability to replace more of its subsystems with alternate or improved implementations.

TP1.1.1

• Fix for service loading order problem on some JVMs (caused a database not found error). Bug 4602

• Some invocations to backend were missing sudo prefix used for Xen3 support.

• Fixed support for Xen3 networking (Bug 3994).

• Better error reporting for sudo misconfigurations (Bug 4601).

• Fix for backend interface problem when the Allocate networking method was used for multiple NICs.

• Xen3 is now the default sample configuration for service and workspace_control.

TP1.1

• Support for a new, "Allocate" networking method that allows the workspace service administrator to specify pools of IP addresses (and DNS information) which are then assigned to virtual machines on deployment.

• The resource properties have been extended to publish deployment information about a workspace, such as its IP address.

• Workspace metadata validation has been extended to support requirement checking for specific architecture, hypervisor version, and CPU. The workspace factory advertises the supported qualities as a resource property; the requirement section of workspace metadata is checked against the supported set.

• Authorization handling has been significantly extended. The workspace service can now accept and process VOMS credentials and SAML attributes (GridShib). Further, an authorization callout has been added to the service for fine grain policies. This callout can be configured to implementations of a simple attribute list lookup or a python script allowing for arbitrary authorization logic.

• Support for Xen3 has been added.

• The workspace client has been extended to accomodate new functionality. In addition the client interface has been extended to enable subscribing for notifications and specifying the resource allocation information at command-line.

• Installation has been improved -- the client now requires only a minimal installation (as opposed to the full service installation).