Thrift module: compute_resource_model

ModuleServicesData typesConstants
compute_resource_model BatchQueue
CloudJobSubmission
ComputeResourceDescription
DMType
FileSystems
GlobusJobSubmission
JobManagerCommand
JobSubmissionInterface
JobSubmissionProtocol
LOCALSubmission
MonitorMode
ProviderName
ResourceJobManager
ResourceJobManagerType
SSHJobSubmission
UnicoreJobSubmission

Enumerations

Enumeration: ResourceJobManagerType

* Enumeration of local resource job manager types supported by Airavata
*
* FORK:
*  Forking of commands without any job manager
*
* PBS:
*  Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine.
*
* SLURM:
*  The Simple Linux Utility for Resource Management is a open source workload manager.
 *
 * UGE:
 *  Univa Grid Engine, a variation of PBS implementation.
 *
 * LSF:
 *  IBM Platform Load Sharing Facility is dominantly installed on IBM clusters.
*


FORK0
PBS1
SLURM2
LSF3
UGE4
CLOUD5
AIRAVATA_CUSTOM6

Enumeration: JobManagerCommand

Enumeration of resource job manager commands

SUBMISSION:
 Ex: qsub, sbatch

JOBMONITORING:
 Ex: qstat, squeue

DELETION:
 Ex: qdel, scancel

CHECK_JOB:
 Detailed Status about the Job. Ex: checkjob

SHOW_QUEUE:
 List of Queued Job by the schedular. Ex: showq

SHOW_RESERVATION:
 List all reservations. Ex:showres, show_res

SHOW_START:
 Display the start time of the specified job. Ex: showstart



SUBMISSION0
JOB_MONITORING1
DELETION2
CHECK_JOB3
SHOW_QUEUE4
SHOW_RESERVATION5
SHOW_START6

Enumeration: FileSystems

Enumeration of File Systems on the resource

FORK:
 Forking of commands without any job manager

PBS:
 Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine.

UGE:
 Univa Grid Engine, a variation of PBS implementation.

SLURM:
 The Simple Linux Utility for Resource Management is a open source workload manager.



HOME0
WORK1
LOCALTMP2
SCRATCH3
ARCHIVE4

Enumeration: JobSubmissionProtocol

Enumeration of Airavata supported Job Submission Mechanisms for High Performance Computing Clusters.

SSH:
 Execute remote job submission commands using via secure shell protocol.

GRAM:
 Execute remote jobs via Globus GRAM service.

UNICORE:
 Execute remote jobs via Unicore services



LOCAL0
SSH1
GLOBUS2
UNICORE3
CLOUD4
SSH_FORK5
LOCAL_FORK6

Enumeration: MonitorMode

Monitoring modes

POLL_JOB_MANAGER:
GFac need to pull job status changes.

XSEDE_AMQP_SUBSCRIBE:
Server will publish job status changes to amqp servert.




POLL_JOB_MANAGER0
CLOUD_JOB_MONITOR1
JOB_EMAIL_NOTIFICATION_MONITOR2
XSEDE_AMQP_SUBSCRIBE3
FORK4
LOCAL5

Enumeration: DMType


COMPUTE_RESOURCE0
STORAGE_RESOURCE1

Enumeration: ProviderName

Provider name



EC20
AWSEC21
RACKSPACE2

Data structures

Struct: ResourceJobManager

KeyFieldTypeDescriptionRequirednessDefault value
1resourceJobManagerIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2resourceJobManagerTypeResourceJobManagerTyperequired
3pushMonitoringEndpointstringoptional
4jobManagerBinPathstringoptional
5jobManagerCommandsmap<JobManagerCommand, string>optional
6parallelismPrefixmap<parallelism_model.ApplicationParallelismType, string>optional

Resource Job Manager Information

resourceJobManagerType:
 A typical HPC cluster has a single Job Manager to manage the resources.

pushMonitoringEndpoint:
 If the job manager pushes out state changes to a database or bus, specify the service endpoint.
  Ex: Moab Web Service, Moab MongoDB URL, AMQP (GLUE2) Broker

jobManagerBinPath:
 Path to the Job Manager Installation Binary directory.

jobManagerCommands:
 An enumeration of commonly used manager commands.


Struct: BatchQueue

KeyFieldTypeDescriptionRequirednessDefault value
1queueNamestringrequired
2queueDescriptionstringoptional
3maxRunTimei32optional
4maxNodesi32optional
5maxProcessorsi32optional
6maxJobsInQueuei32optional
7maxMemoryi32optional
8cpuPerNodei32optional
9defaultNodeCounti32optional
10defaultCPUCounti32optional
11defaultWalltimei32optional
12queueSpecificMacrosstringoptional
13isDefaultQueuebooloptional

Batch Queue Information on SuperComputers

maxRunTime:
 Maximum allowed run time in hours.

Struct: LOCALSubmission

KeyFieldTypeDescriptionRequirednessDefault value
1jobSubmissionInterfaceIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2resourceJobManagerResourceJobManagerrequired
3securityProtocoldata_movement_models.SecurityProtocolrequired

Locally Fork Jobs as OS processes

alternativeSSHHostName:
 If the login to ssh is different than the hostname itself, specify it here

sshPort:
 If a non-default port needs to used, specify it.

Struct: SSHJobSubmission

KeyFieldTypeDescriptionRequirednessDefault value
1jobSubmissionInterfaceIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2securityProtocoldata_movement_models.SecurityProtocolrequired
3resourceJobManagerResourceJobManagerrequired
4alternativeSSHHostNamestringoptional
5sshPorti32optional22
6monitorModeMonitorModeoptional
7batchQueueEmailSenderslist<string>optional

Authenticate using Secured Shell

alternativeSSHHostName:
 If the login to ssh is different than the hostname itself, specify it here

sshPort:
 If a non-default port needs to used, specify it.

batchQueueEmailSenders:
 If a resource always sends the monitoring from a specific address, specify the
  full email address. If a resource sends emails from multiple addresses (
   example: based on the submitted login node) then use the wildchar * to indicate
   the same. Example: *@*.example.com or *@example.com


Struct: GlobusJobSubmission

KeyFieldTypeDescriptionRequirednessDefault value
1jobSubmissionInterfaceIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2securityProtocoldata_movement_models.SecurityProtocolrequired
3globusGateKeeperEndPointlist<string>optional

Struct: UnicoreJobSubmission

KeyFieldTypeDescriptionRequirednessDefault value
1jobSubmissionInterfaceIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2securityProtocoldata_movement_models.SecurityProtocolrequired
3unicoreEndPointURLstringrequired

Unicore Job Submission

unicoreEndPointURL:
 unicoreGateway End Point. The provider will query this service to fetch required service end points.
authenticationMode
 The authenticationMode defines the way certificate is fetched.

Struct: CloudJobSubmission

KeyFieldTypeDescriptionRequirednessDefault value
1jobSubmissionInterfaceIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2securityProtocoldata_movement_models.SecurityProtocolrequired
3nodeIdstringrequired
4executableTypestringrequired
5providerNameProviderNamerequired
6userAccountNamestringrequired

Cloud Job Submission



Struct: JobSubmissionInterface

KeyFieldTypeDescriptionRequirednessDefault value
1jobSubmissionInterfaceIdstringrequired
2jobSubmissionProtocolJobSubmissionProtocolrequired
3priorityOrderi32required0

Job Submission Interfaces

jobSubmissionInterfaceId: The Job Submission Interface has to be previously registered and referenced here.

priorityOrder:
 For resources with multiple interfaces, the priority order should be selected.
  Lower the numerical number, higher the priority


Struct: ComputeResourceDescription

KeyFieldTypeDescriptionRequirednessDefault value
1computeResourceIdstringrequired"DO_NOT_SET_AT_CLIENTS"
2hostNamestringrequired
3hostAliaseslist<string>optional
4ipAddresseslist<string>optional
5resourceDescriptionstringoptional
6enabledbooloptional
7batchQueueslist<BatchQueue>optional
8fileSystemsmap<FileSystems, string>optional
9jobSubmissionInterfaceslist<JobSubmissionInterface>optional
10dataMovementInterfaceslist<data_movement_models.DataMovementInterface>optional
11maxMemoryPerNodei32optional
12gatewayUsageReportingbooloptional
13gatewayUsageModuleLoadCommandstringoptional
14gatewayUsageExecutablestringoptional
15cpusPerNodei32optional
16defaultNodeCounti32optional
17defaultCPUCounti32optional
18defaultWalltimei32optional

Computational Resource Description

computeResourceId: Airavata Internal Unique Identifier to distinguish Compute Resource.

hostName:
  Fully Qualified Host Name.

hostAliases:
  Aliases if any.

ipAddress:
  IP Addresses of the Resource.

resourceDescription:
 A user friendly description of the resource.

JobSubmissionProtocols:
 A computational resources may have one or more ways of submitting Jobs. This structure
   will hold all available mechanisms to interact with the resource.
 The key is the priority

DataMovementProtocol:
 Option to specify a prefered data movement mechanism of the available options.

fileSystems:
 Map of file systems type and the path.