Table of Contents
Note
If you find this EDG documentation helpful please consider DONATING! to keep the doc alive and current.
| Maintainer:David Peter Pilgrim |
As a server side process, a job can be queued by means of a user request (by a servlet) or by means of a simple batch file, which can be triggered by an operating system scheduler to provide scheduled jobs - for example a certain job to run a report might be triggered every night at 10PM. Jobs typically send notice of their results via the event mechanism, and system error events are automatically triggered if a problem occurs.
Jobs are submitted to the queue either by:
user interaction (e.g. manually requesting a job)
timed processes (e.g. cron or NT scheduler)
or by other jobs (such as events triggering a data load or report run).
Job control is similar to the same mechanism used so successfully on mainframes for many years. It can be used for any function which takes more time than is practical for an interactive session, such as a complex calculation, a data load from a legacy system, or the running of a group of long reports or analyses.
The basic mechanism involved is a "job queue". A job is queued by creating a new entry in the job queue and marking it as "available". A job can also have parameters associated with it to control it's functions, much like arguments to a method.
There are one or more "job handler" objects running on the server side, waiting for jobs to enter the queue - when they do, a job handler picks up the job & begins to process it - which job handler does this can be controller so that different jobs run on different systems, for performance reasons if so desired. Each job handler may process either a single job or can run a set number of jobs in parallel, and each job can be designated as "single-threaded" (e.g. should not be run in parallel with other jobs) or "multi-threaded".
The job-handler objects can also be implemented as either stateful or stateless Session EJB's, allowing business logic to be developed with all of the advantages of Enterprise JavaBeans.
The Job Handler is a separate thread (which can be run as a thread in your application/servlet server JVM, or as a separate Java application (not a servlet)) that runs on a server system (usually the same system as your servlet or application server, but not necessarily). The Job Handler constantly monitors the Job Queue, and when Jobs are detected it starts the requested job, monitor's it's processing, and sends email notifications when it is complete. This allows your application to request tasks that would take longer than a few moments as a background job.
Jobs are handled in priority order, and a job can be suspended and it's priority changed even once it has begun running. Certain jobs can be set to process at only specified times, deferring process-intensive tasks until off-peak hours if so desired. Jobs frequently trigger event notifications when they are complete or when problems occur.
Expresso Framework has the ability process tasks asynchronously. You might want to a task that takes a considerable amount of time to execute, for example, to search an entire database, to calculate the payroll of all employees in your company, or to index web pages for a search engine. The list of potential long running processes is endless. It does make sense to deploy these long process over the web directly, but rather active them indirectly using a job handler. Expresso Framework has a built-in job handler.
To create simple job you must extend the standard class com.jcorporate.expresso.core.job.Job and implement the method run.
package org.fooey.testapp.job;
import com.jcorporate.expresso.core.job.Job;
import com.jcorporate.expresso.services.dbobj.JobQueue;
import com.jcorporate.expresso.services.dbobj.JobQueueParam;
public class PrimeNumberSearch extends Job {
public PrimeNumberSearch() {
super();
}
/** Return a string for the security admininistration screens. */
public String getTitle() {
return new String("Prime Number Search");
}
/** The run methods should process the long running task */
public void run() {
try {
doCalculation();
}
catch (Exception e) {
finish("Abnormal termination of calculation", e);
}
finish( "Successfully calculation in database:'"+getDBName()+"'" );
}
}
The run() should be very familiar to programmers who have experience of multi-thread Java applications. In fact the Job is a direct sub-class of java.lang.Thread! If you want to send feed back to a user, then you call one of the following finish methods.
finish( String msg ) - this method accepts a message parameter.
finish( String msg, Exception e ) - this method accepts a message parameter, and a second parameter for reporting an exception.
CAVEAT EMPTOR: Make sure that you have already set up a valid email address for sending error messages. This is because asynchronous jobs are launch as separate threads, and all communication back to the user is either through an email or through a log file. If the Expresso Framework cannot send email about the success or failure of a job then the job handler will fail over continuously in version 5.0 at least. You can set up the email address by going to Administration pages and defining a valid email address for the setup code MAILFrom. For example set up the email address to "yourusername@localhost".
To submit a job you need to be familiar with JobQueue objects. Job queue object are database objects, you create them with system privileges like so.
JobQueue jq = new JobQueue(SecuredDBObject.SYSTEM_ACCOUNT);
Once you create a job queue object you must associate it with an user id, the job class, and a status.
jq.setField(JobQueue.FLD_UID, requestContext.getUid()); jq.setField(JobQueue.FLD_JOBCODE, org.fooey.testapp.job.PrimeNumberSearch.class.getName()); jq.setField(JobQueue.FLD_STATUS_CODE, JobQueue.JOB_STATUS_NEW);
As with an DBObject you need to insert into the data store, so likewise with a job queue.
jq.add();
When the job queue object is added to the database, because the status has been set to JobQueue.JOB_STATUS_NEW, then the internal Expresso job handler will not active it immediately. In order to active the job queue then you must set the status to JobQueue.JOB_STATUS_AV AVAILABLE, then the handler will pick it up on its next scheduled loop and execute it.
jq.setField(JobQueue.FLD_STATUS_CODE, JobQueue.JOB_STATUS_AVAILABLE); jq.update();
Why have these two difference status levels? The answer is that job queue may have associated job queue parameters. When you create job queue parameter you need to already have a job queue in existence. Therefore there has be intermediate state between a new creation and activation.
Many jobs need parameters of some sort to execute. For example, the job described above, might want to have prime numbers calculated up to x many digits. You can use the parameters defined in the job to customize your job for specific tasks, and to provide a means to reuse the job code with different parameters.
You can define a job parameters in the constructor of the job if you want to be able to set values via the Job GUI in the admin pages, or retrieve a list of all possible parameter names. The first argument is the real name of the parameter. The second is the 'friendly' name that is displayed by JobQueue UI.
public class PrimeNumberSearch extends Job {
public static final DIGIT_PARAM = "digits";
public PrimeNumberSearch() {
// optional: this allows job and UI to know about param, for setting value via UI,
// or returning via 'getParameterNamesAndDescriptions()'
addParameter(DIGIT_PARAM, "Number of digits in prime to find");
}
.....
}
In the method below, we queue a job just like you would with a normal JobQueueEntry. But we add parameters, name/value pairs, that are associated with the job.
For example:
//Create a new connection and turn off autocommit for a transaction
DBConnection connection = DBConnectionPool.getInstance("default").getConnection();
connection.setAutoCommit(false);
try {
// Create the jobqueue entry and add it as before
JobQueue jq = new JobQueue(SecuredDBObject.SYSTEM_ACCOUNT);
// Set the connection so it will use the same transaction system
// as the other dbobjects
jq.setConnection(connection);
jq.setField(JobQueue.FLD_UID, requestContext.getUid());
jq.setField(JobQueue.FLD_JOBCODE, org.fooey.testapp.job.PrimeNumberSearch.class.getName());
jq.setJobStatus(JobQueue.JOB_STATUS_AVAILABLE);
// jq.setCronParams(-1, -1, -1, -1, -1, -1); // for testing
jq.setField(JobQueue.FLD_SERVERID, 0);
jq.setField(JobQueue.FLD_STATUS_CODE, JobQueue.JOB_STATUS_NEW);
jq.addOrUpdate();
// Now create a JobQueueParameter dbobject
JobQueueParam jobParam = new JobQueueParam(SecuredDBObject.SYSTEM_ACCOUNT);
// Likewise set the connection so it's part of the same transaction
jobParam.setConnection(connection);
// Set the field number based to the job number that we now have from adding the job above
jobParam.setField(JobQueueParam.FLD_JOB_NUMBER, jq.getField(JobQueue.FLD_JOBNUMBER));
// Arbitrary param number... just must be unique for each parameter
jobParam.setField(JobQueueParam.FLD_PARAM_NUMBER,"1");
// Set the parameter code in this case digits
jobParam.setField(JobQueueParam.FLD_PARAM_CODE, PrimeNumberSearch.DIGIT_PARAM);
//Set the digits parameter to 85
jobParam.setField(JobQueueParam.FLD_PARAM_VALUE,"85");
// Now add the object and commit the entire transaction
jobParam.addOrUpdate();
connection.commit();
}
catch (DBException ex) {
// Rollback the transaction if we have an exception
connection.rollback();
}
finally {
// Finally release the entire database connection
if (connection != null) {
connection.release();
}
}
You use the function getJobParameter(name) to access the actual parameters that are passed to the job itself.
public void run() {
try {
String digitParam = (String)getJobParameter(PrimeNumberSearch.DIGIT_PARAM);
if (digitParam == null )
throw new IllegalArgumentException("Digits parameter must be > 0");
int numDigits = Integer.parseInt(digitParam );
if (numDigits <= 0)
throw new IllegalArgumentException("Digits parameter must be integer > 0");
doCalculation(numDigits);
finish( "Successfully calculation in database: '"+getDBName()+"'" );
}
catch (Exception e) {
finish("Abnormal termination of calculation", e);
}
}
Normally the JobHandler is run as a separate thread in your application server's JVM by adding the 'startJobHandler=y' property to your default.properties (or other property) file.
In order to start the Job Handler in a seperate JVM, you must issue a command line on your server system. It is almost always convenient to package up this command line into a batch or shell script file (depending on your operating system), so that it can be run again easily.
An example Job Handler shell script is shown here (the text is all one line in the shell script, but broken into multiple lines here for readability):
/usr/java/jdk1.2.2/bin/java -classpath /usr/orion/orion/orion.jar:/usr/java/lib/mail.jar:/usr/java/lib/activation.jar:/usr/java/lib:/usr/java/jdk1.2.2/lib/tools.jar com.jcorporate.expresso.core.utility.JobHandler configDir=/usr/expresso/config webAppDir=/usr/web-apps/expresso
This example uses the JDK1.2.2 runtime and the servlet API libraries in the Orion Application Server, assumes that "default.properties" is in the specified configDir directory. Running this script produces output to the standard output, so it is recommended that it be run with the nohup command, or that it's output be redirected.
This script can of course be altered for the specific directories and libraries that you are using, and broken into multiple lines if desired.
The Job Handler is ordinarily left running whenever the server is up, so that Jobs receive immediate consideration based on their priorities.
// allocate empty slot for new job
JobQueue oneJob = new JobQueue();
// this job should be executed on 'Unix' systems Only.
// the valid options are: MSWIN, Unix, any or Specific OS
// as returned by System.getProperty("java.os")
oneJob.setJobOSName("Unix");
// execute this job at specified by cron parameters time.
oneJob.setJobCronParams(minute, hour, dayOfMonth, month, dayOfWeek, year);
// set job code to execute
oneJob.setJobCode("com.jcorporate.expresso.utils.TestJob");
// set the owner of this job
oneJob.setUserName("root");
// job is ready for execution
oneJob.setJobStatus(JobQueue.JOB_STATUS_AVAILABLE);
// Add parameters associated with this job (same as before)
// submit job
oneJob.add();
When the JobHandler searches for jobs in the queue, it only picks up jobs that are set to status = JOB_STATUS_AVAILABLE and server id = 0. I believe the server id is a reference to the jobhandler in jobhandlerregistry table. If you have a job you want initiated on every startup, be sure to set server id to zero (0) (which will happen automatically if the Job is created with the standard constructor).
The jobhandler does the following:
Jobhandler looks for jobs in the queue that are serverid=0, code=available and runs them or submits them to crontab, then sets the status to R (running). I think this means that if you change the crontab params after the job is running it won't do anything since it's already been submitted to crontab.
When the crontab daemon thinks it time to run, the job gets run
The job gets set to C for "Complete"
Each job can be executed in a cron-like manner. The ability to start a repeating job through the "Queue Any Job" controller does not exist. Instead you have to manually create the entry with some additional fields filled out. To do this, go to: Applications -> Job Queue Entries. The easiest way is to pick out a job that you have already executed once that you wish to submit to the cron manager.
In the entries parameters enter the following values:
Table 27.1. Job Queue Entry Parameters
| Entry Name | Value |
|---|---|
| Job Serial Number | Leave Alone Since it is auto-assigned |
| Requested By User | The login name of the user that will be requesting the job and who should be notified when the job is completed. |
| Job Code | The classname of the class derived from com.jcorporate.expresso.core.job.Job that will be run when the job executes |
| Status Code | Set to "Available" |
| Job Priority | Best to leave at the default value |
| Handling Server | Set to "0" Job servers will only pick up items that are not associated with any server yet. |
| Required Job OS Name | Leave to any. Although you can set the value to what the JVM gives as the operating system if you have special job needs. For example, if you have a job server on a Linux box, and a job server on the Windows box, and you need the job to use Windows Automation to build an Excel spreadsheet, then you could assign the particular job to require the Windows system identifier. Note, however, that this is an untested feature. |
| Cron Parameters | Enter six comma-delimited integers. The actual values will be set up as defined later. Leave blank if you do not what this job submitted to the Crontab manager |
Click "Add Record" or "Update Record" to save the job to the database. In approximately 30 seconds, the JobHandler will pick up the job. If there are cron parameters associated with it, it will be sent to the crontab for time management.
The setJobCronParams method as was mentioned earlier, should be 6 comma-delimited integer values like so: -1,-1,-1,-1,-1,-1 . Each item corresponds to a different time slot. The meanings are:
Table 27.2. Job Scheduling Parameters
| Parameter | Description |
| minute | the 'minute' number when the cron should be executed. Standard useful values are 0-59. -1 if you want the job to execute every minute. Set to zero if you want the cron to execute at the top of the hour. |
| hour | the 'hour' number when you want the cron to be execute. Values are -1 to 23. -1 if you want the job to execute every hour. 0 if you want the job to execute at midnight. |
| dayOfMonth | Day of the month to execute (-1 if you don't want this parameter considered). The attribute is exclusive of day of week. Allowed values are: 1-31. java.util.Calendar constants can be used. Be careful of 31 when dealing with months that have less than 31 days as unexpected results can occur. |
| month | Which month should the job be run? Values are 0-11 where 0=January, 1=February, etc. Set to -1 if you do not want this parameter considered. |
| dayOfWeek | Exclusive of Day of Month values are: 1-7 where 1=Sunday, 2 = Monday, etc. Set to -1 if you do not want this parameter considered. |
| year | Year of the alarm. Set the year that this should be executed. Set this value if you do not want the alarm to be repetitive, but want it to execute once. (Example: 2003) |
Note
As you may have noticed, the cron parameters here are absolute. The Job queueing API currently does not allow for items such as "Execute every 20 minutes". Instead you would have to create 3 job entries, one that execute at the top of the hour, one at 20 minutes, and one at 40 minutes.
Cron jobs which are specified to repeat have the special quality of resetting themselves to run again if they have already run successfully during the lifetime of the current JobHandler. That is, after the job is done, even though its status is set to C ("complete"), the job will be rerun at the appropriate (repeating) date and time. However, if you restart the server, you need to make sure, during initialization, that any repeating cron job has its status set to "available" and server = 0.
Execute at midnight every day: 0,0,-1,-1,-1,-1
Execute at half past every hour: 30,-1,-1,-1,-1,-1
Execute at December 31, 11:59 every year: 59,23,31,11,-1,-1
Execute at 3:00 a.m., the first of every month: 0,3,1,-1,-1,-1
Execute at June 1, 2:30 a.m., 2006 AD only: 30,2,1,5,-1,2006
Execute every minute: -1,-1,-1,-1,-1,-1
The following persons have contributed their time to this chapter:
Larry Hamel, CodeGuild, Inc. (JGroup Expert)
Peter Pilgrim
Mike Rimov
Mike Traum (JGroup Expert)
Sandra Cann
Note
Was this EDG documentation helpful? Do you wish to express your appreciation for the time expended over years developing the EDG doc? We now accept and appreciate monetary donations. Your support will keep the EDG doc alive and current. Please click the Donate button and enter ANY amount you think the EDG doc is worth. In appreciation of a $35+ donation, we'll give you a subscription service by emailing you notifications of doc updates; and donations $75+ will also receive an Expresso T-shirt. All online donation forms are SSL secured and payment can be made via Credit Card or your Paypal account. Thank you in advance.
Copyright © 2001-2004 Jcorporate Ltd. All rights reserved.