00205: Ghostly Tales From the BBx Crypt – A Look at Ghost Tasks in Depth

Title:

Ghostly Tales From the BBx Crypt – A Look at Ghost Tasks in Depth

Description:

A couple of caveats before we start this tale of horror. First, the following article is not really about BBx per se, but rather about the useful capability of Unix/Xenix to run BBx processes in “the background”. Second, and closely tied to the first caveat, using background processes, also known as “ghost” tasks, can get moderately complicated. Moreover, there are lots of variations on the Unix/Xenix basics that we present here. Please read your favorite Xenix/Unix manual thoroughly and do lots of testing. 

It was a dark and stormy night…seriously now, have you ever been devilishly interested in creating your own spiritual tasks, metaphorically speaking of course, that sit off all by themselves with no terminal attached and just work away? Read on, and turn those phantom visions into real code that can work all kinds of magic for you. (Ok, ok, no more supernatural stuff well, maybe one or two more.) 

First, what is a background or “ghost” task? Basically, it’s nothing more than a process that is not directly attached to a terminal. Since Unix/Xenix is primarily an interactive environment, i.e., each process usually has an associated terminal for standard input and output, the notion of a process running with no terminal attached can seem a bit foreign. 

There are two principal reasons why an applications system might benefit from the ability to have its processes run independently of a terminal. One reason is, of course, to do batch processing, the other is to help manage system resources. 

In the first case, picture an online transaction processing system that has a full complement of terminals used for data entry, each with its own BBx task for doing data entry. The circumstances might call for a single BBx process running in the background which collects all of the transactions, does transaction formatting and validation, and then batches the transactions to send over the network. If the data collection process finds an error, it might send the transaction back to the originating terminal with an error message. The data entry processes and the data collection process could interact via shared files or through pipes. 

In the second case, an application system process running in background doesn’t tie up a terminal, thus freeing the terminal for interactive processing while still keeping the task going. Any long running process which would tie up a terminal for an extended period is an immediate candidate for submission as a ghost task. This includes large sorts, complicated and lengthy calculations, long print jobs, complex transaction validation and file updates, and so on. 

Ghost tasks (BBx and otherwise) can be activated either directly from the Unix/Xenix shell or from within a BBx program. There are two ways to start a ghost task from BBx, either by OPENing a ghost device previously defined with an alias in the config.bbx file, or by using the BBx SCALL function. In both cases, what you are really doing is sending a shell command from BBx to the operating system to start another BBx program running. 

Let’s start with the Xenix/Unix shell approach first. This is probably the most common way in which ghost tasks are started. All it takes to start a ghost task running is that the last character of the command line be an ampersand (&). For example, from the shell prompt, the command: 

bbx4 ghost_sort & 

will start BBx and run the program ghost_sort in the background. The & tells the shell to start BBx and then to immediately begin taking commands again from the terminal without waiting for the BBx task to complete. When the command is given, the shell will return the process ID that is assigned to the task. You can use that id to refer to the task with other shell commands. 

A key point to keep in mind is that ghost tasks are tied to the shell process that initiated them. In other words, if you log out, all associated ghost tasks are killed. However, you can use the nohup (which stands for “no hangup”) command to keep ghost tasks alive after you have logged out. The basic form is: 

nohup bbx4 ghost_sort & 

Some other useful commands that can be used with ghost tasks are the nice command which runs the ghost task at a lower than normal priority and the at command which can be used to start the ghost task at a particular time. Also note that some variations of the kill command can be used to terminate the background process. Kill 0, for example, will terminate all processes, background and foreground in a process group. 

When using background tasks, you should generally avoid doing terminal input/output. If job control is enabled on your system and a background task attempts to input or output to a busy device, the process will be blocked. You may have to bring the task to the foreground to recover. See your Unix/Xenix manual for more information. 

Given that ghost tasks run in the background and thus their processing status is not readily apparent, the question of how to keep track of what the process is doing arises. One way is to have the process write status messages to a log file as it goes. (Our BBx example below uses this method.) The status of the ghost can thus be externally monitored by examining the log file. Alternatively, a periodic ps on the process can be used to see if the task has finished. A more flexible and sophisticated mechanism for communicating with a ghost task, however, is through the use of named pipes. 

**** Multi Terminal Illustration *************** 

Named pipes can be used when the need exists for close monitoring of the status or progress of the ghost task or where some periodic control of the task is necessary. In order to use named pipes, you must define the pipes before starting the ghost task. It is not possible to create and attach a named pipe to an already running ghost task. 

So what is a named pipe? Most Xenix/Unix users (and DOS users for that matter) are familiar with pipes, but in their unnamed form. A pipe is really a set of two special files, one for input and one for output, that are connected to the standard input and standard output of a task. What Xenix/Unix shell does when the familiar ‘|’ symbol is encountered in a command string is set up two unnamed files which are connected to the standard input and output of a task 
and through which the standard output and input of another task in the piped command are routed. The pipes are called “unnamed” because no directory entry is created for the files. Since no directory entry exists, no other processes can access the files and thus the pipe. These files are transient and are removed by Xenix/Unix once the command is completed. 

Named pipes must be established a priori using the Xenix/Unix mknod command. Named pipes, in counterpoint to unnamed pipes, have a directory entry, thus allowing other processes to access the pipe. The files associated with a named pipe are permanent, i.e., they must be explicitly removed by the rm command, and are special files of type p. For more in-depth information on pipes, named and otherwise, see the Xenix/Unix manuals for your system. 

For our example, we will define two named pipes, using Xenix syntax, as follows: 

mknod G1IN p 
mknod G1OUT p 

Now the following command will start our ghost task with its named pipes: 
         nohup bbx4 ghost_sort <G1IN >G1OUT & 

The “<” associated with G1IN indicates that it is the input pipe, while the “>” indicates the output pipe. Something to keep in mind when reconnecting to the ghost task when you define the named pipes of the task that you will use to talk to the ghost (call it the foreground task), the output pipe of the foreground task uses the same name as the input pipe of the ghost task. Conversely, the output pipe of the ghost task must be the same name as the input pipe of the foreground task. Also, remember when using named pipes to consider the application and its environment to determine if the pipe requires a unique name. 

The following example illustrates starting a ghost task from BBx using the alias method. Assume that the problem is that there is a large sort which needs to be accomplished. The input data is in a file named indata. We will store a series of status messages in a log file, G1.log, so that we can keep track of the progress of the sort. We will use an MKEYED file, sortin, to actually reorganize the data into the desired logical sequence. Given the nature of the task, we don’t need to communicate with it while it is running, so no named pipes are required. 

The first step is to define the shell command line to create the ghost by establishing a “ghost” device alias in the config.bbx file as follows: 

ALIAS G1 “|BBTERM=G1 nohup bbx4 ghost_sort &” 

The | symbol tells BBx that it will spawn a task from the shell. The “G” is a convention for naming ghost tasks in other Business BASICS. It is not meaningfully standard otherwise. 

When defining BBx ghost tasks, you can define as many ghost devices as you need. However, there is nothing to prohibit the existence of several ghost tasks with the same name, so be careful that your BBx application does not insist on a unique FID(0) for each of the BBx ghost tasks. 

With the ghost device G1 defined in config.bbx, the sort task is started with the following BBx code: 

0010 BEGIN 
0020 OPEN (1) “G1” 
0030 CLOSE (1) 

The OPEN (1) will run the program specified on the alias line for device G1. Note that because of the nohup command in the ALIAS line, the CLOSE statement will not kill the process. Also note that if the start-up program is run again, it will generate a new process. It does not reconnect to the previously started process. Reconnection with a running ghost task is accomplished through the use of named pipes, which we previously discussed. 

The “ghost_sort” code is shown below. As can be seen, there is nothing particularly special about it other than it doesn’t expect any I/O using channel 0. 

0010 REM “GHOST_SORT – PROGRAM THAT BUILDS A SORT FILE AS A GHOST 
0020 BEGIN 
0030 LET RECS=0; REM “COUNT NUMBER OF RECORDS WRITTEN TO SORT 
0100 REM “OPEN FILES-1:INDATA-FILE TO BE SORTED,2:G1.LOG-GHOST LOG FILE 
0110 OPEN (1)”INDATA” 
0120 OPEN (2)”G1.LOG” 
0130 REM “(3) SORTIN – DEFINED @500’S 
0200 REM “LOG GHOST STARTED 
0210 PRINT (2)”GHOST STARTED – “+DATE(0:”%Mz/%Dz/%Yz%hz:%mz:%sz%p”) 
0500 REM 500″DEFINE SORT FILE 
0505 PRINT (2)”CREATING SORT FILE-“+DATE(0:”%Mz/%Dz/%Yz %hz:%mz:%sz %p”) 
0510 ERASE “SORTIN”,ERR=0520 
0520 MKEYED “SORTIN”,25,0,0 
0530 OPEN (3)”SORTIN” 
1000 REM 1000 “FILL SORT FILE 
1010 READ (1,END=9000)A1$,B1$ 
1020 LET KEY3$=B1$+A1$ 
1030 WRITE (3,KEY=KEY3$) 
1035 LET RECS=RECS+1 
1040 GOTO 1010 
9000 REM 9000 “FINI 
9010 PRINT (2)”SORT COMPLETED-“,RECS,”RECORDS 
WRITTEN”+DATE(0:”%Mz/%Dz/%Yz%hz:%mz:%sz %p”) 
9020 PRINT (2)”GHOST TASK COMPLETED” 
9030 END 

As mentioned earlier, a ghost task can also be activated by doing a system call, as in the following BBx code. 

10 BEGIN 
20 OKAY=SCALL(“BBTERM=IO nohup bbx4 program &”) 
. 
. 
. 

Most Unix systems have an environment variable ‘!’. The purpose of this variable is to return the last ghost tasks process ID number. With this variable, one can create a semi-intelligent ‘Ghost control’ program. The ‘!’ is set to the ghost task process ID number on a successful invocation of ghost task for the shell that created the ghost task. For example, if you start a ghost task from BBx with the SCALL command, BBx does an ‘exec sh’ plus the command line passed from SCALL. To retrieve the value of the ‘!’ variable, you must ‘echo’ the ‘!’ variable to a temporary file as part of the SCALL command line. After the return to BBx, you can READ the temporary file and extract the process ID of the ghost that was started. For example: 

10 LET A=SCALL(bbx4 program & echo $!>>/tmp/ghostlog”) 
20 OPEN (1)”/tmp/ghostlog”;READ (1)G1_ID 

You can now use the G1_ID variable to determine if the ghost is active or to terminate the ghost as needed. 

The above discussion about pipes, etc., applies here also. Remember not to combine the alias method and the SCALL method for the same task. Otherwise, you can end up with two child tasks running in background both with the same name, although with different process IDs, and both off doing the same thing with the same data potentially, an ugly situation. When using the SCALL method, it is a good idea to use BBTERM=IO rather than, say BBTERM=G1. When a BBx task is running in background, the file name associated with channel 0 is IO. Note that IO has no special capabilities other than to read and write data. 

We hope our little adventure hasn’t been too spooky. Obviously, ghost tasks are not a novice level project, but with care they can be extremely useful. There are many ways to approach and handle ghosts and we have only scratched the surface here. We encourage you to experiment and consider adapting some of your applications (sorts, updates, and long calculations are the prime candidates) to background processing. Happy ghosting. 

********************************************************************************************************** 

Common Questions About Ghost Tasks 

Q. If I want to start a BBx program as a daemon, how do I specify its FID? 

A. I’m not sure if a BBx program can be started as a daemon in the classic sense. BBx can address pipes if the operating system supports them. In the classic Business BASIC syntax, this is known as a ghost task. 

One way to use ghost tasking is to setup an alias in the config.bbx file that reads: 
        alias G0 “|BBTERM=G0 bbx4 -c/usr/bbx/config.bbx” 
and then treat it as a normal device. BBx can also be started as a background process in the normal way, i.e.: 
        bbx4 SOMEPGM </dev/null >/tmp/log.SOMEPGM & 
or, as a “ghost task” by placing “BBTERM=Gxx” in front of the “bbx4” on the same line. Failure to define the FID(0) will make BBx assign “IO” for any process not attached to a terminal on its standard input & output. 



Q. I am starting a background task to talk to a serial port that is connected to a modem. We started the tasks using BBTERM=Txx. Is the Gxx significant? I tried an earlier suggestion of setting up the tty types for the port I am communicating with and it seem to help even though we are opening up the raw device /dev/ttyh2. 

A. Yes, the Txx is significant. It tells BBx to handle processing for a VDT. Since a ghost isn’t a VDT, you want BBTERM set to anything *except* what you’ve set it to. I’d try starting BBx using something like this: 
        BBTERM=IO bbx4 </dev/ttyxxx >/dev/ttyxxx & 

That would give you a FID(0)=”IO” but a process connected to the tty that you want to talk to. Be aware that you might have problems if the tty doesn’t let you startup because of a lack of modem signals, etc. If you aren’t using the tty for raw processing but instead want BBx automatically started on the tty as a real terminal/process there’s a *lot* more to be done. 

Last Modified: 12/22/1997 Product: PRO/5 Operating System: Unix

BASIS structures five components of their technology into the BBx Generations.