| TOP | UP: Applied Frontier | NEXT: NetFrontier |
Frontier speaks and listens across a TCP/IP network, such as the Internet, by way of NetEvents, a scriptable application with a very small RAM footprint. The glue verbs for NetEvents are at system.apps.verbs.netEvents . The user can call any of these directly; there are also higher-level verbs that hide the details, and this chapter concentrates mostly on the latter.
NetEvents basically just manages TCP/IP connections, referred to as streams. One may picture streams as analogous to telephone calls: NetEvents is like a telephone operator, managing a bank of phones. It can open a stream (like calling a server elsewhere on the network), write to a stream (speak over the phone), read from a stream (hear over the phone), and close a stream (hang up the phone). It can also listen for an incoming stream, which means that clients elsewhere can call it on the phone. And it can perform a few utility functions, such as obtaining domain name resolution.
However, NetEvents knows nothing of what to say over the phone, or how to interpret what it hears; that's up to your scripts. The higher-level verbs described in this chapter do all the hard work. Thanks to them, Frontier and NetEvents can easily be made to function as either a client or a server on a TCP/IP network.
A server is an application which receives and obeys commands over a network. To continue our telephone conversation analogy: with TCP/IP, a computer's IP address is its phone number. A server is assigned to a particular port, which is like an extension at that phone number; one computer might have several servers running, so when a client program phones in, it asks for the particular port whose server performs the desired function. By convention, a few standard server types are assigned standard port numbers; other port numbers are free for custom servers.
A server doesn't want clients to get a busy signal, so it implements enough phones on its port so that at least one will pretty surely be free at any given moment. It is then up to the server to manage talking on all the phones at once if need be. This can be a challenge, but Frontier and NetEvents are multithreaded, so they are up to the job if the scale is reasonably contained.1
So, for Frontier to act as a server, it first tells NetEvents to implement a bank of phones and to do the actual "listening" to see if there's a call. When a call comes in, NetEvents patches it through one of the phones and hands the phone to Frontier. Frontier must know how to be a good server on this port: it will be expecting some particular command or commands, and it should obey and reply in whatever the proper manner may be. When the conversation is over, Frontier tells NetEvents to hang up the phone.
Frontier's basic server functionality is implemented through the inetd suite. The name "inetd" comes from the UNIX command which sets up a server, or "Internet daemon." A simple menu interface to the suite appears as the Inetd menu when you choose Internet Daemon from the Suites menu.
To use the inetd suite, you first describe your server as a table (the "daemon table"); this table will usually be an entry in user.inetd.config, but, as we shall see, it doesn't have to be. The daemon table must have three entries:
The TCP/IP port number for your server: if a client phones up asking for this port, your daemon server script will be called.
The number of listeners you want to open on this port (i.e., the size of your server's phone bank). This will be the number of listening threads that Frontier will spawn, and represents the maximum number of clients your server will be able to respond to simultaneously.
The server script which will be called when a client phones up.
The daemon script should take one parameter; this parameter will be the address of a table. The table will contain an entry stream, the stream ID identifying the particular client who has just phoned up; you will need stream in order to speak with this client by way of NetEvents. The table also contains an entry client, the IP address of the client, encoded; you can decode this, if the information is needed, by handing it to netEvents.nameToAddress() . Once your script is called, it should find out what the client is saying, by calling netEvents.readStream() , and then reply, by calling netEvents.writeStream() . Your script does not close the connection; that is handled automatically when your script returns.
To start your server running, call either inetd.startOne() or inetd.start(). inetd.startOne() takes as parameter the address of your daemon table. inetd.start() takes no parameters: it creates listeners for all daemon tables in user.inetd.config. (Start Daemons in the Inetd menu does the same thing.) These verbs call a utility, inetd.supervisor() , in a new thread, count times for each daemon. Each supervisor thread calls netEvents.listenStream() to create a listener (a "phone"), and then loops waiting for a client to appear on that listener. If a client does appear, the supervisor thread calls the daemon script. The daemon script does whatever it does, and when the daemon script returns, the supervisor closes the stream, and returns to the listening state. Your daemon script may need to consider that it could be called simultaneously in different threads.
To shut down all listeners, set user.inetd.shutdown to true. (Stop Daemons in the Inetd menu does the same thing.) This causes all supervisor threads to stop looping, and this in turn causes each NetEvents listener to time out (because the supervisor loop was what was keeping it alive); this may take some time, but after a while you should see all the "state" squares in the NetEvents status window turn gray.2
To illustrate, we use the inetd suite to turn Frontier into a primitive Web server. (This is basically the example included with the inetd suite.) Set up a daemon table; since we're just testing, set count to 1, and the port must be 80, because this is where browsers will expect the server to be. The daemon script is shown as follows.
The fourth line hardcodes a folder which will serve as the root folder of our "Web site"; you'll need to rewrite this line, designating a folder on your hard disk. Put into that folder an HTML file called default.html. Now call inetd.start-OneDaemon(), handing it the address of the daemon table. Wait until NetEvents shows LISTENING in its status window. Then go to another computer, start up a browser, and ask it to open the URL http://otherComputer.com/default.html, where otherComputer.com is the name or IP number of the server computer. The HTML file will be displayed in the browser!
When our daemon script is called, it calls netEvents.readStream() to obtain the browser's request. 10000 is just an arbitrary number of bytes, larger than the expected size of the request. We assume that the request begins like this:
This is followed by a lot of other stuff which we ignore; remember, this is a primitive Web server! We parse the second word to get the file's pathname, read the file from disk, and send it back with netEvents.writeStream().
When you're finished playing, set user.inetd.shutdown to true on the server computer. While Frontier would probably not substitute for an industrial-strength dedicated Web server application, this is a convincing demonstration of its potential.
The odbServer suite sets up Frontier as a server when another copy of Frontier is to talk to it across a TCP/IP network; for this purpose it is easier to use, and more flexible, than inetd.
The odbServer architecture uses a "dispatcher." There is a table, user.odb-Server.commands, containing scripts ready to respond to commands arriving over the network. When a command arrives, it is routed by name to the correct script in user.odbServer.commands. The receiving script should take one parameter, the address of a table. The entries in this table are strings; the script should know what to do based on their names and values. Whatever the script returns will be sent back over the network to the client.
There are only four commands to know:
Given on the server side. Calls inetd.startOne() to create listeners in accordance with the specifications in suites.odbserver.daemon. To stop the threads and time out the listeners, set user.inetd.shutdown to true.
odbServer.commandEncode( commandName, addrAttributeTable)
Used on the client side to prepare a command for sending; commandName is the name of a script on the server machine in user.odbServer.commands to be called and handed as parameter a copy of the table at addrAttributeTable.
May also be used on the server side to return a table as a reply; in this case, commandName must be the empty string.
odbServer.commandDecode( encodedReply, addrTable)
Used on the client side to decode a reply, if it was encoded on the server side. The encoded reply encodedReply is decoded into the table at addrTable.
odbServer.commandSend( encodedCommand, server)
Used on the client side to send to the server the encodedCommand, which is the result of commandEncode(). server is the IP address of the server computer; if omitted, user.odbServer.prefs.serverAddress will be used.
The architecture sets up just one server, on port 1997; but that server can easily handle numerous commands, because each command is dispatched to a different script.
The server is given an opportunity to preprocess the attribute table before it is handed to the command script; to take advantage of this, modify user.odb-Server.commandFilter.
As an example, we will use odbServer to implement a rudimentary version of the NetFrontier remoteGet() functionality described in Chapter 39, NetFrontier. The idea is that a client copy of Frontier will be able to obtain from a server copy of Frontier the value of any entry in the latter's database.
At the server end, we have a script user.odbServer.commands.remoteGet().
The script expects the parameter table to contain one entry, whatToGet, a string that can be coerced to the address of a database entry. The script returns a table with three entries: error, to communicate error information, if any; value, a binary version of the requested database entry's value; and type, the requested database entry's datatype. The reason for this elaborate encoding is that odbserver.commandEncode() will coerce every entry in our returned table to a string; therefore, we pack the database entry's value to a binary to preserve its contents, and send along its datatype separately because that information will otherwise be lost when the binary is coerced to a string.
The following is a script on the client computer that asks the user for the name of the database entry to get, sends the command to the server computer (whose IP address is assumed to be in user.odbServer.prefs.serverAddress already), receives the reply, checks for errors, and deposits the requested entry's value at scratchpad.reply.
The value is coerced from a string back to a binary, assigned the correct internal datatype, and then unpacked, to reconstruct the original value.
In order to act as a client to a remote server, Frontier needs to know the protocol for talking to that server. A protocol is a well-defined grammar of command and response. For instance, in Example 38-3, we knew in advance that the remoteGet command was expecting a table with one entry called whatToGet and that it would return a table with three entries called error, value, and type. A protocol can get quite complicated, because it may involve various commands, where the server must parse the request and do different things in different cases, and then the client must do different things depending on what reply comes back.
The reason we knew the protocol for remoteGet is that we wrote the server ourselves. But apart from this sort of situation, how are we going to know any protocols? The standard Internet server types, such as FTP, HTTP, NNTP (Usenet "news"), and POP and SMTP (mail), each have standardized protocols which are widely available.3 Some simple servers with unique protocols respond to a command HELP by returning a description of their protocol. Also, if you already have a client application that "knows" a protocol, you may be able to intercept its conversation with a server and deduce the salient features of the protocol.4
In Frontier, the most important standard protocols are implemented already, in system.extensions.tcpCmd . The salient features of this table are as follows:
A boolean. If true, the UCMD at tcpCmd.code will be used for talking to the network; if false, NetEvents will be used. It is generally agreed that using NetEvents is now better for most purposes.
The bottom-level commands for talking to NetEvents.
Tables containing scripts for each command in a protocol, along with scripts that combine these into larger commonly needed structures.
A table of scripts and outlines that show some of the commands in action.
Scripts that perform high-level self-contained sessions with a human interface
For example, sendMail() gets your SMTP server to send a mail message; download() obtains a file from an FTP server; and so forth.
The tcpCmd scripts are far too extensive to document fully here.5 They provide the same sort of logic and functionality that a full-fledged client application has to provide - a remarkable body of work, especially considering that they must juggle the details of conforming to protocols, interfacing with NetEvents or the UCMD, interacting with the user, and so forth.
As an illustration, here is a utility script that relies upon tcpCmd. As part of a Web site I manage, I sometimes have to upload an .hqx file to the site via FTP. But the FTP server at the far end always unBinHexes these, which isn't what I want; the workaround is to give the file a meaningless suffix such as .hqxx, upload it, and then rename the remote copy so that it ends in .hqx. The operation is common and tedious, so a scripted version is clearly in order.
To learn to write the script, I first performed the whole operation with OTSessionWatcher running, so that I could capture the TCP/IP messages and figure out the protocol. Then I explored the scripts in tcpCmd. Some of these are very high-level, such as tcpCmd.upload() , which manages an entire session and was clearly going to be the model for the structure of my script. Others perform large, high-level tasks as part of a session, such as tcpCmd.ftp.logon(), tcpCmd.ftp.putfile(), and tcpcmd.ftp.getHostDirectory(). Some implement sending a single command in the protocol, such as tcpCmd.ftp.dele(), which sends the DELE command to delete a file. These are supported by slightly lower-level utilities such as tcpCmd.interfaces.sendCommand(), which sends commands not already in the scripted repertory; I would need this for RNFR and RNTO, which rename a remote file.
We first perform a reality check: if the file isn't prepared by ending the name with .hqxx, the user has made a mistake. Then we log on to the FTP server, get to the right directory with tcpcmd.ftp.cwd(), and upload the file. Then we get a directory listing, to see if the directory already contains an earlier version of the file with the name we are about to give ours; if so, we delete it. Now we rename the file, which the protocol decrees is a two-step process (say what file is to be renamed, then say the new name); notice the technique for checking the response from one command before proceeding to the next. That's it, so we log off; the whole operation is also embedded in a try so that we can log off if anything goes wrong along the way.
For a splendid example of a nonstandard protocol client implemented from the ground up, obtain the webster suite,6 which gets word definitions from the University of Michigan's dictionary server.
We have seen Frontier act as a rudimentary HTTP server; we have seen it perform an entire FTP session without human intervention and without the use of FTP client software. We have also seen how copies of Frontier can easily communicate over the network as custom client and server. The examples given are elementary, but they all perform quite significant tasks, and should serve to suggest Frontier's potential power as a TCP/IP network citizen.
1. Experience will determine what the limits are. Each thread takes up memory in NetEvents's heap space and memory in Frontier's heap space, and slows down other operations slightly.
2. It seems incredible that the only way to shut down one inetd server is to shut all of them down; but that's the way it is, as far as I can tell.
3. For example, the HTTP protocol is available at http://www.w3.org/Protocols/.
4. On Mac OS, using Open Transport, you can do this with Peter N Lewis's OTSessionWatcher application, available at ftp://ftp.stairways.com.
5. See http://www.erols.com/asg/tcpcmdDocs/. The scripts are mostly the work of Alan German.
6. By Preston Holmes; see http://siolibrary.ucsd.edu/Preston/scripting/root/suites/webster.html.
| TOP | UP: Applied Frontier | NEXT: NetFrontier |