TUTORIAL: Command Line Interfaces

Some of you writing servers are only now realizing how limited your use of normal stdin API’s are when working with Twisted. This tutorial shows you how to use Twisted’s command line receiver as well as my more advanced command line interface module.


Twisted’s LineReceiver

You generally can’t use blocking reads in Twisted and most of the stdin tools you’re used to are, in fact, blocking. For example, “rawinput()” doesn’t work well with Twisted at all.

Instead, you should use an asynchronous mechanism for handling stdin and stout. The Twisted “LineReciever” class (https://twistedmatrix.com/documents/9.0.0/api/twisted.protocols.basic.LineReceiver.html) is just such a thing. It works just like your protocols except that, instead of dataReceived, it has a lineReceived() method. Let’s walk through a very simple example:

Let’s suppose we want to do an echo protocol with twisted, but not over the network. We’ll type something in and just write it back out to the screen. In a non-Twisted world, it might look like this:

line = rawinput()
while line.lower() != "quit":
  print "ECHO:", line
  line = rawinput()

We can’t use rawinput very well with twisted because it blocks. Remember that Twisted is not multi-threaded. So while the raw_input blocks waiting for the user to press “enter”, nothing else can happen. Twisted can’t handle any of the networking code, fire any tasks that were scheduled using callLater, and so forth.

But we could rewrite it using an asynchronous I/O protocol:

from twisted.protocols import basic
from twisted.internet import stdio, reactor
import os

class StdioEcho(basic.LineReceiver):
  delimiter = os.linesep
  def lineReceived(self, line):
    if line.lower() == "quit":
      self.transport.loseConnection()
    else:
      self.transport.write("ECHO: %s\n" % line)
stdio.StandardIO(StdioEcho())
reactor.run()

The StdioEcho protocol receives lines input from the keyboard (stdin) in its lineReceived method. It can write out to the screen (stdout) using self.transport.write(). All of this is asynchronous, so it won’t interfere with the reactor

(note, in the example above, the program won’t terminate after “quit” because the reactor is still running. If you want to test this program and have it quit, you could add “reactor.stop()” right after self.transport.loseConnection(). I didn’t put it in here because if you’re interacting with playground, you do not want to stop the reactor.)

You can use this basic protocol for writing any of your command-line interfaces for Playground servers. However, it does not deal well with handling “interruptions” (e.g., new data received while the user is still typing a response), or certain other “advanced” features such as tab-completion. For those types of features, I recommend my enhanced CLI shell.


utils.ui.CLIShell

Almost every command line interface in playground now uses the utils.ui.CLIShell class. CLIShell is a subclass of LineReciever, and uses both “lineReceived” and “transport.write” for stdio.  However, it supports some very powerful advanced features.

First and foremost, CLIShell assumes that you want to be able to issue commands. Accordingly, you can register for command handlers within the class using one of three different mechanisms. Let’s look in apps/bank/OnlineBank.py for some examples. Look at all the commands configured by “__loadCommands”.  You will see a number of command handlers instantiated that look something like this:

 CLIShell.CommandHandler()

A command handler registers the CLIShell class to handle incoming lines and parse them if there is a recognizable command. The very first command in __loadCommands is very simple:

 adminCommandHandler = CLIShell.CommandHandler("admin","Toggle admin mode",self.__toggleAdmin)

The first argument is the command itself. If the user types “admin”, this command handler will intercept it. The second argument is the help text used by a built-in help system. The final argument is the callback. The callback handles the remaining arguments (if any). This class uses shell handling so quoted strings are treated as a single element.

Thus, if this class recieved the line ‘admin this is “a test”‘, it would pass ‘this’,  ‘is’, and ‘a test’ to the callback self.__toggleAdmin. The admin command doesn’t expect any parameters, but the callback could write out an error message to the screen.

Most of the command handlers are more complicated. You’ll notice that serveral have a “mode” argument. There are three modes for command handlers:

  1. SINGLETON_MODE (the default if not specified)
  2. STANDARD_MODE
  3. SUBCMD_MODE

SINGLETON_MODE means that there is always a single callback that will handle all arguments no matter how many are put on the command line. STANDARD_MODE allows you to specify different callbacks for different numbers of arguments (e.g., a callback for 2 arguments versus a callback for 3 arguments). And SUBCMD_MODE allows you to specify additional command words that follow the first.

You can look through __loadCommands to get a handle on these, or you can always just use the default SINGLETON_MODE. It’s perfectly acceptable to just let all arguments get passed to the callback and decide from there what to do with them. The biggest reasons for using the other modes are better help text, better error handling, and tab completion. The CLIShell supports tab completion for registered commands.

Once you’ve fully defined a command handler, it has to be registered.

self.registerCommand(adminCommandHandler)

It is possible to setup tab completion for things other than commands, but that’s outside the scope of this tutorial. You should be aware, however, that there is a built-in mechanism for filename tab completion. If the user starts a parameter with “f://” the shell will assume that it’s a file and will support tab completion based on the filesystem. However, when submitted to the callback ,the “f://” will be removed.

But what if you don’t want everything to be a command? Suppose you’re writing a chat client and you want some commands and otherwise all text to just be received and transmitted?

You can solve this by overloading the lineReceived method. For example, you could mandate that all commands begin with a certain symbol (‘@’, ‘#’, or ‘$’). If a line begins with that symbol, it’s treated like a command. Otherwise you process it differently. Here’s sample code that does that:

def lineReceived(self, line):
  if not line: 
    return (True, None)
  if line[0] == '$':
    return CLIShell.lineReceived(self, line[1:])
  else:
    process(line)
    return (True, None)

Please note that CLIShell needs to use utils.ui.stdio instead of twisted.protocols.basic.stdio to get all of its advanced features. In particular, the lineReceived method returns a result (True or False) and a Deferred (or None). By returning a deferred, this allows long running operations to complete before the protocol will be sent another line for processing.

To explain this last bit a little further, suppose that you run a command that takes 10 seconds to complete. While that command is running, your CLIShell class will normally be allowed to still receive commands. Remember, this is asynchronous. It won’t wait for you. However, if you return a deferred as the second argument from lineReceived, the utils.ui.stdio class will wait until the deferred completes before sending your CLIShell class another line.

Moreover, if you are receiving data in an asynchronous fashion (maybe from a network source), you may want the user typing a command to not be interrupted. The CLIShell system can’t write to the screen in a non-intrusive fashion, but it CAN make the user’s line reappear as it was before the interruption. In other words, if they were trying to type, “Hi, my name is seth” and a message needed to be printed when they had only typed “hi, my nam”, there is a way to print the incoming message, and print “hi, my nam” so that the user (though interrupted) can clearly see what they’ve typed so far. In practice, it would look like this:

>> Hi, my nam
Incoming message: This is a message from somewhere
>> Hi, my nam

The user’s cursor is right where they left off, and they aren’t trying to type “around” the incoming message. To do this, call “transport.refreshDisplay()” after a potentially interrupting write.

This concludes this first tutorial on CLIShells and Twisted’s LineReceiver.

Leave a Reply

Your email address will not be published. Required fields are marked *