This crux of this module is the Terminal class which is a pure-Python implementation of the quintessential Unix terminal emulator. It does its best to emulate an xterm and along with that comes support for the majority of the relevant portions of ECMA-48. This includes support for emulating varous VT-* terminal types as well as the "linux" terminal type.
The Terminal class's VT-* emulation support is not complete but it should suffice for most terminal emulation needs (e.g. all your typical command line programs should work wonderfully). If something doesn't look quite right or you need support for certain modes added please feel free to open a ticket on Gate One's issue tracker: https://github.com/liftoff/GateOne/issues
Note that Terminal was written from scratch in order to be as fast as possible. It is extensively commented and implements some interesting patterns in order to maximize execution speed (most notably for things that loop). Some bits of code may seem "un-Pythonic" and/or difficult to grok but understand that this is probably due to optimizations. If you know "a better way" please feel free to submit a patch, open a ticket, or send us an email. There's a reason why open source software is a superior development model!
Without any special mode settings or parameters Terminal should effectively emulate the following terminal types:
- xterm (the most important one)
- ECMA-48/ANSI X3.64
- Nearly all the VT-* types: VT-52, VT-100, VT-220, VT-320, VT-420, and VT-520
- Linux console ("linux")
If you want Terminal to support something else or it's missing a feature from any given terminal type please let us know. We'll implement it!
The Terminal class is meant to emulate the display portion of a given terminal. It does not translate keystrokes into escape sequences or special control codes--you'll have to take care of that in your application (or at the client-side like Gate One). It does, however, keep track of many keystroke-specific modes of operation such as Application Cursor Keys and the G0 and G1 charset modes with callbacks that can be used to notify your application when such things change.
Many methods inside Terminal start with an underscore. This was done to indicate that such methods shouldn't be called directly (from a program that imported the module). If it was thought that a situation might arise where a method could be used externally by a controlling program, the underscore was omitted.
To support asynchronous usage (and make everything faster), Terminal was written to support extensive callbacks that are called when certain events are encountered. Here are the events and their callbacks:
Callback Constant (ID) | Called when... |
---|---|
terminal.CALLBACK_SCROLL_UP | The terminal is scrolled up (back). |
terminal.CALLBACK_CHANGED | The screen is changed/updated. |
terminal.CALLBACK_CURSOR_POS | The cursor position changes. |
terminal.CALLBACK_DSR | A Device Status Report (DSR) is requested (via the DSR escape sequence). |
terminal.CALLBACK_TITLE | The terminal title changes (xterm-style) |
terminal.CALLBACK_BELL | The bell character (^G) is encountered. |
terminal.CALLBACK_OPT | The special optional escape sequence is encountered. |
terminal.CALLBACK_MODE | The terminal mode setting changes (e.g. use alternate screen buffer). |
Note that CALLBACK_DSR is special in that it in most cases it will be called with arguments. See the code for examples of how and when this happens.
Also, in most cases it is unwise to override CALLBACK_MODE since this method is primarily meant for internal use within the Terminal class.
Gate One makes extensive use of the Terminal class and its callbacks. So that's a great place to look for specific examples (gateone.py and termio.py, specifically). Having said that, implementing Terminal is pretty straightforward:
>>> import terminal
>>> term = terminal.Terminal(24, 80)
>>> term.write("This text will be written to the terminal screen.")
>>> term.dump()
[u'This text will be written to the terminal screen. ',
<snip>
u' ']
Here's an example with some basic callbacks:
>>> def mycallback():
... "This will be called whenever the screen changes."
... print("Screen update! Perfect time to dump the terminal screen.")
... print(term.dump()[0]) # Only need to see the top line for this demo =)
... print("Just dumped the screen.")
>>> import terminal
>>> term = terminal.Terminal(24, 80)
>>> term.callbacks[term.CALLBACK_CHANGED] = mycallback
>>> term.write("This should result in mycallback() being called")
Screen update! Perfect time to dump the terminal screen.
This should result in mycallback() being called
Just dumped the screen.
Note
In testing Gate One it was determined that it is faster to perform the conversion of a terminal screen to HTML on the server side than it is on the client side (via JavaScript anyway).
The Terminal class implements a scrollback buffer. Here's how it works: Whenever a Terminal.scroll_up() event occurs, the line (or lines) that will be removed from the top of the screen will be placed into Terminal.scrollback_buf. Then whenever Terminal.dump_html() is called the scrollback buffer will be returned along with the screen output and reset to an empty state.
Why do this? In the event that a very large Terminal.write() occurs (e.g. 'ps aux'), it gives the controlling program the ability to capture what went past the screen without some fancy tracking logic surrounding Terminal.write().
More information about how this works can be had by looking at the Terminal.dump_html() function itself.
Note
There's more than one function that empties Terminal.scrollback_buf when called. You'll just have to have a look around =)
Used in conjunction with codecs.register_error, will replace special ascii characters such as 0xDA and 0xc4 (which are used by ncurses) with their Unicode equivalents.
Takes a list, renditions, and reduces it to its logical equivalent (as far as renditions go). Example:
[0, 32, 0, 34, 0, 32]
Would become:
[0, 32]
Other Examples:
[0, 1, 36, 36] -> [0, 1, 36]
[0, 30, 42, 30, 42] -> [0, 30, 42]
[36, 32, 44, 42] -> [32, 42]
[36, 35] -> [35]
Terminal controller class.
Initializes the terminal by calling self.initialize(rows, cols). This is so we can have an equivalent function in situations where __init__() gets overridden.
If em_dimensions are provided they will be used to determine how many lines images will take when they're drawn in the terminal. This is to prevent images that are written to the top of the screen from having their tops cut off. em_dimensions should be a dict in the form of:
{'height': <px>, 'width': <px>}
Initializes the terminal (the actual equivalent to __init__()).
Fills screen with empty lines of (unicode) spaces using self.cols and self.rows for the dimensions.
Note
Just because each line starts out with a uniform length does not mean it will stay that way. Processing of escape sequences is handled when an output function is called.
Fills self.renditions with lists of [0] using self.cols and self.rows for the dimenions.
Empties the scrollback buffers (self.scrollback_buf and self.scrollback_renditions).
Attaches the given callback to the given event. If given, identifier can be used to reference this callback leter (e.g. when you want to remove it). Otherwise an identifier will be generated automatically. If the given identifier is already attached to a callback at the given event that callback will be replaced with callback.
event: The numeric ID of the event you're attaching callback to. The callback constants should be used as the numerical IDs. callback: The function you're attaching to the event. identifier: A string or number to be used as a reference point should you wish to remove or update this callback later.
Returns the identifier of the callback. to Example:
>>> term = Terminal()
>>> def somefunc(): pass
>>> id = "myref"
>>> ref = term.add_callback(term.CALLBACK_BELL, somefunc, id)
Note
This allows the controlling program to have multiple callbacks for the same event.
Removes the callback referenced by identifier that is attached to the given event. Example:
>>> term.remove_callback(CALLBACK_BELL, "myref")
Resets the terminal back to an empty screen with all defaults. Calls Terminal.callbacks[CALLBACK_RESET]() when finished.
Note
If terminal output has been suspended (e.g. via ctrl-s) this will not un-suspend it (you need to issue ctrl-q to the underlying program to do that).
Resizes the terminal window, adding or removing rows or cols as needed. If em_dimensions are provided they will be stored in self.em_dimensions (which is currently only used by image output).
DECSTBM - Sets self.top_margin and self.bottom_margin using the provided settings in the form of '<top_margin>;<bottom_margin>'.
Note
This also handles restore/set "DEC Private Mode Values".
Sets self.title to title and executes Terminal.callbacks[CALLBACK_TITLE]()
Saves the cursor position and current rendition settings to self.saved_cursorX, self.saved_cursorY, and self.saved_rendition
Note
Also handles the set/restore "Private Mode Settings" sequence.
Restores the cursor position and rendition settings from self.saved_cursorX, self.saved_cursorY, and self.saved_rendition (if they're set).
Returns the current cursor positition as a DSR response in the form of:
'<self.cursorY>;<self.cursorX>R'
Also executes CALLBACK_DSR with the same output as the first argument. Example:
self.callbacks[CALLBACK_DSR]('20;123R')
Handles Device Control String sequences. Unimplemented. Probablye not appropriate for Gate One. If you believe this to be false please open a ticket in the issue tracker.
This function handles the control sequences that set double and single line heights and widths. It also handles the "screen alignment test" ( fill the screen with Es).
Note
Double-line height text is currently unimplemented (does anything actually use it?).
Sets the terminal's G0 (default) charset to the type specified by char.
Here's the possibilities:
0 DEC Special Character and Line Drawing Set
A United Kingdom (UK)
B United States (USASCII)
4 Dutch
C Finnish
5 Finnish
R French
Q French Canadian
K German
Y Italian
E Norwegian/Danish
6 Norwegian/Danish
Z Spanish
H Swedish
7 Swedish
= Swiss
Sets the terminal's G1 (alt) charset to the type specified by char.
Here's the possibilities:
0 DEC Special Character and Line Drawing Set
A United Kingdom (UK)
B United States (USASCII)
4 Dutch
C Finnish
5 Finnish
R French
Q French Canadian
K German
Y Italian
E Norwegian/Danish
6 Norwegian/Danish
Z Spanish
H Swedish
7 Swedish
= Swiss
Sets the current charset to G0. This should get called when ASCII_SO is encountered.
Sets the current charset to G1. This should get called when ASCII_SI is encountered.
Write chars to the terminal at the current cursor position advancing the cursor as it does so. If chars is not unicode, it will be converted to unicode before being stored in self.screen.
if special_checks is True (default), Gate One will perform checks for special things like image files coming in via chars.
Only here to make Terminal compatible with programs that want to use file-like methods.
Scrolls up the terminal screen by n lines (default: 1). The callbacks CALLBACK_CHANGED and CALLBACK_SCROLL_UP are called after scrolling the screen.
Note
This will only scroll up the region within self.top_margin and self.bottom_margin (if set).
Scrolls down the terminal screen by n lines (default: 1). The callbacks CALLBACK_CHANGED and CALLBACK_SCROLL_DOWN are called after scrolling the screen.
Execute a backspace (x08)
Execute horizontal tab (x09)
LF - Executes a line feed.
Note
This actually just calls Terminal.newline().
CNL - Moves the cursor down one line to the home position. Will not result in a scrolling event like newline() does.
Note
This is not the same thing as Terminal.cursor_next_line() which preserves the cursor's column position.
RI - Executes a reverse line feed: Move the cursor up one line to the home position. If the cursor move would result in going past the top margin of the screen (upwards) this will execute a scroll_down() event.
Increases self.cursorY by 1 and calls Terminal.scroll_up() if that action will move the curor past self.bottom_margin (usually the bottom of the screen).
Executes a carriage return (sets self.cursorX to 0). In other words it moves the cursor back to position 0 on the line.
Handles the XON character (stop ignoring).
Note
Doesn't actually do anything (this feature was probably meant for the underlying terminal program).
Handles the XOFF character (start ignoring)
Note
Doesn't actually do anything (this feature was probably meant for the underlying terminal program).
Cancels any escape sequence currently being processed. In other words it empties self.esc_buffer.
Cancels any escape sequence currently in progress and replaces self.esc_buffer with single question mark (?).
Note
Nothing presently uses this function and I can't remember what it was supposed to be part of (LOL!). Obviously it isn't very important.
Handles the escape character as well as escape sequences that may end with an escape character.
Marks the start of a CSI escape sequence (which is itself a character) by setting self.esc_buffer to '\x1b[' (which is the CSI escape sequence).
This gets called when an image (PNG or JPEG) was detected by Terminal.write() and captured in self.image. It cleans up the data inside self.image (getting rid of carriage returns) and stores self.image as if it were a single character in self.screen at the current cursor position.
It also moves the cursor to the beginning of the last line before doing this in order to ensure that the captured image stays within the browser's current window.
Note
The Terminal._spanify_screen() function is aware of this logic and knows that a 'character' longer than an actual character indicates the presence of something like an image that needs special processing.
Handle the string terminator (ST).
Note
Doesn't actually do anything at the moment. Probably not needed since Terminal._escape() and/or Terminal.bell() will end up handling any sort of sequence that would end in an ST anyway.
Handles Operating System Command (OSC) escape sequences which need special care since they are of indeterminiate length and end with either a bell (\x07) or a sequence terminator (\x9c aka ST). This will usually be called from Terminal.bell() to set the title of the terminal (just like an xterm) but it is also possible to be called directly whenever an ST is encountered.
Handles the bell character and executes Terminal.callbacks[CALLBACK_BELL]() (if we are not in the middle of an escape sequence that ends with a bell character =). If we are in the middle of an escape sequence, calls self._osc_handler() since we can be nearly certain that we're simply terminating an OSC sequence. Isn't terminal emulation grand? ⨀_⨀
Returns '[0n' (terminal OK) and executes:
self.callbacks[CALLBACK_DSR]("\x1b[0n")
Returns '\x1b[1;2c' (Meaning: I'm a vt220 terminal, version 1.0) and executes:
self.callbacks[self.CALLBACK_DSR]("\x1b[1;2c")
Accepts "standard mode" settings. Typically '\x1b[?25h' to hide cursor.
Notes on modes:
'?1h' - Application Cursor Keys
'?5h' - DECSCNM (default off): Set reverse-video mode.
'?12h' - Local echo (SRM or Send Receive Mode)
'?25h' - Hide cursor
'?1049h' - Save cursor and screen
Accepts "standard mode" settings. Typically '\x1b[?25l' to show cursor.
Sets self.application_keys equal to boolean. Literally:
self.application_keys = boolean
If alt is True, copy the current screen and renditions to self.alt_screen and self.alt_renditions then re-init self.screen and self.renditions.
If alt is False, restore the saved screen buffer and renditions then nullify self.alt_screen and self.alt_renditions.
Same as Terminal.toggle_alternate_screen_buffer() but also saves/restores the cursor location.
Turns on or off local echo dependong on the value of onoff:
self.local_echo = onoff
DCH - Deletes (to the left) the specified number of characters at the cursor position. As characters are deleted, the remaining characters between the cursor and right margin move to the left. Character attributes (renditions) move with the characters. The terminal adds blank spaces with no visual character attributes at the right margin. DCH has no effect outside the scrolling margins.
Note
Deletes renditions too. You'd think that would be in one of the VT-* manuals... Nope!
Erases (to the right) the specified number of characters at the cursor position.
Note
Deletes renditions too.
ESCnH CUP (Cursor Position). Move the cursor to the given coordinates.
coordinates: Should be something like, 'row;col' (1-based) but, 'row', 'row;', and ';col' are also valid (assumes 1 on missing value).
Note
If coordinates is '' (an empty string), the cursor will be moved to the top left (1;1).
Vertical Line Position Absolute (VPA) - Moves the cursor to given line.
Clears the screen. Also used to emulate a terminal reset.
Note
The current rendition (self.cur_rendition) will be applied to all characters on the screen when this function is called.
CSI n J ED (Erase Data). This escape sequence uses the following rules:
Esc[J | Clear screen from cursor down | ED0 |
Esc[0J | Clear screen from cursor down | ED0 |
Esc[1J | Clear screen from cursor up | ED1 |
Esc[2J | Clear entire screen | ED2 |
CSI*n*K EL (Erase in Line). This escape sequence uses the following rules:
Esc[K | Clear screen from cursor right | EL0 |
Esc[0K | Clear screen from cursor right | EL0 |
Esc[1K | Clear screen from cursor left | EL1 |
Esc[2K | Clear entire line | ED2 |
Sets the values the dict, self.leds depending on n using the following rules:
Esc[0q | Turn off all four leds | DECLL0 |
Esc[1q | Turn on LED #1 | DECLL1 |
Esc[2q | Turn on LED #2 | DECLL2 |
Esc[3q | Turn on LED #3 | DECLL3 |
Esc[4q | Turn on LED #4 | DECLL4 |
Note
These aren't implemented in Gate One's GUI (yet) but they certainly kept track of!
Sets self.renditions[self.cursorY][self.cursorX] equal to n.split(';').
n is expected to be a string of ECMA-48 rendition numbers separated by semicolons. Example:
'0;1;31'
...will result in:
[0, 1, 31]
Note that the numbers were converted to integers and the order was preserved.
Optional special escape sequence handler for sequences matching RE_OPT_SEQ. If CALLBACK_OPT is defined it will be called like so:
self.callbacks[CALLBACK_OPT](chars)
Applications can use this escape sequence to define whatever special handlers they like. It works like this: If an escape sequence is encountered matching RE_OPT_SEQ this method will be called with the inbetween chars (e.g. ]_;<chars>) as the argument.
Applications can then do what they wish with chars.
Note
I added this functionality so that plugin authors would have a mechanism to communicate with terminal applications. See the SSH plugin for an example of how this can be done (there's channels of communication amongst ssh_connect.py, ssh.js, and ssh.py).
Iterates over the lines in screen and renditions, applying HTML markup (span tags) where appropriate and returns the result as a list of lines. It also marks the cursor position via a <span> tag at the appropriate location.
Spanifies everything inside screen using renditions. This differs from _spanify_screen() in that it doesn't apply any logic to detect the location of the cursor (to make it just a tiny bit faster).
Dumps the terminal screen as a list of HTML-formatted lines.
Note
This places <span class="cursor">(current character)</span> around the cursor location.
Dumps the screen and the scrollback buffer as-is then empties the scrollback buffer.
Dumps the screen and renditions as-is, the scrollback buffer as HTML, and the current cursor coordinates. Also, empties the scrollback buffer
Note
This was used in some performance-related experiments but might be useful for other patterns in the future so I've left it here.
Returns self.screen as a list of strings with no formatting. No scrollback buffer. No renditions. It is meant to be used to get a quick glance of what is being displayed (when debugging).
Note
This method does not empty the scrollback buffer.
Does nothing (on purpose!). Used as a placeholder for unimplemented functions.