High level functional description of a Terminal Emulator, Term - NetBeans API Javadoc (Current Development Version)
High level functional description of an ANSI Terminal Emulator, Term
Author:
Ivan Soleimanipour, Sun Microsystems.
Abstract:
A highly configurable ANSI terminal emulator.
Version:
4 of
Wed Oct 24 2001
Table of contents:
- Motivation
- User visible and API Features (as implemented currently)
- Covenant
- RFE queue
- Performance
- Use as a hypertext widget
- Design issues
- Source code and demos
Motivation
The original impetus for Term was the need for a terminal emulator for use by adaptations of NetBeans for non-Java development. These tools need a proper terminal emulator in at least three areas:- A window for program io, for programs being run under the debugger. These programs will be traditional unix programs and may utilize various levels of terminal manipulation, programs like 'more', 'top', 'vi' and 'ksh'.
- A window for the debugger (e.g. dbx) command io. The dbx command interpreter is a ksh variant and as such allows input history editing or execution of any unix program (like 'vi').
- A window for running alternative text editors, 'vi' or 'vim' under. More on this later.
A terminal emulator can also be used as a within-nb shell or telnet window.
See the
source code for a telnet demo.
As references I used
Robert Frees spec
and
this
as well as the DtTerm(3) man page from Solaris.
Unlike other termulators Term does not
do things like line buffering, echoing, CR/LF processing and so on.
Since it is predominantly intended for use under Unix and since
under Unix all this stuff is done by the kernel (accessed through
pty's) Term doesn't bother with any of this.
However, there is a mechanism analogous to SVR4 io streams, or
pushable network stack elements, that allows customization of
character processing on other systems. A generic simple
stream module,
Stream modules can be used for other purposes like recording of the io
to some history file or for testing purposes.
A set of
The UI ... is just too simple to be worth a picture.
This proposal comes with
Javadoc API documentation
and a
properties table
.
The information is stored in a Vector of Lines. Each Line has a
an array of 'char's for character storage and an Array of 'int's
for attribute storage. These two arrays grow together (which is
more efficient than having two separate Vectors or StringBuffers).
However the attribute array may be null if all the characters in the
line have no attributes.
In order to prevent needless copying, these buffers are directly
available to the paint functions.
Lines w/o attributes are rendered using Graphics.drawChars() in
the double buffered image setup by default by Swing and AWT.
Lines with attributes are rendered a "run" at a time. A run is
a sequence of characters all of which have the same attributes.
There are also various adornments to be rendered, like the cursor,
slection highlight, glyphs and so on.
These two alternatives are usually controlled by a property known as
"jump scrolling". Term doesn't have it but the effect can
be achieved by appropriately buffering the input and or internally
using paintImmediately().
The issues often have to do with my ignorance or lack of good guidelines on a
subject and are really a plea for suggestions, education and discussion.
Also, term is a "composite" widget and it is unclear to me how
one implements UIDelegates for composites, or whether one is
even needed.
However, this is really an internal implementation and a terminal
really interacts with the external world through it's io lines,
so the model is conceptually a "connection".
But a connection doesn't fit in with things like model-changed
events and such ... the data stored in the screen
backing store is ephemeral, it soon winks out of the history buffer.
So, I"m not exactly sure what a model would be for a termulator.
One candidate is the concept of sessions (A concept used in the
KDE 'konsole') where a single widget can multiplex between
multiple connections and the corresponding history buffers.
While such a feature should definitely be on the rfe list,
again, I'm not sure whether the Swing concept of a Model is
appropriate here.
Initially it seems obvious that it should, like every other good Swing
component, implement Scrollable, work properly under a JScrollPane and
not contain it's own scrollbar, but this is not neccessarily a good idea.
Usually the size of a component refers to it's visible size.
But when the component is placed under a JScrollPane, it's size starts
referring to it's "virtual" size over which the user scrolls.
While this makes complete sense for tables, lists, and text documents,
in the case of a terminal emulator we don't want this. We want size
to still govern the visible screen size!
It is perhaps possible to design a Term that works under JScrollPane such
that resizes communicate the viewport size via TermListener.sizeChanged()
and setRows() sets the viewports size (this I actually can't figure how
to do). But this requires that Term discover that it's under a
JScrollPane and things become progressively more complex from there on.
Another argument can be made from a utility standpoint.
I believe this is not so much because of the complex Document and Element
structures but due to the same reasons that could make Term slow, and
that is control over buffering and batching, or rather lack thereof.
Read the
section on performance to see
what I mean.
The offsets used by Term are actually valid as regions go out
of history. This because "absolute coordinates" are used where
each line put into Terms history gets a serial row number which
only grows. Term correctly handles the case where the 32-bit
absolute row number wraps. At a continuous rate of output of
4000 lines/sec the wrapping will happenin about 4 days.
There have been a few glitches like AWT converting a CR to a LF
that are currently being worked around in ad-hoc manner.
A fair amount of the code has to do with the gui setup, javac Process
setup and capture of it's output.
Once the output is received it is processed line-by-line. A crude pattern
is used to detect the first line of a javac error message and a new
ActiveRegion is begun, while the previous one is ended. These active regions
have their feedback attribute set which means that they will be highlighted
as the mouse moves over them. Anyone who has had to decipher a
wrapped compilation line and error line from 'make' would appreciate this.
The sourcefilename and line number are also extracted using simple patterns
and put in their own ActiveRegions as HTML-style links, which if clicked will
bring up the right line # in some editor, or rather fake it using a popup.
The beginning of each error message is also tagged with a simple
glyph in the glyph gutter.
Please keep in mind that this is strictly a Term feature demo.
I"m not advocating using glyphs to tag error messasges, or that textual
pattern recognition should be used for detecting distinct error
messages and their component, nor is capturing of Process output is as
trivial as it seems here (if the demo had instead used 'make' you'd
see some reordering of output because some comes from stdout and
some comes from stderr).
User visible and API Features (as implemented currently)
Note that there are two important differences between this widget and
traditional Java text widgets:
Not All sequences are implemented yet.
This should work particularly well under X-windows with JDK1.4 where they
have fixed X-windows selections.
LogicalLineVistor
Provides access to whole history buffer contents and facilitates
searches and contents dumps etc. Suitable information and mechanisms are
provided for mapping a logical line to screen coordinates to facilitate
highlighting and resumption of search for example.
LineDiscipline
is provided in this package.
This can be used, for example, to ensure that a horizontal line
starts on column 0.
Because of it's nature, Term, by default, consumes all of the keystrokes
it gets and passes them through to the output stream.
KeyStrokes
may be registered with Term though to
allow passing through of keystrokes of the clients choice.
Although there is some discussion to relax this considering the dearth of
monospaced unicode fonts.
It's just a screen area where text appears and a Swing scrollbar
in whatever is the current L&F.
Covenant
(A covenant is a part of a property deed that puts restrictions on the use
of the property on all future owners)
RFE queue
These are things that are planned but not implemented yet.
You average terminal emulator will keep line breaks (due to wrapping)
at the original column when the width of the emulator changes.
It should be possible to re-break them as the width changes.
This used to be an "issue" mainly because I hadn't learned much about it.
Term will implement the Accessible interface. The class Accessible is
really just a front gate to a whole host of other stuff that needs to
be implemented.
Performance
Term is made up of the two main subsystems:
The following points may be made:
When measuring or comparing performance attention should be paid to the
following:
Having said all of this, in my experience with the current implementation,
painting completely dominates buffer maintenance, so when you're
comparing Term with something else make sure they have comparable
refreshes/bytes-of-input.
Use as a hypertext widget
Term has facilities that allow it to be used as a reasonable styled
language and hypertext widget.
Design issues
I've been working off of O'Reilly publishers "JAVA Swing" by Eckstein, Loy
and Wood.
Many of the issues here are in the context of that book.
To that end the current implementation has a Document class used
internally, but there's still a bit to go.
It is not JScrollPane-friendly.
Here's why ...
Most termulator applications (xterm) and widgets (DtTerm) have a property
that governs whether and where they have a scrollbar. It would be much more
convenient to retain this as a runtime controllable property of
Term itself (read user option) instead of
having the container of Term manage that;
create a JScrollPane, put Term under it etc. in response to user
option toggling.
In my experience Swing text components just can't handle
the volumes of stuff coming into them that you typically get
in consoles.
Some ideas I've tossed out:
Term currently doesn't have a key map table. Most other termulators do.
The thing is I haven't yet seen the need. Lot of the usual type
of processing that key maps do seems to be absorbed by Swing and
AWT, so my initial hope was to just piggy-back on that. Same
thing for byte to char conversions.
Source code and demos
The source code for Term is in the NB CVS repository under
core/libsrc/org/netbeans/lib/terminalemulator.
Two demo's are available:
(Actually they are not! I'm not sure whether it is appropriate
to put them into the CVS repository).
Build tool demo
This is to showcase the capabilities of ActiveTerm and how it can be
used to annotate output, say from a build script.
Telnet demo
For a telnet client it uses portions of
The Java(tm) Telnet Application/Applet
by Matthias L. Jugel and Marcus Meisner.
I haven't included that stuff here to save space but you only need
de/mud/ternet/ScriptHandler.java
de/mud/ternet/TelnetProtocolHandler.java
de/mud/ternet/TelnetWrapper.java
You'll have to tweak the settings in the Makefile and the 'telnet' wrapper
script.