PREV UP NEXT ne's manual

Chapter 7: Motivations and Design

In this chapter I will try to outline the rationale behind ne's design choices. Moreover, some present, voluntary limitations of the current implementation will be described. The intended audience of such a description is the programmer wanting to hack up ne's sources, or the informed user wanting to deepen its knowledge of the limitations.

ne has no concept of mode. All shortcuts are defined by a single key, possibly with a modifier (such as CONTROL or META). Modality is in my opinion a Bad Thing unless it has a very clear visual feedback. As an example, menus are a form of modality. After entering the menus, the alphabetic keys and the navigation keys have a different meaning. But the modality is clearly reflected by a change in the graphical user interface. The same can be said about the input line, because it is always preceeded by a (possibly highlighted) prompt ending with a colon.

ne has no sophisticated visual updating system similar, for instance, to the one of curses. All updating is done while manipulating the text, and only if the turbo flag is set some iterated operations can delay the update (in this case, ne keeps track in a very rough way of the part of the screen which changed). Moreover, the output is not preempted by additional input coming in, so that along a slow connection the output could not keep up with the input. However, along fast connections, the responsiveness of the editor is greatly enhanced by the direct update. Moreover, a great deal of memory and computational power is gained, because it is not necessary to keep constantly updated two copies of the screen, and to compare them whenever doing an update. As it is typical in ne, when such design tradeoffs arise preference is given to the solution which is effective on a good part of the existing hardware, and will be very effective on most future hardware.

ne uses a particular scheme for handling the text. There is a doubly linked list of line descriptors which contain pointers to each line of text. The lines themselves are kept in a list of pools, which is expanded and reduced dynamically. The interesting thing is that for each pool ne keeps track just of the first and of the last character used. A character is free iff it containes a null, so there is no need for a list of free chunks. The point is that the free characters laying between that first and the last used characters (the lost characters) can only be allocated locally: whenever a line has to grow in length, ne first checks if there are enough free characters around it. Otherwise, it remaps the line elsewhere. Since editing is essentially a local activity, the number of such lost characters remains very low. And the manipulation of a line is extremely fast and independent of the size of the file, which can be very huge. A mathematical analysis of the space/time tradeoff is rather difficult, but empirical evidence suggests that the idea works.

ne takes the POSIX standard as the basis for UN*X compatibility. The fact that this standard has been designed by a worldwide recognized and impartial organization such as IEEE makes it in my opinion the most interesting effort in his league. No attempt is made of supporting ten thousands different versions and releases by using conditional compilation. Very few assumptions are made about the behaviour of the system calls. This has obvious advantages in terms of code testing, maintenance, and reliability. For the same reasons, the availability of an ANSI C compiler is assumed.

If the system has a terminfo database and the relative functions (which are usually contained in the library `libcurses.a'), ne will use them. The need for a terminal capability database is clear, and the choice of terminfo (with respect to termcap) is compulsory if you want to support a series of features (such as more than ten function keys) which termcap lacks. If terminfo is not available, ne can use a termcap database. Some details about this can be found in Portability Problems.

ne does not allow to redefine the ESCAPE and RETURN keys, and the interrupt character CONTROL-\. This decision has been taken mainly for two reasons. First of all, it is necessary to keep a user from transforming ne's bindings to such a point that another unaware user cannot work with it. These two keys and the alphabetic keys allow to activate any command without any further knowledge of the key bindings, so it seems to me this is a good choice. As a second point, the ESCAPE key usage should generally be avoided. The reason is that most escape sequences that are produced by special keys start exactly with the escape character. When ESCAPE is pressed, ne has to wait for one second (this timing can be changed with the EscapeTime command), just to be sure that it did not receive the first character of an escape sequence. This makes the response of the key very slow, unless it is immediately followed by another key such as `:'. See Hints and Tricks.

Note that it was stated several times that the custom key bindings work also when doing a long input, navigating through the menus or browsing the requester. This is only partially true. In order to keep down the code size and complexity, in these cases ne recognizes only direct bindings to commands, and discards the arguments. Thus, for instance, if a key is bound to the command line `LineUp 2', it will act like `LineUp', while a binding to `Macro MoveItUp' would produce no result. Of course full binding capability is available while writing text. This limitation will probably be lifted in a future version: presently it does not seem to limit seriously the configurability of ne.

ne has some restriction in its terminal handling. It does not support highlighting on terminals which use a magic cookie. Supporting correctly such terminals is a royal pain, and I did not have any means of testing the code anyway. Moreover, they are rather obsolete. Another lack of support is for the capability strings which specify a file to print or a program to launch in order to initialize the terminal.

The macro capabilities of ne are rather limited. For instance, you cannot give an argument to a macro: they are simply scripts which can be played back automatically. This makes them very useful for everyday usage in a learn/play context, but rather unflexible for extending the capabilities of the program. However, it is not reasonable to incorporate in an editor an interpreter for a custom language. Rather, a systemwide macro language should control the editor via interprocess communication. This is the way of the REXX language: unfortunately, a diffused, uniform, standard implementation of REXX under UN*X is not likely to appear. However, the next version of ne will certainly feature a REXX port on the Amiga.

ne has been written with sparing resource usage as a basic goal. Every possible effort has been made in order to reduce the use of CPU time and memory, and the number of system calls. For instance, command parsing is done through hash techniques, and the escape sequence analysis uses the order structure of strings for minimizing the number of comparisons. The optimal cursor motion functions were directly copied from emacs. No busy polling is allowed. Doubly headed, doubly linked lists allow for very fast list operations without any special case whatsoever. The search algorithm is a version of the Boyer-Moore algorithm which provides high performance with a minimal setup time. An effort has been taken to move to the text segment all data which do not change during the program execution.

A word should be spent about lists. Clearly, handling the text as a single block with an insertion gap (a la emacs) allows you to gain some memory, since you do not have to allocate the list nodes, which require usually 16 bytes per line. However, the management of the text as a linked list requires much less CPU time, and the tradeoff seems to be particularly favorable on virtual memory systems, where moving the insertion gap can require a lot of accesses to different pages.

Just to give a pratical example, on the HP-UX systems where ne was developed vi requires more memory than ne, unless the size of the file to edit is rather big, in which case ne requires a data segment about 20% bigger. (Of course, this does not take into account some sophisticated features of ne, such as unlimited undo/redo, which can cause a major memory consumption.)

Contact: - about ne - about these pages