Source:Computer Graphics Quarterly, Volume 13, Issue 2 (1979)
The adage that "a picture is worth a thousand words" is well accepted by the computing community, although only a fortunate few have had access to devices which provide adequate graphic interaction. Visitors to laboratories where facilities for graphics are available leave with the distinct impression that if such tools would become available to them their productivity and enjoyment of computing would increase significantly. An example of an environment where such activity takes place is XEROX Research in Palo Alto where convincing educational  and programming projects  are supported using distributed computers which have bitmap display devices using the computer's main memory. The cost of storing images that way has been one factor in the slow dissemination of graphics capability in commonly available computers. Older systems used bitmaps stored on tracks of drums , and these have been used in situations where digitized images are analyzed for robot control  and in advanced circuit design . The use of drum tracks for picture storage makes the storage support per display economical if there is justification for at least several dozen displays in close proximity, which will share the storage drum. Less storage is needed if vectors are stored since then only the coordinates of the end points are placed in memory. A vector generator translates the coordinates into a signal which is then drawn on a high quality X-Y CRT screen . This mode of operation was already available in 1958 . Certain graphics applications lend themselves very well to vector displays. We have had good experiences with interactive curvefitting [8,9]. Applications in interactive design [10,11], analysis of three-dimensional objects as molecular structures [12,13], and architectural drawings  have been well served by these devices. These applications distinguish themselves by the fact that the original source of the data to be displayed is given as coordinates in space, and the transformation to an image is relatively straightforward.
When the source of the data is an image, as obtained by a camera or a scanner, then vector-oriented displays are of little help until the image is reduced to a much simpler abstraction composed of lines or contours. In order to effectively display images a matrix of picture elements has to be displayed, and each picture element (pixel) has to be displayed at a particular brightness. Commercial TV technology provides a standard here, even though pictures as seen on broadcast television rarely reach the desired level of quality. The specifications for TV include the use of 480 horizontal raster lines, which are displayed using two successive fields of 240 interlaced lines each. The signal being displayed is an analog signal, but we can derive from the aspect ratio (width/height, 4/3) of the TV picture frame that 640 pixels per line will be appropriate. The number of levels of brightness or gray scale per pixel is not specified, although 256 levels, as can be produced using eight bits per pixel, make the result indistinguishable from an analog picture image. In practice, most computer-based equipment attempts to reach that level in terms of raster or gray scale resolution. Useful and pleasing graphics can in fact be generated using half of the raster lines (one field) and only 4 bits per pixel to represent 16 levels of gray scale.
EFFECTIVENESS OF GRAPHICS
The power of graphics to communicate can be considered as being due to the ability to create symbols with deep semantic meaning. Pictures of faces, structures, cross-sections, etc. are perceived as having few, but significant, symbols and convey complex concepts. If we trust the rule that human short-term memory is limited to 'seven plus or minus two' symbols, then it becomes clear that a representation using fewer symbols is advantageous . At a high level of abstraction a certain amount of information becomes much easier to manipulate conceptually by the viewer at the display terminal than the same information when it is represented by a greater number of simpler symbols. An obvious example is a function represented by a line versus a list of numbers; a more interesting example is the use of Chernov-faces, where multi-dimensional data is represented by the degree of a smile, the roundness of the face, the angle of the eyebrows, etc.; color might add yet more information. A further dimension of information can be added if the display can be varied over time. Significant computing resources are needed to rotate even a simple picture, but the effect of interaction with a three-dimensional image, due to the high rate of information transfer, is spectacular .
BARRIERS TO USING GRAPHICS
Since the community which was able to justify graphics is more specialized, no graphic-oriented software has achieved the broad acceptance of FORTRAN or even of FORMAT statements, or the simple elegance of a PASCAL, although certainly many efforts have been made (e.g. [18,19]). Only some manufacturers, notably CALCOMP, have provided support packages which would allow applications to run on more than one machine type. The lack of standards has inhibited sharing of software and hence scientific progress. Nearly every site which wishes to support graphics begins at the most basic level and develops its own conventions. Systems that use interactive displays for menu selection form a sub-area which has seen some high-level efforts. The design of the menu presentation has been aided by frame-oriented programming languages (e.g. ref. , but frequently these languages are limited to tables of coordinates, text, and references to procedures or successor frames.
A factor which adds to the cost of graphics is the demand on processor time and communication bandwidth when graphics are supported from centralized computers. To drive even a limited number of menu selection terminals at a rate acceptable to physician users requires all resources of powerful computers and high speed (50Kbits/sec) communication lines . It is even more difficult to support real-time graphic input. If the subject to be imaged can move then the image has to be entered into the system at video speeds, which implies that the equivalent of 480*640*8 bits have to be transmitted in 1/30th of a second, that is a rate of 74Mbits/sec. In order to transmit color graphics, two or three times this capability is needed to take care of the three components: luminance. hue, and saturation. Commercial TV broadcast equipment actually uses only an analog bandwidth of 3.5 MHz;to get studio quality, 5-15 MHz are needed. Much less demanding have been input devices such as lightpens, joysticks, trackballs, or tablets, which only enter a point at a time and may move a cursor on the display screen. These devices however collect data at rates which are lower than a typist generates on a keyboard, so that complex images become nearly impossible to construct.
The equipment cost to support graphics is composed of the cost of the buffer memory needed to store graphic data, the actual display cost, the cost of internal and external transmission of the signals, and the cost of miscellaneous, but significant, logic and control functions. At high price levels for memory, it was prohibitive to store an entire image as a frame of discrete points and to use the television technique of raster scan to display the image. Recent dramatic price reductions for semiconductor memory make the frame buffer economically possible. For a very high quality picture within the TV image (for instance 454*576*8 bits), 256K bytes of memory are needed; this amount can now be purchased on circuit cards for as low as $4000. At these rates, it is no longer economical to use an X-Y CRT and a vector generator. For many applications less resolution will be adequate. If for instance no gray scale is required then 32K bytes will suffice. A modular approach to the graphic buffer can hence reduce this cost to match the needs.
1. The initial cost and the maintenance cost of the available TV input and output equipment are relatively low, even for the amateur. Any properly insulated and modified home TV receiver can have its bandwidth extended to display data at 10Mbits/second by defeating the sound trap in the video amplifier. This is just enough to display 640 pixels or 80 characters per line. Monitors specifically designed for data display can be obtained for as little as $200.
2. If the system is also capable of accepting and converting standard TV signals then inexpensive CCTV cameras can be used as input devices. They are available for a few hundred dollars. If a video processor has both full speed input and output capability then cassette video recorders, starting around $1,000, are available to greatly extend the I/O capabilities of digital video systems.
3. If the equipment is sufficiently modular then digital computer- controlled graphic processing equipment can be used as a building block for professional digital TV studio equipment. This will allow more innovation in computer-controlled image generation and processing than is available within the more rigid and costly systems now available to TV stations and laboratories where image data are processed.
'Real-time' image digitizers, or 'frame buffers', which are capable of digitizing black-and-white video information (luminance) in 1 or 2 field times (1/60th or 1/30th of a second), are a recent development. Fast analog to digital conversion techniques have begun to make them possible. Since 8-bit converters are yet quite costly, 4-bit and 6-bit converters should also be available; indeed, 16 or 64 levels are quite satisfactory for many applications. In order not to visibly degrade a video signal in the digitization process, a pixel time of about 75 nanoseconds is necessary. The conversion of color images is not now possible with a single converter in real-time. The resolution in picture elements (pixels) and the number of bits which are necessary to represent their characteristics (luminance, and also hue and saturation for color) are key factors in the cost-vs-quality decision process which determines the design of a frame buffer.
It remains a technical challenge to build a sufficiently fast 8-bit A to D converter to process color images; at least two manufacturers (TRW and AMD) have recently introduced IC's or IC building blocks that can be used for that purpose. Since one frame occurs in 33 milliseconds, the throughput speed needed for 256K color pixels per image is 23.6 Mbytes per second. Such a high speed of input signal conversion and the corresponding required size of memory present a potential bottleneck.
In order to select a point or a subsection of an image it is necessary to be able to identify a pixel on the screen. While it is possible to program a cursor in the image that moves on commands from a terminal or joystick, much more rapid interaction is provided by a lightpen. A lightpen is triggered by the writing signal on the CRT face, and the actual pixel address at that instance indicates the pixel pointed at by the lightpen.
Since we wish to assure a low entry cost for a complete system it is desirable to have a basic computer-assisted television system complete on as few boards as possible. Such a basic system for us is comprised of:
1. a four-bit A to D converter, capable of full speed video input acquisition;
2. two banks of graphic buffer memory (32 K bytes total) to support the resulting maximum data rate;
3. a complete set of image representation choices to support various combinations of line resolution and gray scale;
4. a video output capability which matches the video input;
5. an interface to a popular microprocessor bus in order to provide access for processing of the graphic image and control the operation of the graphic system; and
6. a number of important features such as lightpen and photography support.
The system can either accept video synchronization (sync) from external video components, or generate it for both internal and external use. The system can also extract sync from an incoming video signal. If standard negative composite sync is available, it may be fed into the board separately. In both cases, the system is completely synchronized to an external source. The sync signals are also made available for external use.
It is important to realize that, according to NTSC standards, the luminance information reaches the CRT with a relatively wide bandwidth, whereas the chrominance information (hue and saturation) is limited to about 1 MHz. Hence, on a color monitor, good resolution can be achieved by acting only on the luminance bit from one pixel to the next one; on each TV line; the three chrominance bits should be allowed to change slower in order to obtain best results, typically every other or every third pixel only. In other words, fine detail can be obtained within a given color by using its intensity to carry the information.
ADDITIONAL BUILDING BLOCKS