By LinuxHPC.org and Cluster Resources
| Unix |
|
Unix or UNIX is a computer operating system originally developed in the 1960s and 1970s by a group of AT&T Bell Labs employees including Ken Thompson, Dennis Ritchie, and Douglas McIlroy. Today's Unix systems are split into various branches, developed over time by AT&T, several other commercial vendors, as well as several non-profit organizations, such as contributors to the GNU project.
Unix was designed to be portable, multi-tasking and multi-user in a time-sharing configuration. The Unix systems are characterized by various concepts: plain text files, command line interpreter, hierarchical file system, treating devices and certain types of inter-process communication as files, etc. In software engineering, Unix is mainly noted for its use of the C programming language and for the Unix philosophy.
The present owner of the UNIX trademark is The Open Group, while the present claimants on the rights to the UNIX source code are SCO Group and Novell (an issue that is currently being decided in court). Only systems fully compliant with and certified to the Single UNIX Specification qualify as "UNIX" (others are called "UNIX system-like" or Unix-like).
During the late 1970s and early 1980s, Unix's influence in academic circles led to massive adoption (particularly of the BSD variant, originating from the University of California, Berkeley) of Unix by commercial startups, the most notable of which is Sun Microsystems.
Sometimes, Traditional Unix may be used to describe a Unix or GNU operating system that has the characteristics of either Version 7 Unix or UNIX System V.
Overview
Unix operating systems are widely used in both servers and workstations. The Unix environment and the client/server program model were essential elements in the development of the Internet and the reshaping of computing as centered in networks rather than in individual computers. GNU/Linux, a Unix inspired operating system available for free and from commercial distributors, is gaining popularity as an alternative to proprietary operating systems like Microsoft Windows.
Unix is written in C. Both Unix and C were developed by AT&T and distributed to government and academic institutions, causing it to be ported to a wider variety of machine families than any other operating system. As a result, Unix became synonymous with "open systems."
Standards
Beginning in the late 1980s, an open operating system standardization effort known as POSIX provided a common baseline for all operating systems; IEEE based POSIX around the structure of the Unix system. At around the same time a separate but very similar standard, the Single UNIX Specification, was also produced by the Open Group. Starting in 1998 these two standards bodies began work on merging the two standards, and the latest revisions of both are in fact identical.
In an effort towards compatibility, several Unix system vendors agreed on SVR4's ELF format as standard for binary and object code files. The common format allows substantial binary compatibility among Unix systems operating on the same CPU architecture.
The directory layout of some systems, particularly on Linux, is defined by the Filesystem Hierarchy Standard. This type of standard however is controversial among many, and even within the Linux community adoption is far from universal.
Components
The Unix system is composed of several components that are normally packaged together. By including -- in addition to the "kernel" of an operating system -- the development environment, libraries, documents, and the portable, modifiable source-code for all of these components, Unix was a self-contained software system. This was one of the key reasons it emerged into an important teaching and learning tool and had such a broad influence.
Inclusion of these components did not make the system large -- the original V7 Unix distribution, consisting of copies of all of the compiled binaries plus all of the source code and documentation occupied less than 10Mb, and arrived on a single 9-track magtape. The printed documentation was contained in two fairly thin books.
The names and filesystem locations of the Unix components has changed substantially across the history of the system. Nonetheless, the V7 implementation is considered by many to have the canonical early structure:
- Kernel -- originally found in /usr/sys, and composed of several sub-components:
- conf -- originally found in /usr/sys/conf, and composed of configuration and machine-dependent parts, often including boot code
- dev -- Device drivers (originally /usr/sys/dev) for control of hardware (and sometimes pseudo-hardware)
- sys -- The "kernel" of the operating system, handling memory management, system calls, etc
- h (or include) -- Header files, generally defining key interfaces within the system, and important system-specific invariables
- Development Environment -- Most implementations of Unix contained a development environment sufficient to recreate the system from source code. The development environment included:
- cc -- The C language compiler
- as -- The machine-language assembler for the machine
- ld -- The linking loader for combining object files
- lib -- Libraries. Originally libc, the C runtime library, was the primary library, but there have always been additional libraries for (e.g.) floating-point emulation (libm) or a database implementation. V7 Unix introduced the first consistent "Standard I/O" library stdio. Later implementations multiplied the number and type of libraries significantly.
- include -- Header files for software development, defining standard interfaces and system invariants
- Other (secondary) languages -- V7 Unix contained a Fortran-77 compiler, and other versions and implementations have or now contain many other language compilers and toolsets.
- ... and a number of other tools, including an object-code archive manager (ar), symbol-table lister, compiler-development tools (e.g. yacc), make, and debugging tools.
- Commands -- Most Unix implementation make little distinction between commands (user-level programs) for system operation and maintenance (e.g. cron), commands of general utility (e.g. grep), and more general-purpose applications such as the text formatting and typesetting package. Nonetheless, some major categories are:
- sh -- The Shell, the primary user-interface on Unix before window systems appeared, and the center of the command environment. To degrees that varied in different shell implementations, external programs (such as expr) were relied on by the shell.
- Utilities -- the core of the Unix command set, including ls, grep, find and many others. This category could be subcategorized:
- System utilities -- such as mkfs, fsck, and many others; and
- User utilities -- passwd, kill, and others
- Runoff -- Unix systems never lost their heritage as early document preparation and typesetting systems, and included many related programs such as troff, tbl, neqn, refer, plot
- Communications -- early Unix systems contained no inter-system communication, but did include the inter-user communication programs mail and talk. V7 introduced the early inter-system communication system UUCP, and systems beginning with the BSD release included TCP/IP utilities
- Documentation -- While not strictly part of the operating system, Unix was unique in its time for including all of its documentation online in machine-readable form. The documentation included:
- man -- Manual pages for each command, library component, system call, header file, etc
- doc -- Longer documents detailing major subsystems, such as the C language, troff, and other systems.
Impact
The Unix system had a great impact on other operating systems. Unix has been called "the most important operating system you may never use."
Following the lead of Multics, it was written in high level language as opposed to assembly (assembly had been necessary for achieving acceptable performance on early computers).
It had a drastically simplified file model compared to many contemporary operating systems. The file system hierarchy contained machine services and devices (such as printers, terminals, or disk drives), providing a superficially uniform interface, but at the expense of requiring indirect mechanisms such as ioctl and mode flags to access features of the hardware that did not fit the simple "stream of bytes" model.
Unix also popularized the hierarchical file system with arbitrarily nested subdirectories, originally introduced by Multics. Other common operating systems of the era had ways to divide a storage device into multiple directories or sections, but they were a fixed number of levels and often only one level. The major proprietary operating systems all added recursive subdirectory capabilities also patterned after Multics. DEC's RSTS programmer/project hierarchy evolved into VMS directories, CP/M's volumes evolved into MS-DOS 2.0+ subdirectories, and HP's MPE group.account hierarchy and IBM's System 36 and OS/400 library systems were folded into broader POSIX file systems.
Making the command interpreter an ordinary user-level program, with additional commands provided as separate programs, was another Multics innovation popularized by Unix. The Unix shell used the same language for interactive commands as for scripting (shell scripts -- there was no separate job control language, like IBM's JCL for example). Since the shell and OS commands were "just another program", the user could choose (or even write) his/her own shell. Finally, new commands could be added without recompiling the shell. Unix's innovative command-line syntax for creating chains of producer-consumer processes (pipes) made a powerful programming technique (coroutines) widely available.
A fundamental simplifying assumption of Unix was its focus on ASCII text for 100% of its I/O package and the assumption that the machine word was a multiple of 8 bits in size. There were no "binary" editors in the original version of Unix - the entire system was configured using text shell commands and the least and greatest common denominator in the I/O system is the text byte - unlike "record-based" file systems in other computers. The focus on text for representing "everything" made Unix pipes useful. The focus on text and 8-bit bytes made the system far more scalable and portable than other systems. Over time text-based applications have also won in application areas, such as printing languages (PostScript - not InterPress - an earlier effort by the same people), and when feasible, at the application layer of the Internet Protocols, i.e. Telnet, FTP, SMTP, HTTP, SIP, XML, etc.
Unix popularised a syntax for regular expressions that found much wider use. The Unix programming interface became the basis for a standard operating system interface (POSIX, see above).
The C programming language, now ubiquitous in systems and applications programming, originated under Unix, and spread more quickly than Unix. The C language was the first agnostic language that did not attempt to force a coding style upon the programmer (e.g. support for 3 types of loops and all types of parameter passing.) The C language was the first programming language to access a computer's full instruction set (e.g. masking, shifting, auto increment, auto decrement, jump tables, pointers.) However, the unsafeness of C leads to problems such as buffer overflows from C library functions such as gets() and scanf(), which are behind many notorious bugs, including one exploited by the Morris worm.
Early Unix developers were important in bringing the theory of software modularity and re-use into engineering practice.
Unix provided the TCP/IP networking protocol on relatively inexpensive computers, which later resulted in the Internet explosion of world-wide real-time connectivity. This quickly exposed several major security holes in the Unix architecture, kernel, and system utilities.
Over time, the leading developers of Unix (and programs that ran on it) developed a set of cultural norms for developing software, norms which became as important and influential as the technology of Unix itself. See Unix philosophy for more information.
2038
Unix stores time values as the number of seconds from midnight January 1, 1970 (the "Unix Epoch") in variables of size time_t, historically defined as "signed 32-bit integer". On January 19, 2038, the current time will roll over from a zero followed by 31 ones to a one followed by 31 zeros, which will reset time to the year 1901 or 1970, depending on implementation. As many applications use OS library routines for date calculations, the impact of this could be felt much earlier than 2038; for instance, 30-year mortgages may be calculated incorrectly beginning in the year 2008.
Some Unix versions have already addressed this. For example, in Solaris on 64-bit systems, time_t is 64 bits long, meaning that the OS itself and 64-bit applications will correctly handle dates through and beyond the year 2920. Existing 32-bit applications using a 32-bit time_t continue to work on 64-bit Solaris systems but are still prone to the 2038 problem.
Free Unix-like operating systems
In 1983, Richard Stallman announced the GNU project, an ambitious effort to create a free software Unix-like system; "free" in that everyone who received a copy would be free to use, study, modify, and redistribute it. GNU's goal was achieved in 1992. Its own kernel development project, GNU Hurd, had not produced a working kernel, but a compatible kernel called Linux was released as free software in 1992 under the GNU General Public License. The combination of the two is frequently referred to simply as "Linux", although the Free Software Foundation and some Linux distributions, such as Debian GNU/Linux, use the combined term GNU/Linux. Work on GNU Hurd continues, although very slowly.
In addition to their use in the Linux operating system, many GNU packages — such as the GNU Compiler Collection (and the rest of the GNU toolchain), the GNU C library and the GNU core utilities — have gone on to play central roles in other free Unix systems as well.
Linux distributions, comprising Linux and large collections of compatible software have become popular both with hobbyists and in business. Popular distributions include Red Hat Linux, SUSE Linux, Mandriva Linux, Ubuntu, Debian GNU/Linux and Gentoo.
A free derivative of BSD Unix, 386BSD, was also released in 1992 and led to the NetBSD and FreeBSD projects. With the 1994 settlement of a lawsuit that UNIX Systems Laboratories brought against the University of California and Berkeley Software Design Inc. (USL v. BSDi), it was clarified that Berkeley had the right to distribute BSD Unix — for free, if it so desired. Since then, BSD Unix has been developed in several different directions, including the FreeBSD, OpenBSD, NetBSD, PC-BSD, and DragonFly BSD variants.
Linux and the BSD kin are now rapidly occupying the market traditionally occupied by proprietary UNIX operating systems, as well as expanding into new markets such as the consumer desktop and mobile and embedded devices. A measure of this success may be seen when Apple Computer sought out a new foundation for its Macintosh operating system: it chose NEXTSTEP, an operating system developed by NeXT with a freely redistributable core operating system, renamed Darwin after Apple acquired it. It was based on the BSD family and the Mach kernel. The deployment of Darwin BSD Unix in Mac OS X makes it, according to a statement made by an Apple employee at a USENIX conference, the most widely-used Unix-based system in the desktop computer market. Due to the modularity of the Unix design, sharing bits and pieces is relatively common; consequently, most or all Unix and Unix-like systems include at least some BSD code, and modern BSDs also typically include some GNU utilities in their distribution, so Apple's combination of parts from NeXT and FreeBSD with Mach and some GNU utilities has precedent.
All text used in this article is available under the GNU Free Documentation License. It uses material from the Wikipedia article "Unix".