Unicode
By Unicode Consortium
Unicode is a universal character encoding standard that assigns a unique numeric code point to virtually every character used in the world's writing systems, along with symbols, emoji, and technical marks. It exists to solve the problem of…
Definition
Unicode is a universal character encoding standard that assigns a unique numeric code point to virtually every character used in the world's writing systems, along with symbols, emoji, and technical marks. It exists to solve the problem of incompatible regional character sets that made text unreliable to exchange across languages and platforms, giving software one consistent way to represent, store, and interchange text regardless of script or language.
Overview
Before Unicode, computers relied on a patchwork of national and vendor-specific encodings, such as ASCII for English or various code pages for other regions, where the same numeric byte value could mean a different character depending on which encoding was assumed. This made multilingual documents fragile and often produced garbled text, known informally as mojibake, whenever software guessed the wrong encoding. Unicode was created to give every character in every major writing system a single, stable identity that any Unicode-aware system can interpret the same way. Mechanically, Unicode defines a code space of over a million possible code points, each identified by a hexadecimal number such as U+0041 for the Latin letter A, organized into blocks by script or purpose. The standard itself does not dictate how those numbers are stored as bytes; that job belongs to encoding schemes such as UTF-8, UTF-16, and UTF-32, which map code points to byte sequences of varying length. UTF-8 has become the dominant choice on the web and in most modern software because it is backward-compatible with ASCII for English text while still representing any Unicode character using between one and four bytes. Unicode differs from earlier character sets like ASCII or ISO-8859 variants by being a single superset intended to cover all scripts rather than one region's alphabet, and it differs from a font in that it defines what a character is, not how it is drawn. It is closely related to formats like VTT, which rely on Unicode text for captions, and to programming languages and engines such as V8, which must implement Unicode-aware string handling internally to process JavaScript source and data correctly. In practice, essentially all modern operating systems, programming languages, databases, and file formats use Unicode as their text representation, whether through UTF-8 on disk and over networks or UTF-16 in memory for platforms like Windows and Java. Developers rely on it for everything from rendering emoji and right-to-left scripts to normalizing user input and comparing strings across languages. The standard is not without complexity: several code point sequences can render as the visually identical glyph, a challenge called normalization, and combining characters, variation selectors, and bidirectional text add real implementation difficulty that trips up naive string-length or substring logic. Sorting, case conversion, and width calculations are locale-sensitive in ways that a single numeric encoding cannot fully resolve on its own, so applications with serious internationalization needs still require dedicated libraries built on top of Unicode rather than treating it as a complete solution by itself.
Specification
- Single code space covering nearly all of the world's writing systems
- Separation between abstract code points and concrete byte encodings
- UTF-8, UTF-16, and UTF-32 encoding forms for different storage needs
- Backward compatibility with ASCII through the UTF-8 encoding
- Normalization forms for resolving visually equivalent character sequences
- Support for combining characters, emoji, and bidirectional text
- Maintained and versioned by the Unicode Consortium standards body