What is 16-bit Unicode (UTF-16)?
How UTF-16 encodes text, how it differs from UTF-8, and why it matters for the length and cost of your SMS.
16-bit Unicode, or Unicode Transformation Format (UTF-16), is a method of encoding character data, capable of encoding 1,112,064 possible characters in Unicode. It uses one or two 16-bit units for each character, which means two or four bytes.
UTF-16 encodes characters into specific binary sequences using one or two 16-bit sequences. Three encoding schemes map code points to 8-bit, or octet, sequences around the basic 16-bit model: UTF-16, UTF-16BE (big-endian) and UTF-16LE (little-endian).
Key takeaways:
- UTF-16 encodes each Unicode character as one or two 16-bit units, which means two or four bytes.
- Most common characters take two bytes. Others, including many emoji, take four through a surrogate pair.
- UTF-8 starts at one byte per character and UTF-16 starts at two.
- “Unicode 16” is a version of the Unicode Standard, not a reference to 16 bits.
- In SMS, one non-GSM character switches the whole message to Unicode, which allows 70 characters instead of 160.
How does UTF-16 work?
Unicode assigns every character a code point between U+0000 and U+10FFFF, which is 1,114,112 code points in total. UTF-16 reserves 2,048 of them as surrogates, which leaves the 1,112,064 valid code points it can encode.
- Basic Multilingual Plane (BMP): characters up to U+FFFF, which cover most common scripts, use a single 16-bit unit, or two bytes.
- Supplementary characters: everything above U+FFFF, including many emoji, uses two 16-bit units called a surrogate pair, or four bytes.
What is the difference between UTF-16 and UTF-8?
UTF-16 encodes a Unicode character into a string of either two or four bytes. So the distinction between UTF-16 and UTF-8 is visible in their names. In UTF-8, the smallest representation of a character is one byte, or eight bits. For UTF-16, the smallest representation of a character is two bytes, or 16 bits.
Both can represent every Unicode character. They simply pack the same characters into bytes in different ways.
Is Unicode 16 the same as UTF-16?
No. Unicode 16 is a version number of the Unicode Standard, which defines the characters and their code points. UTF-16 is one way of encoding those characters into bytes.
UTF-16 and UCS-2
UCS-2 is an older, fixed-width 16-bit encoding that covers only the Basic Multilingual Plane. UTF-16 encodes those characters the same way and adds surrogate pairs for the rest. SMS documentation, including ours, usually calls this encoding Unicode (UCS-2).
UTF-16 and SMS length
This is where 16-bit Unicode matters for messaging. A message that uses only the GSM-7 alphabet fits 160 characters in one SMS. If it includes even one character outside that alphabet, such as an emoji, an Arabic or Cyrillic letter or an Asian script, the entire message is sent as Unicode, and the limit drops to 70.
- GSM-7: 160 characters in a single SMS, or 153 per segment when concatenated.
- Unicode: 70 characters in a single SMS, or 67 per segment when concatenated. Two segments hold up to 134 characters and three hold up to 201.
Every extra segment is billed, so one emoji can turn a one-part message into two. Learn more in our guide to the SMS character limit.
Infobip is a communications platform that supports both GSM and Unicode messages globally, although some carriers and devices may not support every Unicode character range. If you want to stay within 160 characters, Infobip’s transliteration option can replace unsupported characters with similar GSM-7 characters.
Conclusion
16-bit Unicode, or UTF-16, stores every Unicode character in one or two 16-bit units. That is what makes it flexible enough for any script and emoji, and also why Unicode SMS messages are shorter than GSM-7 ones.
FAQs
16-bit Unicode, or Unicode Transformation Format (UTF-16), is a character encoding that can represent all 1,112,064 valid Unicode code points. It encodes each character as one or two 16-bit units, which means two or four bytes. Most common characters take two bytes, while rarer ones, including many emoji, take four.
No. Unicode 16 is a version number of the Unicode Standard, which defines the characters and their code points. UTF-16 is a way of encoding those characters into bytes. The two names look similar, but one refers to a release of the standard and the other to a 16-bit encoding method.
The difference starts with the smallest unit. UTF-8 uses one byte, or eight bits, as its smallest representation of a character, while UTF-16 uses two bytes, or 16 bits. UTF-8 then grows to as many as four bytes, and UTF-16 grows to four bytes using two 16-bit units.
UTF-16 uses either two or four bytes per character. Characters in the Basic Multilingual Plane, which covers most common scripts, take one 16-bit unit, or two bytes. Characters beyond it, including many emoji, take two 16-bit units called a surrogate pair, which is four bytes.
UCS-2 is an older, fixed-width 16-bit encoding that can only represent characters in the Basic Multilingual Plane. UTF-16 encodes those characters the same way but adds surrogate pairs, so it can represent every Unicode code point. SMS documentation often refers to its Unicode encoding as UCS-2.
A GSM-7 message fits 160 characters, but Unicode uses 16 bits per character, so a single SMS holds only 70. One character outside the GSM-7 alphabet, such as an emoji or a non-Latin letter, switches the whole message to Unicode. Longer messages split into segments of 67 characters.