[{"data":1,"prerenderedAt":477},["ShallowReactive",2],{"guide-unicode-utf8-binary":3},{"id":4,"title":5,"body":6,"date":467,"description":468,"extension":469,"meta":470,"navigation":471,"path":472,"readingTime":473,"seo":474,"stem":475,"__hash__":476},"guides\u002Fguides\u002Funicode-utf8-binary.md","Unicode, UTF-8, and Binary Encoding",{"type":7,"value":8,"toc":453},"minimark",[9,13,18,26,104,107,111,114,183,190,195,266,270,337,341,344,371,382,386,422,426,440,444],[10,11,12],"p",{},"Every character you see on screen — from the letter A to a snowman emoji — is ultimately stored as binary data. The mapping between human-readable characters and binary bits is defined by encoding standards, and UTF-8 is the dominant encoding on the web today. Understanding how Unicode code points translate to UTF-8 byte sequences helps you debug mojibake, validate input, and work with binary protocols.",[14,15,17],"h2",{"id":16},"unicode-code-points","Unicode Code Points",[10,19,20,21,25],{},"Unicode assigns every character a unique integer called a code point, written as ",[22,23,24],"code",{},"U+XXXX"," where XXXX is a hexadecimal number. The Unicode standard currently defines over 149,000 characters across scripts, symbols, and emoji.",[27,28,29,45],"table",{},[30,31,32],"thead",{},[33,34,35,39,42],"tr",{},[36,37,38],"th",{},"Character",[36,40,41],{},"Code Point",[36,43,44],{},"Decimal",[46,47,48,60,71,82,93],"tbody",{},[33,49,50,54,57],{},[51,52,53],"td",{},"A",[51,55,56],{},"U+0041",[51,58,59],{},"65",[33,61,62,65,68],{},[51,63,64],{},"é",[51,66,67],{},"U+00E9",[51,69,70],{},"233",[33,72,73,76,79],{},[51,74,75],{},"€",[51,77,78],{},"U+20AC",[51,80,81],{},"8364",[33,83,84,87,90],{},[51,85,86],{},"中",[51,88,89],{},"U+4E2D",[51,91,92],{},"20013",[33,94,95,98,101],{},[51,96,97],{},"😊",[51,99,100],{},"U+1F60A",[51,102,103],{},"128522",[10,105,106],{},"Code points range from U+0000 to U+10FFFF, giving a maximum of 1,114,112 possible values. This range is divided into planes — the first plane (U+0000 to U+FFFF) is called the Basic Multilingual Plane (BMP) and covers most everyday characters.",[14,108,110],{"id":109},"utf-8-encoding-rules","UTF-8 Encoding Rules",[10,112,113],{},"UTF-8 is a variable-length encoding: each code point is represented by 1 to 4 bytes. The number of bytes depends on the code point's value:",[27,115,116,129],{},[30,117,118],{},[33,119,120,123,126],{},[36,121,122],{},"Code Point Range",[36,124,125],{},"Bytes",[36,127,128],{},"Binary Pattern",[46,130,131,144,157,170],{},[33,132,133,136,139],{},[51,134,135],{},"U+0000 – U+007F",[51,137,138],{},"1",[51,140,141],{},[22,142,143],{},"0xxxxxxx",[33,145,146,149,152],{},[51,147,148],{},"U+0080 – U+07FF",[51,150,151],{},"2",[51,153,154],{},[22,155,156],{},"110xxxxx 10xxxxxx",[33,158,159,162,165],{},[51,160,161],{},"U+0800 – U+FFFF",[51,163,164],{},"3",[51,166,167],{},[22,168,169],{},"1110xxxx 10xxxxxx 10xxxxxx",[33,171,172,175,178],{},[51,173,174],{},"U+10000 – U+10FFFF",[51,176,177],{},"4",[51,179,180],{},[22,181,182],{},"11110xxx 10xxxxxx 10xxxxxx 10xxxxxx",[10,184,185,186,189],{},"The leading bits of the first byte tell a decoder how many bytes follow. Continuation bytes all start with ",[22,187,188],{},"10",", making resynchronization possible even if you start reading mid-stream.",[191,192,194],"h3",{"id":193},"encoding-example-u20ac","Encoding Example: € (U+20AC)",[196,197,198,208,211,256],"ol",{},[199,200,201,202,204,205],"li",{},"Code point ",[22,203,78],{}," = decimal 8364 = binary ",[22,206,207],{},"0010 0000 1010 1100",[199,209,210],{},"This falls in the 3-byte range (U+0800–U+FFFF)",[199,212,213,214,216,217],{},"Distribute the 16 bits into the pattern ",[22,215,169],{},":\n",[218,219,220,234,245],"ul",{},[199,221,222,223,226,227,230,231],{},"First byte: ",[22,224,225],{},"1110"," + ",[22,228,229],{},"0010"," = ",[22,232,233],{},"11100010",[199,235,236,237,226,239,230,242],{},"Second byte: ",[22,238,188],{},[22,240,241],{},"000010",[22,243,244],{},"10000010",[199,246,247,248,226,250,230,253],{},"Third byte: ",[22,249,188],{},[22,251,252],{},"101100",[22,254,255],{},"10101100",[199,257,258,259,262,263],{},"UTF-8 bytes: ",[22,260,261],{},"0xE2 0x82 0xAC"," → binary: ",[22,264,265],{},"11100010 10000010 10101100",[191,267,269],{"id":268},"encoding-example-u1f60a","Encoding Example: 😊 (U+1F60A)",[196,271,272,280,283,332],{},[199,273,201,274,276,277],{},[22,275,100],{}," = decimal 128522 = binary ",[22,278,279],{},"0001 1111 0110 0000 1010",[199,281,282],{},"This falls in the 4-byte range (U+10000–U+10FFFF)",[199,284,285,286,216,288],{},"Distribute 21 bits into the pattern ",[22,287,182],{},[218,289,290,301,311,321],{},[199,291,222,292,226,295,230,298],{},[22,293,294],{},"11110",[22,296,297],{},"000",[22,299,300],{},"11110000",[199,302,236,303,226,305,230,308],{},[22,304,188],{},[22,306,307],{},"011111",[22,309,310],{},"10011111",[199,312,247,313,226,315,230,318],{},[22,314,188],{},[22,316,317],{},"100000",[22,319,320],{},"10100000",[199,322,323,324,226,326,230,329],{},"Fourth byte: ",[22,325,188],{},[22,327,328],{},"001010",[22,330,331],{},"10001010",[199,333,258,334],{},[22,335,336],{},"0xF0 0x9F 0x98 0x8A",[14,338,340],{"id":339},"why-utf-8-won-the-web","Why UTF-8 Won the Web",[10,342,343],{},"UTF-8 has several properties that made it the de facto standard:",[218,345,346,353,359,365],{},[199,347,348,352],{},[349,350,351],"strong",{},"ASCII compatibility."," The first 128 code points (U+0000–U+007F) encode as single bytes identical to ASCII. Any valid ASCII text is also valid UTF-8, which eliminates migration effort for English-centric systems.",[199,354,355,358],{},[349,356,357],{},"Self-synchronization."," Because leading bytes and continuation bytes have distinct bit patterns, a UTF-8 decoder can find the start of the next character from any position in a byte stream — even after data corruption.",[199,360,361,364],{},[349,362,363],{},"No endianness issues."," Unlike UTF-16 and UTF-32, UTF-8 is byte-oriented and does not require a byte-order mark (BOM).",[199,366,367,370],{},[349,368,369],{},"Space efficiency for Latin text."," English and other Latin-script languages use 1 byte per character, compared to 2 bytes minimum for UTF-16.",[10,372,373,374,377,378,381],{},"As of 2026, over 98% of web pages use UTF-8 encoding. The ",[22,375,376],{},"Content-Type: text\u002Fhtml; charset=utf-8"," header and the ",[22,379,380],{},"\u003Cmeta charset=\"utf-8\">"," tag are standard practice.",[14,383,385],{"id":384},"common-pitfalls","Common Pitfalls",[218,387,388,402,408],{},[199,389,390,393,394,397,398,401],{},[349,391,392],{},"Confusing code points with bytes."," JavaScript's ",[22,395,396],{},"String.length"," counts UTF-16 code units, not Unicode code points. The string ",[22,399,400],{},"\"😊\".length"," returns 2, not 1, because the emoji is encoded as a surrogate pair in UTF-16.",[199,403,404,407],{},[349,405,406],{},"Overlong encodings."," An overlong encoding uses more bytes than necessary to represent a code point (e.g., encoding U+0041 as two bytes). The UTF-8 standard forbids overlong encodings, and conformant decoders reject them as a security measure.",[199,409,410,413,414,417,418,421],{},[349,411,412],{},"Invalid byte sequences."," A lone continuation byte (",[22,415,416],{},"0x80","–",[22,419,420],{},"0xBF",") without a leading byte, or a leading byte not followed by enough continuation bytes, makes the stream invalid. How decoders handle this — replacing with U+FFFD (�) or raising an error — varies by implementation.",[14,423,425],{"id":424},"key-takeaways","Key Takeaways",[218,427,428,431,434,437],{},[199,429,430],{},"Unicode code points are abstract integer identifiers; UTF-8 is one way to encode them as binary bytes.",[199,432,433],{},"UTF-8 uses 1–4 bytes per character, with the byte count encoded in the leading bits of the first byte.",[199,435,436],{},"ASCII compatibility and self-synchronization are the key reasons UTF-8 dominates the web.",[199,438,439],{},"Confusing code points, UTF-16 code units, and bytes is the root cause of most string-handling bugs.",[14,441,443],{"id":442},"try-it-yourself","Try It Yourself",[10,445,446,447,452],{},"Want to see the binary representation of any text? Paste a string into the ",[448,449,451],"a",{"href":450},"\u002Ftools\u002Ftext-binary","Text to Binary Converter"," to visualize its UTF-8 byte sequences and binary encoding in real time.",{"title":454,"searchDepth":455,"depth":455,"links":456},"",2,[457,458,463,464,465,466],{"id":16,"depth":455,"text":17},{"id":109,"depth":455,"text":110,"children":459},[460,462],{"id":193,"depth":461,"text":194},3,{"id":268,"depth":461,"text":269},{"id":339,"depth":455,"text":340},{"id":384,"depth":455,"text":385},{"id":424,"depth":455,"text":425},{"id":442,"depth":455,"text":443},"2026-05-28","How Unicode characters map to UTF-8 byte sequences and their binary representations.","md",{"immutable":471},true,"\u002Fguides\u002Funicode-utf8-binary",6,{"title":5,"description":468},"guides\u002Funicode-utf8-binary","T3VOX_PE0INTrOn-DGhcclXLj5UGEKruSIZGcjIbFvo",1780401337816]