ToolBoxes

Common HTML Encoding Bugs and How to Fix Them

May 28, 20267 min read

Why Encoding Bugs Happen

HTML encoding bugs occur when characters are transformed incorrectly between storage, transmission, and display. The root cause is almost always a mismatch: the bytes were encoded with one character encoding but decoded with another.

These bugs range from cosmetic annoyances (curly quotes showing as â€") to security vulnerabilities (unescaped output enabling XSS). Understanding the common patterns helps you spot and fix them quickly.

Bug 1: Mojibake — The Garbled Text Problem

Symptom: Smart quotes, em dashes, and accented characters render as gibberish like â€", “, é.

Cause: UTF-8 encoded bytes are being interpreted as Latin-1 (ISO-8859-1). UTF-8 uses multiple bytes for non-ASCII characters. When a two-byte UTF-8 sequence is read as Latin-1, each byte becomes a separate character.

UTF-8 bytes:    E2 80 94  (em dash —)
Latin-1 decode: â  €  "   (mojibake)

Fix: Ensure your document declares UTF-8 encoding:

<meta charset="UTF-8">

Also verify that your server sends the correct Content-Type header:

Content-Type: text/html; charset=UTF-8

The HTTP header takes precedence over the meta tag. If the server declares Latin-1, the meta tag won't save you.

Bug 2: Double Escaping

Symptom: Users see literal entity text like &amp;lt; instead of <, or &amp; instead of &.

Cause: The text was entity-encoded twice. The first pass converts & to &amp;. The second pass converts the & in &amp; to &amp;amp;.

// Double escaping
function encodeTwice(text) {
  let first = text.replace(/&/g, '&amp;')
  let second = first.replace(/&/g, '&amp;')  // catches the & in &amp;
  return second
}
encodeTwice('Tom & Jerry')  // "Tom &amp;amp; Jerry"

Fix: Encode exactly once, at the output boundary. If you're using a template engine (Vue, React, Twig), it handles encoding automatically. Don't pre-encode values before passing them to templates.

Bug 3: Missing Entity Encoding (XSS Risk)

Symptom: User input containing <script> tags executes as code instead of displaying as text.

Cause: User-supplied data was inserted into HTML without entity encoding.

Fix: Always encode user output:

function escapeHtml(text) {
  const map = { '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&apos;' }
  return text.replace(/[&<>"']/g, c => map[c])
}

Modern frameworks escape by default. Vue's {{ }} syntax and React's JSX auto-encode. Only v-html and dangerouslySetInnerHTML bypass this protection.

Bug 4: Charset Declaration in the Wrong Place

Symptom: Characters display correctly in some browsers but not others, or work locally but break in production.

Cause: The <meta charset> tag appears after the first 1024 bytes of the document. The browser must guess the encoding for those initial bytes, and its guess may be wrong.

Fix: Place the charset declaration as early as possible — ideally the first child of <head>:

<!DOCTYPE html>
<html>
<head>
  <meta charset="UTF-8">
  <title>My Page</title>
</head>

Bug 5: Inconsistent Encoding Across Layers

Symptom: Data looks correct in the database but displays incorrectly on the page, or works in one environment but not another.

Cause: Different parts of the stack use different encodings. The database stores Latin-1, the application processes UTF-8, and the server header declares something else.

Fix: Audit every layer:

LayerCheck
DatabaseSHOW VARIABLES LIKE 'character_set%'; (MySQL)
ConnectionSET NAMES utf8mb4;
ApplicationSource files saved as UTF-8
TemplateMeta charset declared
HTTPContent-Type: text/html; charset=UTF-8 header

Use utf8mb4 (not utf8) in MySQL to support full Unicode including 4-byte characters like emojis.

Symptom: A mysterious blank space or  appears at the top of the page. PHP complains about "headers already sent."

Cause: The file was saved with a UTF-8 Byte Order Mark (BOM) — three invisible bytes (EF BB BF) at the start of the file.

Fix: Save files as "UTF-8 without BOM." Most editors have this option. For existing files, strip the BOM:

# Remove BOM from a file
sed -i '1s/^\xEF\xBB\xBF//' file.html

Prevention Checklist

  • Set <meta charset="UTF-8"> as the first element in <head>
  • Verify the server sends charset=UTF-8 in the Content-Type header
  • Save all source files as UTF-8 without BOM
  • Encode user output exactly once, at the HTML boundary
  • Use utf8mb4 for MySQL databases
  • Test with non-ASCII characters: accented letters, emojis, CJK characters

Key Takeaways

  • Mojibake happens when UTF-8 bytes are decoded as Latin-1 — fix the charset declaration
  • Double escaping produces visible entity text like &amp;amp; — encode exactly once
  • Unescaped output enables XSS — always encode user data before rendering
  • Place <meta charset> within the first 1024 bytes of the document
  • Audit every layer of your stack for encoding consistency
  • Save files as UTF-8 without BOM to avoid invisible byte issues

Try It Yourself

Encode and decode HTML entities instantly with our free HTML Entity Encoder/Decoder. Paste text with encoding issues and get correctly encoded output in one click.