Characters and encodings

Understand the idea

An encoding maps text to bytes; Unicode code points identify abstract characters. A visible character can contain several code points. Define the representation your algorithm counts or compares.

Work through an example

A precomposed é and an e followed by a combining accent can look identical yet compare unequal. UTF-8 uses variable-width byte encodings.

Time and space complexity

Code-point scan O(n); encoding and normalization can allocate O(n) storage.

Why it works

Indexing and equality must follow the same chosen representation.

Common mistakes

• Byte length need not equal code-point count.
• casefold can change length.

Chapter lab · Python implementation

These examples operate on Python Unicode code points. A user-visible character can span several code points; normalization and grapheme segmentation need separate policies. The lab compares mirror positions without allocating a reversed copy. For every text problem, state whether case, punctuation, and whitespace matter.

This shared lab connects the chapter's topics. Follow the current lesson's worked example first, then trace the complete implementation. Additional algorithms described in topic exercises are variations you can implement independently.

def palindrome(text):
    left, right = 0, len(text)-1
    while left < right:
        if text[left] != text[right]:
            return False
        left += 1
        right -= 1
    return True

def frequencies(text):
    counts = {}
    for char in text:
        counts[char] = counts.get(char,0)+1
    return counts

Run and test the lab

Copy the implementation and these assertions into a Python 3 file and run it. An assertion failure means the actual result differs from the expectation. Add the current topic's exercise and boundary tests to deepen your understanding.

assert palindrome('level') and palindrome('')
assert not palindrome('lever')
assert frequencies('abaca')=={'a':3,'b':1,'c':1}