Skip to lesson
0/174 complete
LEARN THE CONCEPTFree forever

Integer arithmetic and overflow

Topic 1 of 6 · Math and geometry

How do math and geometry help algorithms?

Some problems ask about numbers or positions rather than collections alone. Arithmetic rules help test divisibility, find common factors, and count possibilities. Geometry represents points with coordinates and compares distances or directions. We will define each formula before using it and check the number sizes the program can represent.

Example: The point (2, 3) has horizontal coordinate 2 and vertical coordinate 3. The greatest common divisor of 12 and 18 is 6: it is the largest positive integer that divides both without a remainder.

INTERACTIVE WALKTHROUGHPredict · step · explain
SHARED CHAPTER LAB · TABLE

Euclid begins with 48 and 18.

048
118
1 / 4

Step 1 of 4 · 48%18=12

AdvancedWorked exampleTry it yourself
Self-paced learning
Helpful before this chapter

If these ideas are new, begin with:

Understand the idea

Integer arithmetic includes division and remainder conventions, bounds, and overflow. Python // floors; languages may truncate toward zero. Use sufficient width for products and sums in fixed-width implementations.

Work through an example

-7//3 is -3 in Python. A 32-bit signed sum can overflow even when each input fits. Python instead allocates more integer digits as needed.

Time and space complexity

Bounded arithmetic O(1); big-integer cost depends on digit length.

Why it works

Arithmetic follows the language's defined integer semantics and selected range.

Common mistakes

  • Do not use floating division to compute large exact integer ceilings.
  • Midpoint (lo+hi) can overflow in fixed-width languages.

Chapter lab · Python implementation

Arithmetic assumptions affect algorithm correctness. The lab uses Euclid for GCD, modular repeated squaring, and a square-matrix rotation. Python integers avoid fixed-width overflow but grow in storage and arithmetic cost. Treat coordinate conventions and modular inverses explicitly rather than relying on memorized formulas.

This shared lab connects the chapter's topics. Follow the current lesson's worked example first, then trace the complete implementation. Additional algorithms described in topic exercises are variations you can implement independently.

def gcd(a,b):
    a,b=abs(a),abs(b)
    while b: a,b=b,a%b
    return a

def mod_power(base,exponent,modulus):
    if exponent<0 or modulus<1: raise ValueError('invalid exponent/modulus')
    result=1%modulus; base%=modulus
    while exponent:
        if exponent&1: result=result*base%modulus
        base=base*base%modulus; exponent//=2
    return result

def rotate(matrix):
    n=len(matrix)
    if any(len(row)!=n for row in matrix): raise ValueError('square required')
    for r in range(n):
        for c in range(r+1,n):
            matrix[r][c],matrix[c][r]=matrix[c][r],matrix[r][c]
    for row in matrix: row.reverse()

def count_primes(limit):
    if limit<2: return 0
    prime=bytearray([1])*limit
    prime[0]=prime[1]=0
    p=2
    while p*p<limit:
        if prime[p]:
            for multiple in range(p*p,limit,p):
                prime[multiple]=0
        p+=1
    return sum(prime)

def digit_power(base,digits,modulus):
    if modulus<1 or any(not 0<=d<=9 for d in digits):
        raise ValueError('positive modulus and decimal digits required')
    result=1%modulus
    for digit in digits:
        result=mod_power(result,10,modulus)*mod_power(base,digit,modulus)%modulus
    return result

Run and test the lab

Copy the implementation and these assertions into a Python 3 file and run it. An assertion failure means the actual result differs from the expectation. Add the current topic's exercise and boundary tests to deepen your understanding.

assert gcd(48,18)==6 and gcd(-8,0)==8 and gcd(0,0)==0
assert mod_power(3,5,7)==pow(3,5,7)==5
assert mod_power(9,0,1)==0
a=[[1,2],[3,4]]; rotate(a); assert a==[[3,1],[4,2]]
assert all(mod_power(5,e,13)==pow(5,e,13) for e in range(30))
assert count_primes(10)==4 and count_primes(0)==0
assert count_primes(2)==0 and count_primes(100)==25
assert digit_power(2,[1,0],1337)==pow(2,10,1337)
assert digit_power(3,[2,0,1],1337)==pow(3,201,1337)
CHECK YOUR UNDERSTANDING

What is a safe positive-integer ceiling division?

Further reading

Original documentation for the implementation details used in this chapter.

Sieve of Eratosthenes: marking and complexity
Course outline
Software Engineer
DSA
THE SOFTWARE ENGINEERING PATH

Data Structures
& Algorithms

Build intuition. Develop a pattern.
Make it your own.

Free course30 chapters
Your progress0%

0 of 174 published lessons complete