The N+1 problem is a common performance issue in programming, especially when dealing with databases and object-relational mapping (ORM) frameworks. It occurs when an application makes one query to fetch a list of items (the “1”) and then makes an additional query for each item in the list (the “N”), resulting in many more database queries than necessary. This can severely impact performance, especially with large datasets.
Let me explain with an example:
Scenario:
Suppose you have a database of authors and their books:
Tables:
Authors
| id | name |
|---|---|
| 1 | Alice |
| 2 | Bob |
Books
| id | title | author_id |
|---|---|---|
| 1 | Book A | 1 |
| 2 | Book B | 1 |
| 3 | Book C | 2 |
Problematic Code Example (Using an ORM like Django or SQLAlchemy):
# Fetch all authors
authors = Author.objects.all() # Query 1
for author in authors:
# Fetch books for each author
books = author.books.all() # Query N (one per author)
print(author.name, [book.title for book in books])
What happens in the database:
- Query to fetch all authors:
SELECT * FROM authors;
- Then, for each author, a separate query to fetch books:
SELECT * FROM books WHERE author_id = 1;
SELECT * FROM books WHERE author_id = 2;
- If there are 100 authors, this will result in 1 + 100 = 101 queries, which is very inefficient.
Solution: Use eager loading / join fetch
Most ORMs provide a way to fetch related data in one query. For example, in Django:
authors = Author.objects.prefetch_related('books') # One query for authors + one query for all books
for author in authors:
print(author.name, [book.title for book in author.books.all()])
- Now the database queries are reduced to 2 queries, no matter how many authors exist.
✅ Key takeaway:
The N+1 problem happens when your code fetches a list of objects, then lazily fetches related objects for each item, causing a large number of queries. Use joins or prefetching to reduce queries and improve performance.