AWS
DynamoDBItemCollectionSizeLimitExceededException - DynamoDB Item Collection Size Limit Exceeded
Hitting a **DynamoDBItemCollectionSizeLimitExceededException** means a single item collection (all items sharing the same partition key) exceeds the 10GB limit—this typically happens with Local Secondary Indexes (LSI) or Global Secondary Indexes (GSI) when too many items share the same partition key. This client-side error (4xx) happens when AWS enforces DynamoDB collection size limits. Most common when hot partitions have excessive data, but also appears when LSI/GSI exceeds 10GB, too many items share the same partition key, or index size limits are reached.
#Common Causes
- →Identity: IAM policy allows DynamoDB operations but collection size exceeded. Service Control Policy (SCP) enforces collection limits.
- →Network: VPC endpoint DynamoDB collection restrictions. Regional collection size limits.
- →Limits: Local Secondary Index exceeds 10GB. Global Secondary Index exceeds 10GB. Too many items with same partition key. Hot partition with excessive data. Index size limit reached.
✓Solutions
- 1Step 1: Diagnose - Check table and index sizes: aws dynamodb describe-table --table-name TABLE_NAME --query 'Table.[TableSizeBytes,ItemCount]' --output table. Check GSI sizes: aws dynamodb describe-table --table-name TABLE_NAME --query 'Table.GlobalSecondaryIndexes[*].[IndexName,IndexSizeBytes,ItemCount]' --output table.
- 2Step 2: Diagnose - Identify hot partitions: Review partition key distribution. Check if single partition key has too many items. Verify if collection size approaches 10GB.
- 3Step 3: Diagnose - Check index sizes: List all GSI/LSI indexes. Check index sizes. Verify if any index exceeds 10GB.
- 4Step 4: Fix - Redesign partition key: Use composite partition key: partition_key = "user123#shard001" instead of "user123". Distribute items across multiple partition keys. Use hash suffix to spread load.
- 5Step 5: Fix - Archive old data or remove indexes: Archive old data to reduce collection size. Remove unused indexes: aws dynamodb update-table --table-name TABLE_NAME --global-secondary-index-updates. Or use sparse indexes to reduce size.
</>Code Examples
Check DynamoDB Table and Index Sizes
1#!/bin/bash
2TABLE_NAME="my-table"
3
4echo "=== Table Size and Item Count ==="
5aws dynamodb describe-table --table-name ${TABLE_NAME} \
6 --query 'Table.[TableName,TableSizeBytes,ItemCount]' \
7 --output table
8
9# Check Global Secondary Indexes
10echo "\n=== Global Secondary Indexes (GSI) ==="
11aws dynamodb describe-table --table-name ${TABLE_NAME} \
12 --query 'Table.GlobalSecondaryIndexes[*].[IndexName,IndexSizeBytes,ItemCount]' \
13 --output table
14
15# Check Local Secondary Indexes
16echo "\n=== Local Secondary Indexes (LSI) ==="
17aws dynamodb describe-table --table-name ${TABLE_NAME} \
18 --query 'Table.LocalSecondaryIndexes[*].[IndexName,IndexSizeBytes,ItemCount]' \
19 --output table
20
21# Check for collections approaching 10GB limit
22echo "\n=== Collection Size Limits ==="
23echo "Single item collection (same partition key) limit: 10GB"
24echo "If any GSI/LSI exceeds 10GB, you'll get this error"Identify Hot Partitions
1#!/bin/bash
2TABLE_NAME="my-table"
3
4echo "=== Identifying Hot Partitions ==="
5echo "Hot partitions have too many items with same partition key"
6echo "This can cause ItemCollectionSizeLimitExceededException"
7
8# Note: DynamoDB doesn't provide direct partition key distribution
9# You need to analyze your data or use CloudWatch metrics
10
11echo "\n=== Check CloudWatch Metrics ==="
12echo "Monitor ConsumedReadCapacityUnits and ConsumedWriteCapacityUnits"
13echo "High values for specific partition keys indicate hot partitions"
14
15# Example: Query to check item distribution
16echo "\n=== Analyzing Partition Key Distribution ==="
17echo "Use DynamoDB Streams or scan with ProjectionExpression"
18echo "Count items per partition key to identify hot partitions"
19
20echo "\n=== Solution: Composite Partition Key ==="
21echo "Instead of: partition_key = 'user123'"
22echo "Use: partition_key = 'user123#shard001'"
23echo "This distributes items across multiple partition keys"Redesign Table with Composite Partition Key
1#!/bin/bash
2echo "=== Redesigning Partition Key to Avoid Hot Partitions ==="
3echo "\nProblem: Single partition key has too many items (>10GB)"
4echo "Solution: Use composite partition key with hash suffix"
5
6echo "\n=== Example Partition Key Redesign ==="
7echo "Old: partition_key = 'user123'"
8echo "New: partition_key = 'user123#shard001'"
9
10# Generate hash suffix for distribution
11USER_ID="user123"
12SHARD_COUNT=10
13SHARD_NUM=$((RANDOM % SHARD_COUNT))
14NEW_PARTITION_KEY="${USER_ID}#shard$(printf '%03d' ${SHARD_NUM})"
15
16echo "\n=== New Partition Key Format ==="
17echo "User ID: ${USER_ID}"
18echo "New partition key: ${NEW_PARTITION_KEY}"
19
20echo "\n=== Benefits ==="
21echo "1. Distributes items across multiple partition keys"
22echo "2. Prevents single collection from exceeding 10GB"
23echo "3. Improves performance by reducing hot partitions"
24echo "4. Allows better parallelization"
25
26echo "\n=== Migration Strategy ==="
27echo "1. Create new table with composite partition key"
28echo "2. Migrate data with new partition key format"
29echo "3. Update application to use new partition key"
30echo "4. Archive or delete old table"↗Related Errors
Provider Information
This error code is specific to AWS services. For more information, refer to the official AWS documentation.