Optimise Firestore Queries: The UUID List Strategy for Minimal Indexes

A comprehensive guide to reducing index complexity while maintaining query performance

A comprehensive guide to reducing index complexity while maintaining query performance

Introduction

Cloud Firestore is a powerful NoSQL database, but its indexing requirements can quickly become complex and costly as your application scales. One elegant solution that many developers overlook is using a separate field containing a list of UUIDs to optimize filtering operations. This approach can dramatically reduce the number of composite indexes needed while maintaining excellent query performance.

In this article, we’ll explore how to implement this strategy, when to use it, and the performance benefits it provides.

The Problem: Index Explosion

Traditional Firestore optimization often leads to what I call “index explosion.” Consider a typical e-commerce scenario where you need to filter products by multiple criteria:

Category
Price range
Brand
Availability
User preferences
Geographic region

With traditional approaches, you’d need composite indexes for every possible combination of these filters. This quickly becomes unmanageable:

// Traditional approach requiring multiple composite indexes 
db.collection('products') 
  .where('category', '==', 'electronics') 
  .where('price', '<=', 500) 
  .where('brand', '==', 'Apple') 
  .where('available', '==', true)

Each combination requires a separate composite index, leading to:

Increased storage costs
Slower write operations
Complex index management
Higher maintenance overhead

The UUID List Solution

The UUID list strategy involves creating a separate field that contains an array of UUIDs representing different filter combinations. Instead of querying multiple fields, you query a single array field using Firestore’s array-contains or array-contains-any operators.

Core Concept

// Document structure with UUID list 
{ 
  id: "product123", 
  name: "iPhone 14", 
  price: 999, 
  category: "electronics", 
  brand: "Apple", 
  // The magic field - contains UUIDs for different filter combinations 
  filterUUIDs: [ 
    "uuid-electronics-apple", 
    "uuid-electronics-premium",  
    "uuid-apple-available", 
    "uuid-premium-available", 
    "uuid-electronics-apple-premium" 
  ], 
  // ... other product data 
}

Implementation Strategy

Step 1: Design Your UUID Schema

Create a systematic approach to generating UUIDs that represent filter combinations:

class FilterUUIDGenerator { 
  static generateUUID(filters) { 
    // Sort filters to ensure consistency 
    const sortedFilters = Object.keys(filters) 
      .sort() 
      .map(key => `${key}-${filters[key]}`) 
      .join('-'); 
     
    return `uuid-${sortedFilters}`; 
  } 
 
  static generateAllCombinations(productData) { 
    const uuids = []; 
    const filters = { 
      category: productData.category, 
      brand: productData.brand, 
      priceRange: this.getPriceRange(productData.price), 
      availability: productData.available 
    }; 
    // Generate UUIDs for single filters 
    Object.keys(filters).forEach(key => { 
      uuids.push(this.generateUUID({ [key]: filters[key] })); 
    }); 
    // Generate UUIDs for combinations 
    this.generateCombinations(filters, 2).forEach(combo => { 
      uuids.push(this.generateUUID(combo)); 
    }); 
    this.generateCombinations(filters, 3).forEach(combo => { 
      uuids.push(this.generateUUID(combo)); 
    }); 
    return uuids; 
  } 
  static getPriceRange(price) { 
    if (price < 100) return 'budget'; 
    if (price < 500) return 'mid-range'; 
    return 'premium'; 
  } 
  static generateCombinations(obj, r) { 
    const keys = Object.keys(obj); 
    const combinations = []; 
     
    function combine(start, combo) { 
      if (combo.length === r) { 
        const result = {}; 
        combo.forEach(key => result[key] = obj[key]); 
        combinations.push(result); 
        return; 
      } 
       
      for (let i = start; i < keys.length; i++) { 
        combine(i + 1, [...combo, keys[i]]); 
      } 
    } 
     
    combine(0, []); 
    return combinations; 
  } 
}

Step 2: Document Creation with UUID Generation

async function createProductWithUUIDs(productData) { 
  const filterUUIDs = FilterUUIDGenerator.generateAllCombinations(productData); 
   
  const docData = { 
    ...productData, 
    filterUUIDs, 
    createdAt: new Date(), 
    updatedAt: new Date() 
  }; 
   
  await db.collection('products').add(docData); 
} 
// Example usage 
await createProductWithUUIDs({ 
  name: "iPhone 14", 
  category: "electronics", 
  brand: "Apple", 
  price: 999, 
  available: true 
});

Step 3: Optimized Querying

class OptimizedQuery { 
  static async queryProducts(filters) { 
    const targetUUID = FilterUUIDGenerator.generateUUID(filters); 
     
    const snapshot = await db.collection('products') 
      .where('filterUUIDs', 'array-contains', targetUUID) 
      .get(); 
     
    return snapshot.docs.map(doc => ({ id: doc.id, ...doc.data() })); 
  } 
 
  static async queryProductsMultiple(filterSets) { 
    const targetUUIDs = filterSets.map(filters =>  
      FilterUUIDGenerator.generateUUID(filters) 
    ); 
     
    // Use array-contains-any for multiple filter combinations 
    const snapshot = await db.collection('products') 
      .where('filterUUIDs', 'array-contains-any', targetUUIDs) 
      .get(); 
     
    return snapshot.docs.map(doc => ({ id: doc.id, ...doc.data() })); 
  } 
} 
// Usage examples 
const electronics = await OptimizedQuery.queryProducts({ 
  category: 'electronics' 
}); 
const appleElectronics = await OptimizedQuery.queryProducts({ 
  category: 'electronics', 
  brand: 'Apple' 
}); 
const multipleFilters = await OptimizedQuery.queryProductsMultiple([ 
  { category: 'electronics', brand: 'Apple' }, 
  { category: 'books', brand: 'Penguin' } 
]);

Index Requirements

The beauty of this approach is its minimal index requirements. Array-contains / array-contains-any / in / not-in queries use special operators and do not require extra indexes, meaning you only need:

Single-field index on filterUUIDs (automatically created)
Composite indexes only when combining with other non-UUID filters

# firestore.indexes.json - Minimal index configuration 
{ 
  "indexes": [ 
    { 
      "collectionGroup": "products", 
      "queryScope": "COLLECTION", 
      "fields": [ 
        { "fieldPath": "filterUUIDs", "arrayConfig": "CONTAINS" }, 
        { "fieldPath": "createdAt", "order": "DESCENDING" } 
      ] 
    } 
  ] 
}

Advanced Optimization Techniques

1. Hierarchical UUID Structure

For complex filtering scenarios, implement hierarchical UUIDs:

class HierarchicalUUIDGenerator { 
  static generateHierarchicalUUIDs(productData) { 
    const uuids = []; 
     
    // Level 1: Single attributes 
    uuids.push(`L1-cat-${productData.category}`); 
    uuids.push(`L1-brand-${productData.brand}`); 
    uuids.push(`L1-price-${this.getPriceRange(productData.price)}`); 
     
    // Level 2: Pairs 
    uuids.push(`L2-cat-brand-${productData.category}-${productData.brand}`); 
    uuids.push(`L2-cat-price-${productData.category}-${this.getPriceRange(productData.price)}`); 
     
    // Level 3: Triples and beyond 
    uuids.push(`L3-cat-brand-price-${productData.category}-${productData.brand}-${this.getPriceRange(productData.price)}`); 
     
    return uuids; 
  } 
}

2. Dynamic UUID Management

Implement systems to add new UUIDs when filter requirements change:

async function updateProductUUIDs(productId, newFilters) { 
  const docRef = db.collection('products').doc(productId); 
  const doc = await docRef.get(); 
  const currentData = doc.data(); 
   
  const newUUIDs = FilterUUIDGenerator.generateAllCombinations({ 
    ...currentData, 
    ...newFilters 
  }); 
   
  await docRef.update({ 
    ...newFilters, 
    filterUUIDs: newUUIDs, 
    updatedAt: new Date() 
  }); 
}

3. UUID Cleanup and Maintenance

Regular maintenance ensures optimal performance:

class UUIDMaintenance { 
  static async cleanupUnusedUUIDs() { 
    const snapshot = await db.collection('products').get(); 
    const usedUUIDs = new Set(); 
     
    snapshot.docs.forEach(doc => { 
      const data = doc.data(); 
      data.filterUUIDs?.forEach(uuid => usedUUIDs.add(uuid)); 
    }); 
     
    // Analytics: track UUID usage patterns 
    console.log(`Total unique UUIDs in use: ${usedUUIDs.size}`); 
     
    // Implement cleanup logic based on usage patterns 
    return Array.from(usedUUIDs); 
  } 
 
  static async optimizeUUIDDistribution() { 
    const products = await db.collection('products').get(); 
    const uuidCounts = {}; 
     
    products.docs.forEach(doc => { 
      const uuids = doc.data().filterUUIDs || []; 
      uuids.forEach(uuid => { 
        uuidCounts[uuid] = (uuidCounts[uuid] || 0) + 1; 
      }); 
    }); 
     
    // Identify most/least common UUIDs for optimization 
    return Object.entries(uuidCounts) 
      .sort(([,a], [,b]) => b - a); 
  } 
}

Performance Benefits

Query Performance

Reduced read costs: Single array-contains query vs multiple field queries
Faster execution: Cloud Firestore selects the optimal index for your query with minimal index scanning
Predictable performance: Array operations scale consistently

Index Management

Minimal composite indexes: Dramatically reduced from O(n²) to O(1) for most queries
Faster writes: Fewer indexes to update during document creation/updates
Reduced storage costs: Significantly smaller index footprint

Scalability Benefits

Flexible filtering: Easy to add new filter combinations without index changes
Maintenance simplicity: Single field to manage vs multiple composite indexes
Future-proofing: New filter requirements don’t require infrastructure changes

Critical Limitations and Constraints

Before implementing the UUID list strategy, you must understand Firestore’s array-contains limitations:

Array-Contains Query Limitations

Query Limits:

array-contains-any queries are limited to 30 comparison values maximum (previously 10, updated in recent versions)
Only one array-contains-any operation per query - you cannot combine multiple array-contains queries
not-in queries remain limited to 10 values
You cannot combine array-contains-any with in or not-in in the same query

Document Size Constraints:

Each document has a maximum size of 1 MiB (1,048,576 bytes)
Maximum 20,000 fields per document including nested fields
Array size is practically limited by the document size constraint

Working Within the Limitations

class LimitationAwareUUIDGenerator { 
  static generateOptimizedUUIDs(productData, maxUUIDs = 100) { 
    const uuids = []; 
     
    // Prioritize most common filter combinations first 
    const priorityFilters = this.getPriorityFilters(productData); 
     
    // Add high-priority single filters 
    priorityFilters.forEach(filter => { 
      if (uuids.length < maxUUIDs) { 
        uuids.push(this.generateUUID(filter)); 
      } 
    }); 
     
    // Add combinations only if space allows 
    const combinations = this.generateCombinations(productData, 2); 
    combinations.forEach(combo => { 
      if (uuids.length < maxUUIDs) { 
        uuids.push(this.generateUUID(combo)); 
      } 
    }); 
     
    return uuids; 
  } 
 
  static estimateDocumentSize(productData, uuids) { 
    // Rough estimation: field names + values + overhead 
    const baseDataSize = JSON.stringify(productData).length; 
    const uuidArraySize = JSON.stringify(uuids).length; 
    const fieldNamesSize = Object.keys(productData).join('').length; 
     
    const totalSize = baseDataSize + uuidArraySize + fieldNamesSize + 1000; // overhead 
     
    if (totalSize > 1000000) { // Close to 1MiB limit 
      console.warn(`Document size approaching limit: ${totalSize} bytes`); 
    } 
     
    return totalSize; 
  } 
}

Handling Array-Contains-Any Limit

When you need to query more than 30 UUIDs, implement pagination or chunking:

class LimitedArrayQuery { 
  static async queryWithLargeUUIDSet(uuidList) { 
    const results = []; 
    const chunkSize = 30; 
     
    // Split into chunks of 30 (the array-contains-any limit) 
    for (let i = 0; i < uuidList.length; i += chunkSize) { 
      const chunk = uuidList.slice(i, i + chunkSize); 
       
      const snapshot = await db.collection('products') 
        .where('filterUUIDs', 'array-contains-any', chunk) 
        .get(); 
       
      snapshot.docs.forEach(doc => { 
        const docData = { id: doc.id, ...doc.data() }; 
        // Avoid duplicates if documents match multiple UUIDs 
        if (!results.find(item => item.id === doc.id)) { 
          results.push(docData); 
        } 
      }); 
    } 
     
    return results; 
  } 
 
  static async queryMultipleFilters(filterGroups) { 
    // When you need complex OR logic across different filter types 
    const promises = filterGroups.map(async (filters) => { 
      const uuid = FilterUUIDGenerator.generateUUID(filters); 
      const snapshot = await db.collection('products') 
        .where('filterUUIDs', 'array-contains', uuid) 
        .get(); 
       
      return snapshot.docs.map(doc => ({ id: doc.id, ...doc.data() })); 
    }); 
    const resultArrays = await Promise.all(promises); 
     
    // Merge and deduplicate results 
    const allResults = resultArrays.flat(); 
    const uniqueResults = Array.from( 
      new Map(allResults.map(item => [item.id, item])).values() 
    ); 
     
    return uniqueResults; 
  } 
}

Document Size Management Strategy

class DocumentSizeManager { 
  static async createProductWithSizeValidation(productData) { 
    let uuids = FilterUUIDGenerator.generateAllCombinations(productData); 
     
    // Check if document would exceed size limits 
    let estimatedSize = this.estimateSize(productData, uuids); 
     
    if (estimatedSize > 900000) { // 90% of 1MiB limit 
      // Reduce UUIDs to essential combinations only 
      uuids = this.getEssentialUUIDs(productData); 
      estimatedSize = this.estimateSize(productData, uuids); 
       
      console.warn(`Reduced UUIDs due to size constraints. New size: ${estimatedSize}`); 
    } 
     
    const docData = { 
      ...productData, 
      filterUUIDs: uuids, 
      createdAt: new Date() 
    }; 
 
    await db.collection('products').add(docData); 
  } 
  static getEssentialUUIDs(productData) { 
    // Only include the most important filter combinations 
    return [ 
      `category-${productData.category}`, 
      `brand-${productData.brand}`, 
      `price-${this.getPriceRange(productData.price)}`, 
      `category-brand-${productData.category}-${productData.brand}`, 
      `category-price-${productData.category}-${this.getPriceRange(productData.price)}` 
    ]; 
  } 
  static estimateSize(productData, uuids) { 
    // More accurate size estimation 
    const jsonString = JSON.stringify({ 
      ...productData, 
      filterUUIDs: uuids 
    }); 
     
    return Buffer.byteLength(jsonString, 'utf8'); 
  } 
}

When to Use This Strategy

Ideal Scenarios

✅ Multiple filter combinations: When you need to support many different filter combinations
✅ Frequent filter changes: When filter requirements evolve regularly
✅ Cost optimization: When index costs are a significant concern
✅ Complex boolean logic: When you need OR-like operations across filters
✅ Moderate UUID counts: When you can keep UUID arrays under 100 elements per document
✅ Known query patterns: When you can predict and prioritize common filter combinations

Not Recommended For

❌ Simple single-field queries: Traditional indexes are more efficient
❌ Range queries: When you need numerical range filtering
❌ Real-time analytics: When you need complex aggregation queries
❌ Large UUID arrays: When documents would exceed Firestore’s size limits
❌ Unpredictable queries: When filter combinations are completely dynamic
❌ More than 30 simultaneous filters: Due to array-contains-any limitations

Implementation Checklist

Planning Phase

Identify all possible filter combinations
Design UUID naming convention
Estimate UUID array sizes per document (keep under 100 UUIDs)
Plan for future filter requirements
Validate document sizes stay under 1 MiB limit
Consider array-contains-any 30-value limitation
Plan chunking strategy for large UUID sets

Development Phase

Implement UUID generation logic
Create document creation/update functions
Build query optimization layer
Add error handling and validation

Testing Phase

Performance test with realistic data volumes
Compare query costs vs traditional approaches
Test edge cases (empty filters, large combinations)
Validate index efficiency

Production Phase

Monitor query performance metrics
Track UUID usage patterns
Implement maintenance routines
Plan for scaling considerations
Monitor document sizes regularly
Handle array-contains-any limit gracefully
Implement fallback strategies for complex queries

Real-World Example: E-commerce Product Filtering

Let’s implement a complete e-commerce filtering system:

class EcommerceFilterSystem { 
  constructor() { 
    this.db = firebase.firestore(); 
  } 
  async addProduct(productData) { 
    const enrichedData = { 
      ...productData, 
      priceRange: this.calculatePriceRange(productData.price), 
      createdAt: new Date() 
    }; 
    const filterUUIDs = this.generateProductUUIDs(enrichedData); 
     
    await this.db.collection('products').add({ 
      ...enrichedData, 
      filterUUIDs 
    }); 
  } 
 
  generateProductUUIDs(product) { 
    const attributes = { 
      category: product.category, 
      brand: product.brand, 
      priceRange: product.priceRange, 
      inStock: product.inventory > 0, 
      rating: product.rating >= 4 ? 'high' : 'standard' 
    }; 
 
    const uuids = []; 
     
    // Single attribute UUIDs 
    Object.entries(attributes).forEach(([key, value]) => { 
      uuids.push(`${key}-${value}`); 
    }); 
 
    // Popular combination UUIDs 
    uuids.push(`category-brand-${attributes.category}-${attributes.brand}`); 
    uuids.push(`category-price-${attributes.category}-${attributes.priceRange}`); 
    uuids.push(`brand-price-${attributes.brand}-${attributes.priceRange}`); 
    uuids.push(`category-stock-${attributes.category}-${attributes.inStock}`); 
     
    // Premium combination for high-performance queries 
    uuids.push(`premium-${attributes.category}-${attributes.brand}-${attributes.priceRange}`); 
 
    return uuids; 
  } 
 
  async searchProducts(filters, options = {}) { 
    let query = this.db.collection('products'); 
     
    if (filters && Object.keys(filters).length > 0) { 
      const targetUUID = this.buildFilterUUID(filters); 
      query = query.where('filterUUIDs', 'array-contains', targetUUID); 
    } 
 
    if (options.orderBy) { 
      query = query.orderBy(options.orderBy, options.order || 'asc'); 
    } 
 
    if (options.limit) { 
      query = query.limit(options.limit); 
    } 
 
    const snapshot = await query.get(); 
    return snapshot.docs.map(doc => ({ id: doc.id, ...doc.data() })); 
  } 
  buildFilterUUID(filters) { 
    const parts = Object.entries(filters) 
      .sort(([a], [b]) => a.localeCompare(b)) 
      .map(([key, value]) => `${key}-${value}`); 
     
    return parts.join('-'); 
  } 
  calculatePriceRange(price) { 
    if (price < 50) return 'budget'; 
    if (price < 200) return 'mid'; 
    if (price < 500) return 'premium'; 
    return 'luxury'; 
  } 
} 
 
// Usage 
const filterSystem = new EcommerceFilterSystem(); 
 
// Add products 
await filterSystem.addProduct({ 
  name: 'iPhone 14', 
  category: 'electronics', 
  brand: 'Apple', 
  price: 999, 
  inventory: 50, 
  rating: 4.5 
}); 
 
// Search with optimized queries 
const results = await filterSystem.searchProducts({ 
  category: 'electronics', 
  brand: 'Apple' 
}, {  
  orderBy: 'createdAt',  
  order: 'desc',  
  limit: 20  
});

Monitoring and Analytics

Track the performance of your UUID-based filtering:

class FilterAnalytics { 
  static async trackQueryPerformance(filterUUID, startTime) { 
    const duration = Date.now() - startTime; 
     
    await db.collection('query_analytics').add({ 
      filterUUID, 
      duration, 
      timestamp: new Date() 
    }); 
  } 
 
  static async getPopularFilters(days = 7) { 
    const cutoff = new Date(Date.now() - days * 24 * 60 * 60 * 1000); 
     
    const snapshot = await db.collection('query_analytics') 
      .where('timestamp', '>=', cutoff) 
      .get(); 
     
    const filterCounts = {}; 
    snapshot.docs.forEach(doc => { 
      const uuid = doc.data().filterUUID; 
      filterCounts[uuid] = (filterCounts[uuid] || 0) + 1; 
    }); 
     
    return Object.entries(filterCounts) 
      .sort(([,a], [,b]) => b - a) 
      .slice(0, 10); 
  } 
}

Conclusion

The UUID list strategy represents a paradigm shift in Firestore optimization. By trading a small amount of additional storage for dramatically simplified index management, you can achieve:

90% reduction in required composite indexes
Consistent query performance regardless of filter complexity
Simplified maintenance and easier feature development
Cost optimization through reduced index overhead

This approach particularly shines in applications with complex filtering requirements, frequent filter changes, or cost-sensitive scenarios. While it requires upfront planning and implementation effort, the long-term benefits in terms of performance, maintainability, and cost make it an invaluable strategy for scaling Firestore applications.

Remember to always test this approach with your specific data patterns and query requirements. Gradually ramp up traffic to new collections to give Cloud Firestore sufficient time to prepare documents for increased traffic, starting with a maximum of 500 operations per second when implementing this optimization.

The UUID list strategy isn’t just an optimization technique — it’s a architectural pattern that can fundamentally improve how you think about and implement data filtering in NoSQL databases.

Ready to implement UUID-based filtering in your Firestore application? Start with a small subset of your data, measure the performance improvements, and gradually expand the strategy across your entire application.

Optimise Firestore Queries: The UUID List Strategy for Minimal Indexes

Introduction

The Problem: Index Explosion

The UUID List Solution

Core Concept

Implementation Strategy

Step 1: Design Your UUID Schema

Step 2: Document Creation with UUID Generation

Step 3: Optimized Querying

Index Requirements

Advanced Optimization Techniques

1. Hierarchical UUID Structure

2. Dynamic UUID Management

3. UUID Cleanup and Maintenance

Performance Benefits

Query Performance

Index Management

Scalability Benefits

Critical Limitations and Constraints

Array-Contains Query Limitations

Working Within the Limitations

Handling Array-Contains-Any Limit

Document Size Management Strategy

When to Use This Strategy

Ideal Scenarios

Not Recommended For

Implementation Checklist

Planning Phase

Development Phase

Testing Phase

Production Phase

Real-World Example: E-commerce Product Filtering

Monitoring and Analytics

Conclusion

Read next

Deploying Ghost CMS on Oracle Cloud’s Always Free Tier: A Comprehensive Guide to Setup and Port…

Exploring Oracle Cloud’s Generous Free Tier: Always Free VPS and More

Fixing the “Failed to Send Email” Error in Ghost CMS Login

Comments ()

Introduction

The Problem: Index Explosion

The UUID List Solution

Core Concept

Implementation Strategy

Step 1: Design Your UUID Schema

Step 2: Document Creation with UUID Generation

Step 3: Optimized Querying

Index Requirements

Advanced Optimization Techniques

1. Hierarchical UUID Structure

2. Dynamic UUID Management

3. UUID Cleanup and Maintenance

Performance Benefits

Query Performance

Index Management

Scalability Benefits

Critical Limitations and Constraints

Array-Contains Query Limitations

Working Within the Limitations

Handling Array-Contains-Any Limit

Document Size Management Strategy

When to Use This Strategy

Ideal Scenarios

Not Recommended For

Implementation Checklist

Planning Phase

Development Phase

Testing Phase

Production Phase

Real-World Example: E-commerce Product Filtering

Monitoring and Analytics

Conclusion

Read next

Comments ( )

Comments ()