Building Search for High-Volume Salesforce Experiences
“Fast retrieval without forcing the UI to become the search engine.”
Fast retrieval without forcing the UI to become the search engine.
What Made the Problem Difficult
Enterprise quoting, asset discovery, and multi-dimensional catalog configuration require users to filter across hundreds of thousands of location nodes, component specifications, and commercial attributes in real time.
Core Platform Constraint
Direct database querying via SOQL on high-volume tables triggers severe table-scan locks, index non-selectivity errors, and governor limit ceilings. Conversely, dumping large datasets into the browser turns the client JavaScript thread into an overburdened, fragile search engine.
- !Search queries must return targeted results with sub-second responsiveness directly into native Salesforce LWC
- !Zero governor limit exhaustion or table-scan timeouts on high-volume platform database objects
- !Strict separation between search index computation and browser client resources
- !Do not fabricate index sizes, query latency numbers, benchmark results, or ranking formulas without verification
What System Was Designed
Architected a specialized dual-engine search topology tailored for high-volume Salesforce workflows. High-throughput indexing and deep multi-attribute retrieval are handled by a dedicated Rust search engine service. A complementary TypeScript search engine provides agile, lightweight in-memory filtering and client/edge query resolution. Only precisely targeted, hydrated result records return to the native LWC interface, keeping the browser view lean and responsive.
Pivotal Architectural Choices
Key Architectural Decisions
Dual Specialized Search Engines (Rust + TypeScript) vs. CRM SOQL Queries
CRM transactional databases are engineered for ACID consistency, not multidimensional inverted index searching.
- Complex SOQL queries with wildcards and formula fields
- Third-party SaaS search iframe embeds
- Full client-side JavaScript array scanning
Aligns computational demands with specialized tools: Rust provides unmatched memory safety and raw indexing speed, while TypeScript delivers seamless edge integration.
Targeted Result Hydration vs. Full Record Streaming
Transferring thousands of unneeded record fields across the wire degrades both network transit and browser rendering.
- Streaming full records on every filter change
- Naive pagination with full record payloads
Keeps network payloads tiny and guarantees instant DOM rendering even when querying massive underlying datasets.
What Was Actually Built
Engineered a high-performance Rust search engine compiling tokenized inverted indexes for multi-attribute matching.
Developed an in-memory TypeScript search module executing rapid fuzzy matching and category filtering on active subsets.
Constructed a workload distribution router classifying query complexity and dispatching to the appropriate engine.
Implemented a targeted retrieval protocol delivering strictly bounded response payloads to native Salesforce LWC.
Deliberate Architectural Compromises
Trade-offs & Mitigations
⚖External Specialized Search Index vs Standard Salesforce Search (SOSL)
Sub-second response times, complex multidimensional filtering, and zero consumption of CRM search limits.
Requires an asynchronous synchronization stream to keep external indexes aligned with Salesforce CRM updates.
Utilized Change Data Capture (CDC) platform event streams to update search indices asynchronously on record changes.
⚖Compiled Rust Service Maintenance vs Single-Language Node.js Stack
Predictable near-zero memory overhead and exceptional concurrency under heavy catalog query loads.
Requires maintaining a compiled Rust service alongside TypeScript web application stacks.
Encapsulated Rust binaries behind clean, contract-first API boundaries.
Verified Outcomes
Enabled lightning-fast, multi-attribute catalog search for high-volume Salesforce users without UI freezes
Shielded Salesforce CRM from costly full-table scans and query limit timeouts
Delivered targeted result sets directly into native LWC interfaces with zero browser memory strain
Query latency, index throughput, and catalog capacity benchmarks: [VERIFY WITH KULDEEP]
System Schematic & Data Flow
High-throughput Rust engine handles deep indexing while TypeScript engine performs agile dynamic filtering, returning targeted results to native LWC.
Text alternative for screen readers: Architecture flow: Native LWC Interface to Query Workload Router via Debounced Query Ingress; Query Workload Router to Rust Search Engine via High-Volume Inverted Index; Query Workload Router to TypeScript Search Engine via In-Memory Dynamic Filter; Search Engine Core to Native Salesforce UX via Targeted Result Hydration
Architectural Conclusion
“Fast retrieval without forcing the UI to become the search engine.”
- •Never force a relational transactional CRM database or a browser UI to perform the job of a specialized search engine.
- •Designing dedicated search infrastructure that feeds clean, targeted results back into native platform experiences provides the best of both worlds: extreme speed and seamless user familiarity.