Time-Sliced Array Processing for Non-Blocking JavaScript

Question

You have to process 10 million items but can’t block the UI.
How would you split the work efficiently?

Implementation

function processInChunks(items, process, chunkSize = 1000) {
  let index = 0;

  function work() {
    const chunk = items.slice(index, index + chunkSize);
    chunk.forEach(process);
    index += chunkSize;

    if (index < items.length) {
      setTimeout(work); // yield control
    }
  }

  work();
}

or using requestIdleCallback (for browsers that support it):

function processIdle(items, process) {
  let index = 0;
  function work(deadline) {
    while (deadline.timeRemaining() > 0 && index < items.length) {
      process(items[index++]);
    }
    if (index < items.length) requestIdleCallback(work);
  }
  requestIdleCallback(work);
}

Visualization

graph LR
A[10M Items] --> B[Chunk 1] --> C[Pause UI Render] --> D[Chunk 2] --> E[Next Frame Render]

Summary

• Use time slicing to keep apps responsive
• Ideal for large dataset processing
• requestIdleCallback is perfect for background work
• Avoid long synchronous loops in the main thread