Qwen Document Ingestion: Batch Size Four Reduced Elapsed Time in a Bounded Trial

Summary

A first-party OpenWebUI document-ingestion trial found that batches of four chunks reduced mean elapsed time by 39.1% compared with one chunk per request, with HTTP concurrency held at two. Larger batches did not improve the measured latency further. Separate upload checks confirmed completion and retrieval, but neither experiment establishes unlimited capacity or a latency guarantee.

Method and collection date

Measurements were collected in October 2026 through OpenWebUI, a compatibility proxy and a native Apple Silicon Qwen3-Embedding-4B service. The model, 2560-dimensional output, chunk boundaries, permissions and indexed documents were unchanged.

Six sequential trials each embedded 16 unique synthetic chunks of 950 characters. Batch sizes followed the order 1, 4, 8, 8, 4, 1 to reduce simple order bias; inputs differed between trials. Every trial allowed two active HTTP requests. All 96 output vectors were finite, normalized and correctly indexed.

Public OpenWebUI documentation describes external-provider batching [1] and concurrency troubleshooting [2]. The invocation-level semaphore scope and performance below come from the inspected implementation and first-party tests, not from those references alone.

Findings

Four texts per request performed best in this sample

Texts per request	Requests per 16 chunks	Trial times (s)	Mean time (s)
1	16	54.016, 49.013	51.515
4	4	29.268, 33.484	31.376
8	2	33.161, 33.825	33.493

At the same concurrency of two, batch four cut request count by 75% and mean elapsed time by 39.1% relative to batch one. Batch eight reduced transport requests further but was slower than batch four in the measured sample.

Separate English, Korean and source-code comparisons found a minimum matching-vector cosine similarity above 0.⁹⁹⁹⁹ between scalar and batch outputs. This supports numerical consistency for the tested inputs, not semantic retrieval quality in general.

The concurrency limit was per invocation, not global

The applied settings changed RAG_EMBEDDING_BATCH_SIZE from 1 to 4 and RAG_EMBEDDING_CONCURRENT_REQUESTS from 0, meaning unlimited in the observed baseline, to 2. Async embedding remained enabled. Chunk size stayed at 1000 characters with 100-character overlap.

The inspected factory created its semaphore inside each embedding invocation, normally one document ingestion. Two invocations could therefore issue four requests together. The setting was not a cluster-wide, per-replica or per-user global limit. A separate downstream admission layer retained two active requests, 64 pending requests, a 120-second queue wait and explicit HTTP 429 rejection.

Reducing HTTP requests does not remove the computation required for each chunk. It also does not make the per-invocation limit a substitute for downstream global admission.

flowchart TD
    subgraph I["OpenWebUI embedding: per invocation, normally one document"]
        C["Chunks: 1000 characters, overlap 100"] --> B["Batch: four texts per request"]
        B --> S["Invocation-local semaphore: two active HTTP requests"]
    end
    S --> A["Downstream proxy admission: two active, 64 pending, 120-second queue wait"]
    A -->|"Admitted"| P["Native Qwen embedding service"]
    A -->|"Rejection"| R["HTTP 429"]
    T["Two invocations can issue four requests together"] -.-> S

Upload and mixed-load checks succeeded within a narrow scope

Three synthetic UTF-8 files were uploaded and processed concurrently through an ordinary authenticated user's APIs.

Synthetic file	Original bytes	Retrieved indexed chunks	Verification time (s)
A	88,958	80	245.631
B	35,582	32	165.054
C	35,582	32	167.710

All 144 indexed chunks were retrievable with matching source metadata. Each file reached completion, retained its owner and matched its original SHA-256 when downloaded; anonymous downloads returned HTTP 401. Total verification took 245.⁶³⁴ seconds. These times include processing, download/readback and search, not just embedding computation.

The 80-chunk file completed without queue overflow. Batching and bounded publication meant its chunk count did not require all 80 chunks to occupy the downstream queue simultaneously.

While ingestion was in flight, ten concurrent searches of existing synthetic reference documents all returned HTTP 200 and the expected source. Median latency was 5.⁸⁹⁰ seconds and p95 was 20.⁰¹⁷ seconds. Successful mixed-load completion did not imply low query latency.

Limitations

There were only two trials per batch size, on hardware shared with other model workloads. This was not a before/after timing comparison against the former unlimited-concurrency configuration. The 39.1% difference applies only to the bounded batching experiment.

An initial attempt produced a redacted failure; its cause was not established. Completed trials provide the reported acceptance evidence. The tests do not establish capacity for ten ingesting users, arbitrary file sizes, bulk reindexing or sustained service levels. Implementation and configuration findings are as of October 2026 and may differ in other OpenWebUI versions.

References

1. OpenWebUI embedding configuration: https://docs.openwebui.com/reference/env-configuration/#rag_embedding_batch_size
2. OpenWebUI RAG troubleshooting: https://docs.openwebui.com/troubleshooting/rag/