Record Processing
What it does
The Data Processor is the core engine of DOSE. It picks up each submitted file, reads it in batches, validates every record, maps it to the RxClaim format, and writes each record to RxClaim via the CAT API — one record at a time, in parallel where possible.
Key features:
- Processes files in batches of ~100 records, streaming from blob storage — no full-file load into memory
- Each record is independent — a failure on one record never blocks others in the same batch
- Retries transient failures (network errors, timeouts) automatically with exponential backoff
- Rate-limit aware — pauses automatically when the CAT API throttles
- Publishes a notification to CLIC when a record is rejected before reaching RxClaim
- Every downstream call is fully audited — request, response, latency, and outcome recorded
How it works
Record outcomes
| Outcome | When | Retry? |
|---|---|---|
| Completed | RxClaim accepted the record | — |
| Validation Failed | Record didn't pass DOSE schema | No — notifies upstream |
| Failed | RxClaim rejected with business errors | No |
| Retry | Network error, timeout, or server error | Yes — up to configured limit with backoff |
Field mapping
DOSE record fields do not match RxClaim CAT API fields directly. The mapping is defined in a JOLT specification file stored in blob storage. This means field mapping changes are a configuration update — not a code change or deployment.
Example: recordData.gpi (DOSE) → genericProductId (RxClaim), recordData.fromDate → both currentEffDate and intentProductEffDate.
Per-record environment routing
Each record specifies which RxClaim environment it targets (targetRxClaimEnv). Records in the same file can route to different RxClaim environments.
Upstream notification
When a record fails DOSE schema validation — before ever reaching RxClaim — DOSE publishes a notification to an external Kafka topic so CLIC knows the specific record was rejected and why.
What the processor reads and writes
| Collection | Read | Written |
|---|---|---|
upstream_job | Yes | Updates status: Processing → Completed / Partial Failed / Error |
IntentRecord | — | Creates per record; updates outcome per CAT API response |
DeliveryAuditLog | — | One entry per CAT API call |
spec_mapping | Yes (resolve JOLT spec path) | — |
| Kafka topic | — | Rejection notification per validation failure |
Technical reference
| Detail | Value |
|---|---|
| Service | RxDose-DataProcessor-MS |
| Port | 8092 |
| Language | Java 21, Spring Boot 4.0.7, Spring WebFlux (reactive) |
| Batch size | ~100 records per batch |
| Field mapping | JOLT 0.1.8, spec loaded from Azure Blob |
| Upstream notification | CloudEvents 1.0 over Kafka (mTLS) |
| API docs | http://localhost:8092/docs/rxdose/ui |