PDF Ingest Report

Generated 2026-06-12 05:34:53 · 1 run(s) · 25 snapshots
RunSnapshotsDuration (min)PDFs donePDFs/minFailedOOMsPeak sys mem (GB)Peak docling RSS (GB)Peak data/ (GB)
one-docling25108.32262.090058.9031.7741.80
Cumulative PDFs completed 0.0 50.0 100 150 200 0 20 40 60 80 100 minutes since run start PDFs one-docling 0.00 min 0.00 PDFs one-docling 0.50 min 0.00 PDFs one-docling 5.31 min 3.00 PDFs one-docling 10.32 min 42.00 PDFs one-docling 15.30 min 89.00 PDFs one-docling 20.31 min 109.00 PDFs one-docling 25.32 min 133.00 PDFs one-docling 30.29 min 149.00 PDFs one-docling 35.31 min 159.00 PDFs one-docling 40.30 min 166.00 PDFs one-docling 45.32 min 173.00 PDFs one-docling 50.31 min 176.00 PDFs one-docling 55.33 min 180.00 PDFs one-docling 60.32 min 183.00 PDFs one-docling 65.29 min 185.00 PDFs one-docling 70.31 min 190.00 PDFs one-docling 75.31 min 194.00 PDFs one-docling 80.31 min 196.00 PDFs one-docling 85.71 min 202.00 PDFs one-docling 86.19 min 202.00 PDFs one-docling 90.31 min 206.00 PDFs one-docling 95.30 min 212.00 PDFs one-docling 99.30 min 214.00 PDFs one-docling 104.05 min 221.00 PDFs one-docling 108.31 min 226.00 PDFs one-docling Throughput 0.0 50.0 100 150 200 0 20 40 60 80 100 minutes since run start PDFs one-docling (last 1h) 0.00 min 0.00 PDFs one-docling (last 1h) 0.50 min 0.00 PDFs one-docling (last 1h) 5.31 min 3.00 PDFs one-docling (last 1h) 10.32 min 42.00 PDFs one-docling (last 1h) 15.30 min 89.00 PDFs one-docling (last 1h) 20.31 min 109.00 PDFs one-docling (last 1h) 25.32 min 133.00 PDFs one-docling (last 1h) 30.29 min 149.00 PDFs one-docling (last 1h) 35.31 min 159.00 PDFs one-docling (last 1h) 40.30 min 166.00 PDFs one-docling (last 1h) 45.32 min 173.00 PDFs one-docling (last 1h) 50.31 min 176.00 PDFs one-docling (last 1h) 55.33 min 180.00 PDFs one-docling (last 1h) 60.32 min 183.00 PDFs one-docling (last 1h) 65.29 min 182.00 PDFs one-docling (last 1h) 70.31 min 148.00 PDFs one-docling (last 1h) 75.31 min 105.00 PDFs one-docling (last 1h) 80.31 min 87.00 PDFs one-docling (last 1h) 85.71 min 67.00 PDFs one-docling (last 1h) 86.19 min 67.00 PDFs one-docling (last 1h) 90.31 min 57.00 PDFs one-docling (last 1h) 95.30 min 53.00 PDFs one-docling (last 1h) 99.30 min 50.00 PDFs one-docling (last 1h) 104.05 min 50.00 PDFs one-docling (last 1h) 108.31 min 50.00 PDFs one-docling (last 30m) 0.00 min 0.00 PDFs one-docling (last 30m) 0.50 min 0.00 PDFs one-docling (last 30m) 5.31 min 3.00 PDFs one-docling (last 30m) 10.32 min 42.00 PDFs one-docling (last 30m) 15.30 min 89.00 PDFs one-docling (last 30m) 20.31 min 109.00 PDFs one-docling (last 30m) 25.32 min 133.00 PDFs one-docling (last 30m) 30.29 min 149.00 PDFs one-docling (last 30m) 35.31 min 156.00 PDFs one-docling (last 30m) 40.30 min 125.00 PDFs one-docling (last 30m) 45.32 min 84.00 PDFs one-docling (last 30m) 50.31 min 67.00 PDFs one-docling (last 30m) 55.33 min 47.00 PDFs one-docling (last 30m) 60.32 min 34.00 PDFs one-docling (last 30m) 65.29 min 26.00 PDFs one-docling (last 30m) 70.31 min 24.00 PDFs one-docling (last 30m) 75.31 min 21.00 PDFs one-docling (last 30m) 80.31 min 20.00 PDFs one-docling (last 30m) 85.71 min 22.00 PDFs one-docling (last 30m) 86.19 min 20.00 PDFs one-docling (last 30m) 90.31 min 23.00 PDFs one-docling (last 30m) 95.30 min 27.00 PDFs one-docling (last 30m) 99.30 min 24.00 PDFs one-docling (last 30m) 104.05 min 29.00 PDFs one-docling (last 30m) 108.31 min 30.00 PDFs one-docling (last 1h) one-docling (last 30m) Failed vs Succeeded (queue counters) 0.0 50.0 100 150 200 0 20 40 60 80 100 minutes since run start jobs one-docling succeeded 0.00 min 0.00 jobs one-docling succeeded 0.50 min 0.00 jobs one-docling succeeded 5.31 min 3.00 jobs one-docling succeeded 10.32 min 42.00 jobs one-docling succeeded 15.30 min 89.00 jobs one-docling succeeded 20.31 min 109.00 jobs one-docling succeeded 25.32 min 133.00 jobs one-docling succeeded 30.29 min 149.00 jobs one-docling succeeded 35.31 min 159.00 jobs one-docling succeeded 40.30 min 166.00 jobs one-docling succeeded 45.32 min 173.00 jobs one-docling succeeded 50.31 min 176.00 jobs one-docling succeeded 55.33 min 180.00 jobs one-docling succeeded 60.32 min 183.00 jobs one-docling succeeded 65.29 min 185.00 jobs one-docling succeeded 70.31 min 190.00 jobs one-docling succeeded 75.31 min 194.00 jobs one-docling succeeded 80.31 min 196.00 jobs one-docling succeeded 85.71 min 202.00 jobs one-docling succeeded 86.19 min 202.00 jobs one-docling succeeded 90.31 min 206.00 jobs one-docling succeeded 95.30 min 212.00 jobs one-docling succeeded 99.30 min 214.00 jobs one-docling succeeded 104.05 min 221.00 jobs one-docling succeeded 108.31 min 225.00 jobs one-docling failed 0.00 min 0.00 jobs one-docling failed 0.50 min 0.00 jobs one-docling failed 5.31 min 0.00 jobs one-docling failed 10.32 min 0.00 jobs one-docling failed 15.30 min 0.00 jobs one-docling failed 20.31 min 0.00 jobs one-docling failed 25.32 min 0.00 jobs one-docling failed 30.29 min 0.00 jobs one-docling failed 35.31 min 0.00 jobs one-docling failed 40.30 min 0.00 jobs one-docling failed 45.32 min 0.00 jobs one-docling failed 50.31 min 0.00 jobs one-docling failed 55.33 min 0.00 jobs one-docling failed 60.32 min 0.00 jobs one-docling failed 65.29 min 0.00 jobs one-docling failed 70.31 min 0.00 jobs one-docling failed 75.31 min 0.00 jobs one-docling failed 80.31 min 0.00 jobs one-docling failed 85.71 min 0.00 jobs one-docling failed 86.19 min 0.00 jobs one-docling failed 90.31 min 0.00 jobs one-docling failed 95.30 min 0.00 jobs one-docling failed 99.30 min 0.00 jobs one-docling failed 104.05 min 0.00 jobs one-docling failed 108.31 min 0.00 jobs one-docling succeeded one-docling failed Disk: data/ on disk vs filesystem free 0.0 100 200 300 400 500 0 20 40 60 80 100 minutes since run start GB one-docling data/ (GB) 0.00 min 0.00 GB one-docling data/ (GB) 0.50 min 0.00 GB one-docling data/ (GB) 5.31 min 0.02 GB one-docling data/ (GB) 10.32 min 3.91 GB one-docling data/ (GB) 15.30 min 19.39 GB one-docling data/ (GB) 20.31 min 29.77 GB one-docling data/ (GB) 25.32 min 41.80 GB one-docling data/ (GB) 30.29 min 37.62 GB one-docling data/ (GB) 35.31 min 36.41 GB one-docling data/ (GB) 40.30 min 25.28 GB one-docling data/ (GB) 45.32 min 23.44 GB one-docling data/ (GB) 50.31 min 18.04 GB one-docling data/ (GB) 55.33 min 15.98 GB one-docling data/ (GB) 60.32 min 19.33 GB one-docling data/ (GB) 65.29 min 14.82 GB one-docling data/ (GB) 70.31 min 21.68 GB one-docling data/ (GB) 75.31 min 22.45 GB one-docling data/ (GB) 80.31 min 23.22 GB one-docling data/ (GB) 85.71 min 24.91 GB one-docling data/ (GB) 86.19 min 24.91 GB one-docling data/ (GB) 90.31 min 29.43 GB one-docling data/ (GB) 95.30 min 31.45 GB one-docling data/ (GB) 99.30 min 35.95 GB one-docling data/ (GB) 104.05 min 36.33 GB one-docling data/ (GB) 108.31 min 35.88 GB one-docling fs free (GB) 0.00 min 491.88 GB one-docling fs free (GB) 0.50 min 491.88 GB one-docling fs free (GB) 5.31 min 491.87 GB one-docling fs free (GB) 10.32 min 487.96 GB one-docling fs free (GB) 15.30 min 472.40 GB one-docling fs free (GB) 20.31 min 461.97 GB one-docling fs free (GB) 25.32 min 449.90 GB one-docling fs free (GB) 30.29 min 453.74 GB one-docling fs free (GB) 35.31 min 454.40 GB one-docling fs free (GB) 40.30 min 465.05 GB one-docling fs free (GB) 45.32 min 466.90 GB one-docling fs free (GB) 50.31 min 472.32 GB one-docling fs free (GB) 55.33 min 474.39 GB one-docling fs free (GB) 60.32 min 471.02 GB one-docling fs free (GB) 65.29 min 475.55 GB one-docling fs free (GB) 70.31 min 468.69 GB one-docling fs free (GB) 75.31 min 467.91 GB one-docling fs free (GB) 80.31 min 467.15 GB one-docling fs free (GB) 85.71 min 465.45 GB one-docling fs free (GB) 86.19 min 465.45 GB one-docling fs free (GB) 90.31 min 460.93 GB one-docling fs free (GB) 95.30 min 458.79 GB one-docling fs free (GB) 99.30 min 454.40 GB one-docling fs free (GB) 104.05 min 454.02 GB one-docling fs free (GB) 108.31 min 454.47 GB one-docling data/ (GB) one-docling fs free (GB) LanceDB live size 0.0 500 1,000 1,500 2,000 2,500 0 20 40 60 80 100 minutes since run start MB one-docling 0.00 min 0.00 MB one-docling 0.50 min 0.00 MB one-docling 5.31 min 7.98 MB one-docling 10.32 min 143.86 MB one-docling 15.30 min 329.43 MB one-docling 20.31 min 415.23 MB one-docling 25.32 min 521.74 MB one-docling 30.29 min 813.08 MB one-docling 35.31 min 864.76 MB one-docling 40.30 min 996.18 MB one-docling 45.32 min 1,044.21 MB one-docling 50.31 min 1,090.33 MB one-docling 55.33 min 1,169.44 MB one-docling 60.32 min 1,415.87 MB one-docling 65.29 min 1,491.35 MB one-docling 70.31 min 1,689.78 MB one-docling 75.31 min 1,767.54 MB one-docling 80.31 min 1,813.38 MB one-docling 85.71 min 1,872.77 MB one-docling 86.19 min 1,872.77 MB one-docling 90.31 min 1,924.03 MB one-docling 95.30 min 2,164.70 MB one-docling 99.30 min 2,209.80 MB one-docling 104.05 min 2,320.34 MB one-docling 108.31 min 2,478.55 MB one-docling System memory used 0.0 10.0 20.0 30.0 40.0 50.0 60.0 0 20 40 60 80 100 minutes since run start GB one-docling 0.00 min 44.27 GB one-docling 0.50 min 44.19 GB one-docling 5.31 min 47.04 GB one-docling 10.32 min 43.42 GB one-docling 15.30 min 48.76 GB one-docling 20.31 min 53.18 GB one-docling 25.32 min 54.88 GB one-docling 30.29 min 55.92 GB one-docling 35.31 min 57.32 GB one-docling 40.30 min 27.82 GB one-docling 45.32 min 40.27 GB one-docling 50.31 min 47.66 GB one-docling 55.33 min 46.06 GB one-docling 60.32 min 57.44 GB one-docling 65.29 min 58.90 GB one-docling 70.31 min 33.79 GB one-docling 75.31 min 49.64 GB one-docling 80.31 min 30.88 GB one-docling 85.71 min 42.03 GB one-docling 86.19 min 44.30 GB one-docling 90.31 min 44.56 GB one-docling 95.30 min 46.66 GB one-docling 99.30 min 50.47 GB one-docling 104.05 min 52.57 GB one-docling 108.31 min 55.89 GB one-docling Process memory (ingester tree vs docling procs) 0.0 5.0 10.0 15.0 20.0 25.0 30.0 0 20 40 60 80 100 minutes since run start GB one-docling ingester 0.00 min 0.33 GB one-docling ingester 0.50 min 0.33 GB one-docling ingester 5.31 min 0.48 GB one-docling ingester 10.32 min 1.55 GB one-docling ingester 15.30 min 2.35 GB one-docling ingester 20.31 min 2.54 GB one-docling ingester 25.32 min 2.77 GB one-docling ingester 30.29 min 2.76 GB one-docling ingester 35.31 min 2.72 GB one-docling ingester 40.30 min 2.93 GB one-docling ingester 45.32 min 3.04 GB one-docling ingester 50.31 min 3.01 GB one-docling ingester 55.33 min 3.62 GB one-docling ingester 60.32 min 3.55 GB one-docling ingester 65.29 min 4.10 GB one-docling ingester 70.31 min 3.58 GB one-docling ingester 75.31 min 3.64 GB one-docling ingester 80.31 min 3.62 GB one-docling ingester 85.71 min 3.60 GB one-docling ingester 86.19 min 3.60 GB one-docling ingester 90.31 min 3.49 GB one-docling ingester 95.30 min 5.04 GB one-docling ingester 99.30 min 3.53 GB one-docling ingester 104.05 min 3.65 GB one-docling ingester 108.31 min 5.00 GB one-docling docling 0.00 min 20.71 GB one-docling docling 0.50 min 20.87 GB one-docling docling 5.31 min 23.55 GB one-docling docling 10.32 min 19.09 GB one-docling docling 15.30 min 23.60 GB one-docling docling 20.31 min 27.81 GB one-docling docling 25.32 min 29.28 GB one-docling docling 30.29 min 30.34 GB one-docling docling 35.31 min 31.77 GB one-docling docling 40.30 min 1.88 GB one-docling docling 45.32 min 14.45 GB one-docling docling 50.31 min 21.86 GB one-docling docling 55.33 min 19.43 GB one-docling docling 60.32 min 29.27 GB one-docling docling 65.29 min 29.96 GB one-docling docling 70.31 min 5.40 GB one-docling docling 75.31 min 21.12 GB one-docling docling 80.31 min 2.29 GB one-docling docling 85.71 min 13.54 GB one-docling docling 86.19 min 15.80 GB one-docling docling 90.31 min 16.19 GB one-docling docling 95.30 min 16.75 GB one-docling docling 99.30 min 22.03 GB one-docling docling 104.05 min 24.00 GB one-docling docling 108.31 min 26.38 GB one-docling ingester one-docling docling CPU usage (delta between snapshots) 0.0 2.0 4.0 6.0 8.0 10.0 0 20 40 60 80 100 minutes since run start % one-docling 0.50 min 5.51 % one-docling 5.31 min 4.96 % one-docling 10.32 min 5.33 % one-docling 15.30 min 6.05 % one-docling 20.31 min 5.62 % one-docling 25.32 min 6.41 % one-docling 30.29 min 5.59 % one-docling 35.31 min 4.77 % one-docling 40.30 min 4.96 % one-docling 45.32 min 3.96 % one-docling 50.31 min 5.22 % one-docling 55.33 min 5.67 % one-docling 60.32 min 10.46 % one-docling 65.29 min 7.24 % one-docling 70.31 min 8.95 % one-docling 75.31 min 4.74 % one-docling 80.31 min 4.37 % one-docling 85.71 min 4.59 % one-docling 86.19 min 3.74 % one-docling 90.31 min 5.14 % one-docling 95.30 min 5.53 % one-docling 99.30 min 4.16 % one-docling 104.05 min 5.39 % one-docling 108.31 min 5.21 % one-docling GPU utilization 0.0 20.0 40.0 60.0 80.0 100 0 20 40 60 80 100 minutes since run start % gpu0 0.00 min 0.00 % gpu0 0.50 min 0.00 % gpu0 5.31 min 0.00 % gpu0 10.32 min 0.00 % gpu0 15.30 min 0.00 % gpu0 20.31 min 0.00 % gpu0 25.32 min 0.00 % gpu0 30.29 min 0.00 % gpu0 35.31 min 0.00 % gpu0 40.30 min 0.00 % gpu0 45.32 min 0.00 % gpu0 50.31 min 0.00 % gpu0 55.33 min 100.00 % gpu0 60.32 min 100.00 % gpu0 65.29 min 0.00 % gpu0 70.31 min 0.00 % gpu0 75.31 min 0.00 % gpu0 80.31 min 0.00 % gpu0 85.71 min 0.00 % gpu0 86.19 min 0.00 % gpu0 90.31 min 0.00 % gpu0 95.30 min 0.00 % gpu0 99.30 min 0.00 % gpu0 104.05 min 0.00 % gpu0 108.31 min 0.00 % gpu1 0.00 min 0.00 % gpu1 0.50 min 0.00 % gpu1 5.31 min 0.00 % gpu1 10.32 min 0.00 % gpu1 15.30 min 0.00 % gpu1 20.31 min 0.00 % gpu1 25.32 min 0.00 % gpu1 30.29 min 0.00 % gpu1 35.31 min 0.00 % gpu1 40.30 min 0.00 % gpu1 45.32 min 0.00 % gpu1 50.31 min 0.00 % gpu1 55.33 min 99.00 % gpu1 60.32 min 100.00 % gpu1 65.29 min 0.00 % gpu1 70.31 min 0.00 % gpu1 75.31 min 0.00 % gpu1 80.31 min 0.00 % gpu1 85.71 min 0.00 % gpu1 86.19 min 0.00 % gpu1 90.31 min 0.00 % gpu1 95.30 min 0.00 % gpu1 99.30 min 0.00 % gpu1 104.05 min 0.00 % gpu1 108.31 min 0.00 % gpu2 0.00 min 0.00 % gpu2 0.50 min 0.00 % gpu2 5.31 min 0.00 % gpu2 10.32 min 0.00 % gpu2 15.30 min 0.00 % gpu2 20.31 min 0.00 % gpu2 25.32 min 0.00 % gpu2 30.29 min 0.00 % gpu2 35.31 min 0.00 % gpu2 40.30 min 0.00 % gpu2 45.32 min 0.00 % gpu2 50.31 min 0.00 % gpu2 55.33 min 61.00 % gpu2 60.32 min 57.00 % gpu2 65.29 min 0.00 % gpu2 70.31 min 0.00 % gpu2 75.31 min 0.00 % gpu2 80.31 min 0.00 % gpu2 85.71 min 0.00 % gpu2 86.19 min 0.00 % gpu2 90.31 min 0.00 % gpu2 95.30 min 0.00 % gpu2 99.30 min 0.00 % gpu2 104.05 min 0.00 % gpu2 108.31 min 0.00 % gpu3 0.00 min 11.00 % gpu3 0.50 min 29.00 % gpu3 5.31 min 42.00 % gpu3 10.32 min 11.00 % gpu3 15.30 min 100.00 % gpu3 20.31 min 0.00 % gpu3 25.32 min 100.00 % gpu3 30.29 min 3.00 % gpu3 35.31 min 0.00 % gpu3 40.30 min 0.00 % gpu3 45.32 min 0.00 % gpu3 50.31 min 10.00 % gpu3 55.33 min 98.00 % gpu3 60.32 min 0.00 % gpu3 65.29 min 0.00 % gpu3 70.31 min 0.00 % gpu3 75.31 min 55.00 % gpu3 80.31 min 0.00 % gpu3 85.71 min 7.00 % gpu3 86.19 min 2.00 % gpu3 90.31 min 33.00 % gpu3 95.30 min 1.00 % gpu3 99.30 min 40.00 % gpu3 104.05 min 28.00 % gpu3 108.31 min 100.00 % gpu0 gpu1 gpu2 gpu3 GPU memory used 0.0 5,000 10,000 15,000 20,000 0 20 40 60 80 100 minutes since run start MB gpu0 0.00 min 20,754.00 MB gpu0 0.50 min 20,754.00 MB gpu0 5.31 min 20,754.00 MB gpu0 10.32 min 20,754.00 MB gpu0 15.30 min 20,754.00 MB gpu0 20.31 min 20,754.00 MB gpu0 25.32 min 20,754.00 MB gpu0 30.29 min 20,754.00 MB gpu0 35.31 min 20,754.00 MB gpu0 40.30 min 20,754.00 MB gpu0 45.32 min 20,754.00 MB gpu0 50.31 min 20,754.00 MB gpu0 55.33 min 20,806.00 MB gpu0 60.32 min 21,472.00 MB gpu0 65.29 min 21,472.00 MB gpu0 70.31 min 21,472.00 MB gpu0 75.31 min 21,472.00 MB gpu0 80.31 min 21,472.00 MB gpu0 85.71 min 21,472.00 MB gpu0 86.19 min 21,472.00 MB gpu0 90.31 min 21,472.00 MB gpu0 95.30 min 21,472.00 MB gpu0 99.30 min 21,472.00 MB gpu0 104.05 min 21,472.00 MB gpu0 108.31 min 21,472.00 MB gpu1 0.00 min 20,834.00 MB gpu1 0.50 min 20,834.00 MB gpu1 5.31 min 20,834.00 MB gpu1 10.32 min 20,834.00 MB gpu1 15.30 min 20,834.00 MB gpu1 20.31 min 20,834.00 MB gpu1 25.32 min 20,834.00 MB gpu1 30.29 min 20,834.00 MB gpu1 35.31 min 20,834.00 MB gpu1 40.30 min 20,834.00 MB gpu1 45.32 min 20,834.00 MB gpu1 50.31 min 20,834.00 MB gpu1 55.33 min 20,884.00 MB gpu1 60.32 min 21,632.00 MB gpu1 65.29 min 21,632.00 MB gpu1 70.31 min 21,632.00 MB gpu1 75.31 min 21,632.00 MB gpu1 80.31 min 21,632.00 MB gpu1 85.71 min 21,632.00 MB gpu1 86.19 min 21,632.00 MB gpu1 90.31 min 21,632.00 MB gpu1 95.30 min 21,632.00 MB gpu1 99.30 min 21,632.00 MB gpu1 104.05 min 21,632.00 MB gpu1 108.31 min 21,632.00 MB gpu2 0.00 min 21,918.00 MB gpu2 0.50 min 21,918.00 MB gpu2 5.31 min 21,918.00 MB gpu2 10.32 min 21,918.00 MB gpu2 15.30 min 21,918.00 MB gpu2 20.31 min 21,918.00 MB gpu2 25.32 min 21,918.00 MB gpu2 30.29 min 21,918.00 MB gpu2 35.31 min 21,918.00 MB gpu2 40.30 min 21,918.00 MB gpu2 45.32 min 21,918.00 MB gpu2 50.31 min 21,918.00 MB gpu2 55.33 min 21,950.00 MB gpu2 60.32 min 21,954.00 MB gpu2 65.29 min 21,956.00 MB gpu2 70.31 min 21,956.00 MB gpu2 75.31 min 21,956.00 MB gpu2 80.31 min 21,956.00 MB gpu2 85.71 min 21,956.00 MB gpu2 86.19 min 21,956.00 MB gpu2 90.31 min 21,956.00 MB gpu2 95.30 min 21,956.00 MB gpu2 99.30 min 21,956.00 MB gpu2 104.05 min 21,956.00 MB gpu2 108.31 min 21,956.00 MB gpu3 0.00 min 16,799.00 MB gpu3 0.50 min 17,079.00 MB gpu3 5.31 min 17,085.00 MB gpu3 10.32 min 18,179.00 MB gpu3 15.30 min 18,179.00 MB gpu3 20.31 min 18,219.00 MB gpu3 25.32 min 18,703.00 MB gpu3 30.29 min 18,703.00 MB gpu3 35.31 min 18,703.00 MB gpu3 40.30 min 12,390.00 MB gpu3 45.32 min 18,031.00 MB gpu3 50.31 min 18,031.00 MB gpu3 55.33 min 18,677.00 MB gpu3 60.32 min 18,689.00 MB gpu3 65.29 min 18,689.00 MB gpu3 70.31 min 16,329.00 MB gpu3 75.31 min 16,867.00 MB gpu3 80.31 min 13,153.00 MB gpu3 85.71 min 17,929.00 MB gpu3 86.19 min 17,929.00 MB gpu3 90.31 min 18,421.00 MB gpu3 95.30 min 18,481.00 MB gpu3 99.30 min 18,861.00 MB gpu3 104.05 min 18,861.00 MB gpu3 108.31 min 18,861.00 MB gpu0 gpu1 gpu2 gpu3 OOM kills (cumulative) 0.0 0.2 0.4 0.6 0.8 1.0 0 20 40 60 80 100 minutes since run start kills one-docling 0.50 min 0.00 kills one-docling 5.31 min 0.00 kills one-docling 10.32 min 0.00 kills one-docling 15.30 min 0.00 kills one-docling 20.31 min 0.00 kills one-docling 25.32 min 0.00 kills one-docling 30.29 min 0.00 kills one-docling 35.31 min 0.00 kills one-docling 40.30 min 0.00 kills one-docling 45.32 min 0.00 kills one-docling 50.31 min 0.00 kills one-docling 55.33 min 0.00 kills one-docling 60.32 min 0.00 kills one-docling 65.29 min 0.00 kills one-docling 70.31 min 0.00 kills one-docling 75.31 min 0.00 kills one-docling 80.31 min 0.00 kills one-docling 85.71 min 0.00 kills one-docling 86.19 min 0.00 kills one-docling 90.31 min 0.00 kills one-docling 95.30 min 0.00 kills one-docling 99.30 min 0.00 kills one-docling 104.05 min 0.00 kills one-docling 108.31 min 0.00 kills one-docling

Hover any point for exact values. Sawtooth lines reflect periodic LanceDB compaction.