A few things stand out from the numbers you've shared. On a 64-core machine, ingesting ~11 million rows in 17 minutes (around 10–11K rows/sec) is significantly below what I'd expect if the workload were effectively parallelized. Before focusing on CPU count, I'd investigate where the bottleneck actually is. Some areas worth checking: Storage throughput: Is the data being written to local SSDs, network-attached storage, or slower disks? Ingest workloads are often I/O-bound rather than CPU-bound. File size and partitioning strategy: Large numbers of small files can severely impact write performance. Compression settings: Certain codecs provide better compression but consume more CPU during ingest. Thread parallelism: Verify that the ingestion framework is actually utilizing all available cores rather than being limited by a small worker pool. Memory pressure and GC activity: If the JVM is spending significant time in garbage collection, additional CPU cores won't help much. Network throughput: If data is being pulled from a remote source, the bottleneck may be upstream rather than on the ingest node itself. I'd also recommend collecting: CPU utilization during ingest Disk IOPS and throughput metrics Memory usage and GC logs Number of concurrent ingest tasks Average file size being generated One quick diagnostic is to look at overall CPU utilization. If the machine is only using 10–20% of available CPU during ingest, then the workload is likely blocked on I/O, synchronization, network transfers, or application-level limits rather than raw compute capacity. Can you share: Which ingestion tool/framework you're using? The storage type (SSD, NVMe, HDD, cloud volume, etc.)? Average CPU utilization during the 17-minute ingest? Whether the target table is partitioned and, if so, by what key? Those details would make it much easier to determine whether the bottleneck is CPU, disk, network, or configuration-related.
... View more
