Hello @ipson first sorry my previous comment was not posted for some reason. It is great to see you are exploring DataFlow Data Service on Cloudera Public Cloud. To answer your question: Does it persist state? Unfortunately there is currently no ability to change the flow on a running deployment, so you'd have to publish v2 to the catalog from flow designer terminate original deployment perform new deployment with v2 If you have to deploy a new version of the flow it means you are standing up a new namespace and in turn  it will have a new zookeeper / NiFi / disk among other things contained within the kubernetes namespace. ... View more

Yes SplitRecord is what you should use. Attached is a flow definition as an example. Note that I had to rename the file with a "txt" extension once you download it rename it to a .json extension You can then drag a processor group and it gives you an option to upload the flow definition.   That example generates a file with 102 records and on SlitRecord we use a JsontTreeReader that will split by 3 records and writes the flowfiles out, In this case per 3 per flowFile generating 34 FlowFiles. 1-2 / 3 = 34   In your case and based on your screenshot I would change split count to be 1500000 ( or another number based on your needs ) ... View more

@linssab I am not well versed with the docker aspect of this however I see this: GET http://localhost:8443/nifi-api/flow/current-user That is going to http   I feel the reason this is happening is because you have this in your yaml file: ports: - '8443:8080' cpus : 2 mem_limit: 2G mem_reservation: 2G environment: - NIFI_WEB_HTTP_PORT=8080 That to me tells me that anything going in on 8443 route it to 8080 and 8080 is set on nifi.properties as the value for nifi.web.http.port= because you are passing in through your yaml file: NIFI_WEB_HTTP_PORT=8080 I recommend that you change  ports: - '8443:8080' to ports: - '8443:8443' AND environment: - NIFI_WEB_HTTP_PORT=8080 to environment: - NIFI_WEB_HTTPS_PORT=8443   notice the "S" in HTTP<S>   Then NiFi should start up securely and you will get a login page where you can pass it the values you set for: - SINGLE_USER_CREDENTIALS_USERNAME= - SINGLE_USER_CREDENTIALS_PASSWORD= ... View more