Hi @Eva0 , You were close . I think the first astrik "*" was  not needed because the input json is not contained in an array brackets. Then to include the email in each data element it has to be added at the date & type level but to reference the value you need to go to 2 levels up using the @ character as in @(2,mail) where the email is located in relative to date & type. [ { "operation": "shift", "spec": { //level 2 where email originally is "dates": { // level 1 "*": { // start count here at level 0 "@(2,email)": "[&1].email", "date": "[&1].date", "type": "[&1].type" } } } } ] If you find this helpful please accept solution. Thanks ... View more

Hello @valerio29  Upgrading single component is not supported ... View more

Hello @one4like , Pushing every local file of a job to HDFS will cause issues, especially in larger clusters. Local directories are used as scratch location. Spills of mappers are written there and moving that over to the network will have performance impacts. The local storage of the scratch files and shuffle files is done exactly to prevent this. It also has security impacts as the NM now pushes the keys for each application on to a network location which could be accessible for others. A far better solution is to use the fact that the value of yarn.nodemanager.local-dirs can point to multiple mount points and thus spreading the load over all mount points. So the answer is NO. local-dirs must contain a list of local paths. There's an explicit check in code which only allows local FS to be used. See here: https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LocalDirsHandlerService.java#L224 Please note that an exception is thrown when a non local file system is referenced. If you found this response assisted with your query, please take a moment to log in and click on KUDOS 🙂 & ”Accept as Solution" below this post. Thank you. Bjagtap ... View more