Member since
08-24-2017
32
Posts
7
Kudos Received
5
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1172 | 06-11-2018 11:24 AM | |
765 | 03-23-2018 03:07 PM | |
552 | 03-07-2018 10:56 AM | |
629 | 03-07-2018 10:35 AM | |
502 | 03-06-2018 01:20 PM |
08-29-2018
08:24 AM
3 Kudos
Short Description:
Using JOLT transformation to modify a JSON object. Article:
In NIFI, JOLT Transformation can be used to help modifying a JSON object, the below code snippets are an example of how JOLT can be used to shift elements inside a JSON Object, in this case, the intention is to move and repeat "productionID" and "processID" inside of the array.
Original JSON Object:
{
"productionID": "BHS0928MDB8391",
"productionLine": [
{
"model": "Cooper",
"year": 2018,
"type": "Hatchback",
"motorization": "Electric",
"colour": "Midnight Black",
"stageID": "MGOP94810482042"
},
{
"model": "Cooper",
"year": 2018,
"type": "Hatchback",
"motorization": "Diesel",
"colour": "Silver",
"stageID": "MGOP9183740194"
}
],
"processID": "884910421743POR4183UK"
}
Target JSON Object:
{
"productionLine" : [ {
"model" : "Cooper",
"year" : 2018,
"type" : "Hatchback",
"motorization" : "Electric",
"colour" : "Midnight Black",
"stageID" : "MGOP94810482042",
"productionID" : "BHS0928MDB8391",
"processID" : "884910421743POR4183UK"
}, {
"model" : "Cooper",
"year" : 2018,
"type" : "Hatchback",
"motorization" : "Diesel",
"colour" : "Silver",
"stageID" : "MGOP9183740194",
"productionID" : "BHS0928MDB8391",
"processID" : "884910421743POR4183UK"
} ]
}
JOLT transformation code used: [
{
"operation": "shift",
"spec": {
"productionLine": {
"*": {
"@": "[&1]",
"@(2,productionID)": "[&1].productionID",
"@(2,processID)": "[&1].processID"
}
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"@": "productionLine"
}
}
}
]
... View more
- Find more articles tagged with:
- Data Ingestion & Streaming
- How-ToTutorial
- jolt
- joltransformjson
- jolttransformjson
- NiFi
- nifi-processor
Labels:
07-05-2018
04:45 AM
@geoffrey Shelton Okot I have created ranger policy as you said. I gave all access to the Anjali user. I tried to access through hive. the result is, when I hit query ' use testh3;' no error. But when I hit query ' show tables;' then there is empty list(NO tables found).
... View more
02-02-2019
06:52 PM
Hi Roberto, So you mean you were not able to install Nifi through Ambari? When you install manually, which mean you managing it separately? example, start stop in command line and not through ambari?
... View more
04-03-2018
11:30 AM
Here's the issue... you need to install docker ./platform-info.sh: line 64: docker: command not found
... View more
02-15-2018
12:24 PM
1 Kudo
Short Description
Storm Service Check fails with a Kerberos error after performing an Ambari upgrade in a cluster that had Metron installed.
Article In a cluster where Metron was removed before upgrading Ambari server, the following error can occur when performing a Service Check in Storm service:
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: The TGT found is not renewable
at org.apache.storm.security.auth.kerberos.AutoTGT.populateCredentials(AutoTGT.java:1)
at org.apache.storm.StormSubmitter.populateCredentials(StormSubmitter.java:3)
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:748)
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:590)
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:239)
at storm.starter.WordCountTopology.main(WordCountTopology.java:492)
Caused by: java.lang.RuntimeException: The TGT found is not renewable
at org.apache.storm.security.auth.kerberos.AutoTGT.populateCredentials(AutoTGT.java:291)
Solution
Go to Ambari -> Storm Service -> Configs -> Custom storm-site and remove the following properties:
topology.auto-credentials=['org.apache.storm.security.auth.kerberos.
AutoTGT']
nimbus.credential.renewers.classes=['org.apache.storm.security.auth.kerberos.AutoTGT']
supervisor.run.worker.as.user=true
... View more
- Find more articles tagged with:
- ambari-server
- ambari-upgrade
- CyberSecurity
- Issue Resolution
- issue-resolution
- Kerberos
- Metron
- Storm
Labels: