Member since
09-18-2018
92
Posts
5
Kudos Received
0
Solutions
10-19-2018
10:23 AM
The JSON I have looks like this: {
"year":2018,
"type":"N",
"issued":"2018-10-17T20:16:00.000Z",
"affectedFIR":"ZOA",
"minimumFL":"000"
}
I'd like to figure out how to extract the value of issued, and decompose it into a JSON that looks like the following {
"issued_year":"2018",
"issued_month":"10",
"issued_day":"17",
"issued_hour":"20",
"issued_minute":"16",
"issued_second":"00"
}
... View more
10-19-2018
10:14 AM
Thank You. I am brand new to jolt and am finding it a bit of a confusing syntax. I will get there however. It seems powerful. Do you have a description of what the following is doing:
"@":"&2.&"
... View more
10-18-2018
03:23 PM
I think I'm getting close with the following spec: [
{
"operation": "shift",
"spec": {
"textNOTAM": {
"NOTAM": {
// simple match. Put the value '4' in the output under the "Rating" field
"issued": "&0",
"affectedFIR": "&0"
}
}
}
}, {
"operation": "default",
"spec": {}
}]
The output has extracted fields.. but it does not also contain the original
{
"issued" : "2018-10-17T20:16:00.000Z",
"affectedFIR" : "ZOA"
}
... View more
10-18-2018
12:45 PM
I have the following JSON. I'd like to transform the JSON by extracting the "issued" field ("issued":"2018-10-17T20:16:00.000Z") and outputting a new JSON that preserves the existing JSON, but appends a new JSON that would contain a new property "issued_datetime":"2018-10-17T20:16:00.000Z". I'd also like to extract the effectiveStart and effectiveEnd and add them to the top level of the output JSON as well. So the output JSON would be the input JSON, with 3 properties added to it (replicas of issued, effectiveStart, effectiveEnd). Could someone please help me with the JOLT transform that would facilitate this. Input JSON: {
"textNOTAM": {
"NOTAM": {
"id": "NOTAM_1_51125176",
"number": 108,
"year": 2018,
"type": "N",
"issued": "2018-10-17T20:16:00.000Z",
"affectedFIR": "ZOA",
"minimumFL": "000",
"maximumFL": 999,
"radius": 5,
"location": "SCK",
"effectiveStart": 201810201100,
"effectiveEnd": 201811032000,
"schedule": "SAT SUN",
"text": "SVC PCL RWY 11L/29R REDL OUT OF SERVICE SAT SUN 1100-2000",
"translation": [
{
"NOTAMTranslation": {
"id": "NT01_51125176",
"type": "LOCAL_FORMAT",
"simpleText": "!SCK 10/108 SCK SVC PCL RWY 11L/29R REDL OUT OF SERVICE SAT SUN 1100-2000 1810201100-1811032000"
}
},
{
"NOTAMTranslation": {
"id": "NT02_51125176",
"type": "OTHER:ICAO",
"formattedText": {
"div": "10/108 NOTAMR \r\nQ) ZOA/QLEXX/IV/NBO/A/000/999/3753N12114W005 \r\nA) KSCK \r\nB) 1810201100 \r\nC) 1811032000 \r\n D) SAT SUN 1100-2000 \r\n\r\nE) \r\nSVC PCL RWY 11L/29R REDL PCL OUT OF SERVICE"
}
}
}
]
}
}
}
<br> Desired Output JSON: {
"issued":"2018-10-17T20:16:00.000Z",
"effectiveStart":201810201100,
"effectiveEnd":201811032000, {
"textNOTAM": {
"NOTAM": {
"id": "NOTAM_1_51125176",
"number": 108,
"year": 2018,
"type": "N",
"issued": "2018-10-17T20:16:00.000Z",
"affectedFIR": "ZOA",
"minimumFL": "000",
"maximumFL": 999,
"radius": 5,
"location": "SCK",
"effectiveStart": 201810201100,
"effectiveEnd": 201811032000,
"schedule": "SAT SUN",
"text": "SVC PCL RWY 11L/29R REDL OUT OF SERVICE SAT SUN 1100-2000",
"translation": [
{
"NOTAMTranslation": {
"id": "NT01_51125176",
"type": "LOCAL_FORMAT",
"simpleText": "!SCK 10/108 SCK SVC PCL RWY 11L/29R REDL OUT OF SERVICE SAT SUN 1100-2000 1810201100-1811032000"
}
},
{
"NOTAMTranslation": {
"id": "NT02_51125176",
"type": "OTHER:ICAO",
"formattedText": {
"div": "10/108 NOTAMR \r\nQ) ZOA/QLEXX/IV/NBO/A/000/999/3753N12114W005 \r\nA) KSCK \r\nB) 1810201100 \r\nC) 1811032000 \r\n D) SAT SUN 1100-2000 \r\n\r\nE) \r\nSVC PCL RWY 11L/29R REDL PCL OUT OF SERVICE"
}
}
}
]
}
}
}
}
... View more
Labels:
- Labels:
-
Apache NiFi
10-17-2018
06:30 PM
to me.. this is not the simple approach.. its very limiting and its no simpler than using the jolttransformation suggested above.
... View more
10-17-2018
06:29 PM
this worked for me so easily.. i tried the replacetext suggested below.. but it replaced every {. Jolt seems like the clean and easy way to do this. Kudos @Matt Burgess
... View more
10-17-2018
11:38 AM
Follow up detail.. I have no problem doing a git push from the command line. This tells me that the failure is strictly within the nifi-registry. [root@sensis-90076 eids-registry]# pwd
/var/lib/nifiregistry/git/eids-registry
[root@sensis-90076 eids-registry]# ls
FAA_Terminal_Procedures_Publications hi
[root@sensis-90076 eids-registry]# git push
warning: push.default is unset; its implicit value is changing in
Git 2.0 from 'matching' to 'simple'. To squelch this message
and maintain the current behavior after the default changes, use:
git config --global push.default matching
To squelch this message and adopt the new behavior now, use:
git config --global push.default simple
See 'git help config' and search for 'push.default' for further information.
(the 'simple' mode was introduced in Git 1.7.11. Use the similar mode
'current' instead of 'simple' if you sometimes use older versions of Git)
git@mali.corp.sensis.com's password:
Counting objects: 34, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (30/30), done.
Writing objects: 100% (30/30), 3.18 KiB | 0 bytes/s, done.
Total 30 (delta 12), reused 0 (delta 0)
To git@mali.corp.sensis.com:eids-registry.git
35c4861..72fd26c master -> master
[root@sensis-90076 eids-registry]#
... View more
10-16-2018
07:38 PM
Ty @Bryan Bende. That is sensible answer.. the follow-up comment also makes sense. I do think that even with the GetHTTP processor.. just knowing when the process has begun is useful.. indeed i'd think thats useful detail from any processor. I'd think that a DEBUG bulletin at the right level.. could provide that for any processor. Perhaps its already there and I just missed it.
... View more
10-16-2018
05:10 PM
The problem seems to relate to this stackoverlow post https://stackoverflow.com/questions/44346552/how-to-push-to-gitlab-with-java-api-with-ssh/44348972 https://stackoverflow.com/questions/44346552/how-to-push-to-gitlab-with-java-api-with-ssh/44348972#comment75923373_44348972
... View more
10-16-2018
03:00 PM
I've set up a nifi-registry.. and a GIT server to store my versioned flows in. I've changed my nifi-registry providers.xml to use a GIT flowPersistenceprovider
<flowPersistenceProvider>
<class>org.apache.nifi.registry.provider.flow.git.GitFlowPersistenceProvider</class>
<property name="Flow Storage Directory">/var/lib/nifiregistry/git/eids-registry</property>
<property name="Remote To Push">origin</property>
<property name="Remote Access User">git</property>
<property name="Remote Access Password">abc123</property>
</flowPersistenceProvider>
I've also cloned the repository seen in the Flow Storage Directory property [root@sensis-90076 conf]# cd /var/lib/nifiregistry/git/eids-registry/
[root@sensis-90076 eids-registry]# git status
# On branch master
nothing to commit, working directory clean
[root@sensis-90076 eids-registry]#
I see the following errors (tail -f nifi-registry-app.log) when I try to save a flow
2018-10-16 10:54:49,462 ERROR [GitFlowMetaData Push thread] o.a.n.r.p.flow.git.GitFlowMetaData Failed to push commits to origin due to org.eclipse.jgit.api.errors.TransportException: git@mali.corp.sensis.com:eids-registry.git: remote hung up unexpectedly
org.eclipse.jgit.api.errors.TransportException: git@mali.corp.sensis.com:eids-registry.git: remote hung up unexpectedly
at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:180) ~[org.eclipse.jgit-4.11.0.201803080745-r.jar:4.11.0.201803080745-r]
at org.apache.nifi.registry.provider.flow.git.GitFlowMetaData.lambda$startPushThread$1(GitFlowMetaData.java:220) ~[nifi-registry-framework-0.3.0.jar:0.3.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_181]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[na:1.8.0_181]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_181]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[na:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_181]
Caused by: org.eclipse.jgit.errors.TransportException: git@mali.corp.sensis.com:eids-registry.git: remote hung up unexpectedly
at org.eclipse.jgit.transport.TransportGitSsh$SshPushConnection.<init>(TransportGitSsh.java:363) ~[org.eclipse.jgit-4.11.0.201803080745-r.jar:4.11.0.201803080745-r]
at org.eclipse.jgit.transport.TransportGitSsh.openPush(TransportGitSsh.java:176) ~[org.eclipse.jgit-4.11.0.201803080745-r.jar:4.11.0.201803080745-r]
at org.eclipse.jgit.transport.PushProcess.execute(PushProcess.java:160) ~[org.eclipse.jgit-4.11.0.201803080745-r.jar:4.11.0.201803080745-r]
at org.eclipse.jgit.transport.Transport.push(Transport.java:1344) ~[org.eclipse.jgit-4.11.0.201803080745-r.jar:4.11.0.201803080745-r]
at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:169) ~[org.eclipse.jgit-4.11.0.201803080745-r.jar:4.11.0.201803080745-r]
... 8 common frames omitted
Caused by: org.eclipse.jgit.errors.UnsupportedCredentialItem: ssh://git@mali.corp.sensis.com:22: org.eclipse.jgit.transport.CredentialItem$YesNoType:The authenticity of host 'mali.corp.sensis.com' can't be established.
RSA key fingerprint is ac:77:d6:6b:28:5f:6c:70:b8:34:13:b1:4b:ec:ad:c0.
Are you sure you want to continue connecting?
at org.eclipse.jgit.transport.UsernamePasswordCredentialsProvider.get(UsernamePasswordCredentialsProvider.java:127) ~[org.eclipse.jgit-4.11.0.201803080745-r.jar:4.11.0.201803080745-r]
at org.eclipse.jgit.transport.CredentialsProviderUserInfo.promptYesNo(CredentialsProviderUserInfo.java:137) ~[org.eclipse.jgit-4.11.0.201803080745-r.jar:4.11.0.201803080745-r]
at com.jcraft.jsch.Session.checkHost(Session.java:793) ~[jsch-0.1.54.jar:na]
at com.jcraft.jsch.Session.connect(Session.java:345) ~[jsch-0.1.54.jar:na]
at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:136) ~[org.eclipse.jgit-4.11.0.201803080745-r.jar:4.11.0.201803080745-r]
at org.eclipse.jgit.transport.SshTransport.getSession(SshTransport.java:140) ~[org.eclipse.jgit-4.11.0.201803080745-r.jar:4.11.0.201803080745-r]
at org.eclipse.jgit.transport.TransportGitSsh$SshPushConnection.<init>(TransportGitSsh.java:338) ~[org.eclipse.jgit-4.11.0.201803080745-r.jar:4.11.0.201803080745-r]
... 12 common frames omitted
... View more
Labels:
- Labels:
-
Apache NiFi