Member since
12-10-2015
76
Posts
30
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2134 | 03-10-2021 08:35 AM | |
1550 | 07-25-2019 06:34 AM | |
3430 | 04-20-2016 10:03 AM | |
2618 | 04-11-2016 03:07 PM |
04-24-2018
11:50 AM
in your place I would have:
convert record to JSON not to CSV
use EvaluateJsonPath to set the attribute from the JSON in the contentFile use updateAttribute to transform datetime and user use ReplaceText to replace all contentFile with the altered attribute write the contentFile to FS
... View more
04-24-2018
11:49 AM
in your place I would have: convert record to JSON not to CSV use EvaluateJsonPath to set the attribute from the JSON in the contentFile use updateAttribute to transform datetime and user use ReplaceText to replace all contentFile with the altered attribute write the contentFile to FS
... View more
04-23-2018
03:00 PM
sorry for the mistake, obviously the operation is performed on the attribute, not on the FlowFile. So you must first set the values you need in the attributes and then transform them. Thanks @Matt Clarke
... View more
04-23-2018
01:58 PM
you must change the attribute on which to perform the transformation. I never used the GrokReader, but I believe you have to change ts to in timestamp in the EL string
... View more
04-23-2018
10:42 AM
1 Kudo
Hi @Wojtek,
I believe the problem is that the format() function works only with numbers. Your timestamp has a '.' (dot) which makes it interpret as a string.
To solve the problem I have adopted the following EL un a updateAttribute processor: ${ts:substringBefore('.'):append(${ts:substringAfter('.')})
:toNumber():format('MM/dd/yyyy HH:mm:ss.SSS')
}
ts attribute contains this value 1518442283.483 the result is 02/12/2018 13:31:23.483 I hope I have been of help
... View more
12-22-2017
04:02 PM
I have a cluster with 7 NiFi nodes. After a node crash, on restart, NiFi could not find the file /usr/hdf/current/nifi/conf/keystore.jks and truststore.jks I have re-created the files with tls-toolkit.sh client -c tp-hostname.domain.com -t passwordPassword -p 10443 In Ambari config the keystore and truststore pasword are empty. When I start the NiFi services Ihave: Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'protocolSocketConfiguration': FactoryBean threw exception on object creation; nested exception is java.io.IOException: Keystore was tampered with, or password was incorrect
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1585)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:317)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)
... 78 common frames omitted
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:780)
at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:56)
at sun.security.provider.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:224)
at sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad(JavaKeyStore.java:70)
at java.security.KeyStore.load(KeyStore.java:1445)
at org.apache.nifi.io.socket.SSLContextFactory.<init>(SSLContextFactory.java:65)
at org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:45)
at org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:30)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168)
... 83 common frames omitted
Caused by: java.security.UnrecoverableKeyException: Password verification failed
at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:778)
... 91 common frames omitted
In nifi.properties I have: nifi.security.keyPasswd=
nifi.security.keystore=/usr/hdf/current/nifi/conf/keystore.jks
nifi.security.keystorePasswd=
nifi.security.keystoreType=jks
nifi.security.needClientAuth=False
nifi.security.ocsp.responder.certificate=
nifi.security.ocsp.responder.url=
nifi.security.truststore=/usr/hdf/current/nifi/conf/truststore.jks
nifi.security.truststorePasswd=
nifi.security.truststoreType=jks
nifi.security.user.authorizer=ranger-provider
nifi.security.user.login.identity.provider=
nifi.sensitive.props.additional.keys=
nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
nifi.sensitive.props.key=sdlkjdslkjsdlkjdjjd||xyGZZ+R3FO04BxcUHSL5U6+OGqtQQevXbFfecQ
nifi.sensitive.props.key.protected=aes/gcm/256
nifi.sensitive.props.provider=BC On other NiFi nodes I have an encrypted password in nifi.properties, but the truststore and the keystore has an empty string as a password. Do you have any idea for this issue? Thanks in advance
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache NiFi
11-29-2016
11:59 AM
Hi guys, I see surfing the internet that the max numbers of files stored in HDFS is equals at Integer.MAX_VALUE of JVM. Any confirmation that the maximum number of files is this (2.147.483.647)?
... View more
Labels:
- Labels:
-
Apache Hadoop
04-26-2016
10:23 AM
1 Kudo
Ok, in advaced zookeeper.log4j set: log4j.appender.ROLLINGFILE.File=${zookeeper.log.dir}/zookeeper.log ${zookeeper.log.dir} is missing in defaul configuration.
... View more
04-23-2016
09:39 AM
Hi @Bernhard Walter, I tried to change log4j properties of zookeeper from ambari, but the server has not acquired the changes.
... View more
04-22-2016
01:06 PM
1 Kudo
Hi, I have HDP 2.4.0 with Ambari 2.2.1. I tried to modify log4j.properti by advancec-zookeeper-log4j but when you restart of zookeeper, the server has not acquired the changes. I saw that in the file zkEnv.sh if [ "x${ZOO_LOG4J_PROP}" = "x" ]
then
ZOO_LOG4J_PROP="INFO,ROLLINGFILE"
fi
I tried to set DEBUG but has not changed the logging level
... View more
Labels: