Member since
02-15-2019
18
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1176 | 03-22-2019 01:02 AM |
08-09-2019
03:02 AM
Is there some way to convert "file.creationtime" format to timestamp format? Like below: to
... View more
Labels:
- Labels:
-
Apache NiFi
07-29-2019
02:37 AM
Hi Geoffrey, Thank you very much. The problem is solved. I found that 50010 port (data transfer port to the HDFS Datanode ) is not added to HDP docker container. But the 50070 port (port to the HDFS namenode) is added. So I can read the metadata of Files in HDFS with the ListHDFS processor but can not get from HDFS and put files to HDFS with the two processors. I am using VirtualBox HDP sandbox, so I still use the default NAT network but changed the IP addresses to the current localhost IP at the setting of Port forwarding. And access to docker host terminal: root@sandbox-hdp.hortonworks.com 2200 Modified the sandbox proxy file generate-proxy-deploy-script.sh (/sandbox/proxy) Add 50010 port as follow, then run the shell program, ./ generate-proxy-deploy-script.sh, Then the proxy-deploy program is updated; Then run the shell (./proxy-deploy.sh) to generate a proxy container and start it (docker start sandbox-proxy). Run NiFi: Files are loaded in HDFS at HDP and fetched from HDFS to NiFi local directory:
... View more
07-28-2019
06:16 PM
Core-site.xml and hdfs-site.xml are attached. hdfs-site.xml core-site.xml
... View more
07-28-2019
06:16 PM
Hi Geoffrey, Thank you for your reply. Yes, I added these two files in NiFi. So ListHDFS processor in NiFi is able to show the list of files in HDFS. But PutHDFS can not put files in HDFS. I guess that the communication between NiFi and the namenode is ok, but something wrong between NiFi and the datanode. I can not know what configurations would affect that. Core-site.xml and hdfs-site.xml are attached. hdfs-site.xmlcore-site.xml The statuses from Ambari are shown below: Do you know where to get logs of Namenode and Datanode?
... View more
07-26-2019
05:01 AM
I used NiFi from HDF sandbox (PC A) to copy files to HDFS of HDP sandbox (PC B). NiFi can read files list from HDFS without any error. The flowchart is shown below: However, NiFi can not get files or put files on HDFS, the errors are shown below: GetHDFS processor error: PutHDFS processor error: Can anyone help me? Thank you very much.
... View more
07-26-2019
04:50 AM
@Aasha Medhi , could you please explain more details about which tcp port to open? How do you open it or them? Thanks.
... View more
03-27-2019
12:46 AM
The link is invalid. JOAO's link is valid now.
... View more
03-22-2019
01:02 AM
I got the results: The time is calculated in milliseconds from 1st Jan 1970. Python codes are shown below:
... View more
03-21-2019
06:12 AM
How to explain the createtime (1551243941685 ) and Timestamp ( Wed Feb 27 2019 16:05:41 GMT+1100 (Australian Eastern Daylight Time)? Thank you.
... View more
Labels:
- Labels:
-
Apache Atlas