Member since
07-30-2019
53
Posts
136
Kudos Received
16
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 11455 | 01-30-2017 05:05 PM | |
| 6748 | 01-13-2017 03:46 PM | |
| 3236 | 01-09-2017 05:36 PM | |
| 2141 | 01-09-2017 05:29 PM | |
| 1543 | 10-07-2016 03:34 PM |
09-30-2024
07:40 AM
非安全集群被阻止rpc通信,使用webhdfs协议,hadoop distcp -D ipc.client.fallback-to-simple-auth-allowed=true webhdfs://nn1:50070/foo/bar hdfs://nn2:8020/bar/foo
... View more
05-04-2023
02:43 AM
Hi Team, i have added Hbase client and phoenix core jar which has hbase client jar within it. I have the hbase and core site xmls. i get the Hconnection established message. But then i get "MESSAGE":"Reading reply sessionid:0x200239e26e51663, packet:: clientPath:/hbase serverPath:/hbase finished:false header:: 16,8 replyHeader:: 16,313534022349,0 request:: '/hbase,F response:: v{'meta-region-server,'rs,'splitWAL,'backup-masters,'flush-table-proc,'master-maintenance,'online-snapshot,'switch,'master,'running,'tokenauth,'draining,'namespace,'hbaseid,'table What could be the issue
... View more
09-17-2019
11:02 AM
Hi @dstreev Thanks for your article, I was checking and correct me if I'm wrong, but the same could be done using Knox service, that comes by default with HDP, it's that correct? Or there is some extra feature with this service? Regards Gerard
... View more
06-29-2017
10:50 PM
12 Kudos
@Saurab Dahal - As in the other answers the change in hdfs-site.xml will only affect the newly created files with the new replication factor. If you want to change replication factor for already existing files. You can run the hdfs cli commands like below: hdfs dfs -setrep -w 1 /myfile
hdfs dfs -setrep -w 1 /mydir
Hope this helps !
... View more
01-30-2017
05:05 PM
Rahul, Are the logs making it to HDFS? It sounds like you might be combining the "spooling" directory with the "local audit archive directory". What properties did you use during the Ranger HDFS Plugin installation? Are you doing a manual install or using Ambari? If manual, then this reference might help: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_command-line-installation/content/installing_ranger_plugins.html#installing_ranger_hdfs_plugin I wasn't able to locate your "...filespool.archive.dir" property on my cluster. I'm not sure the property is required. And may be responsible for keeping the files "locally" that you've already posted to HDFS. If the files are making it to HDFS, I would try removing this setting. What do you have set for the property below? And are the contents being flushed from that location on a regular basis? xasecure.audit.destination.hdfs.batch.filespool.dir Compression doesn't happen during this process. Once they're on HDFS, you're free to do with them as you see fit. If compression is a part of that, then write an MR job to do so. (WARNING: Could affect other systems that might want to use these files as is) Cheers, David
... View more
01-15-2017
03:00 PM
This worked: [hdfs@test232 ~]$ hadoop distcp -Dhadoop.security.credential.provider.path=jceks://hdfs/aws/aws.jceks /test s3a://kartik-test/ Thanks for all your help!!
... View more
01-09-2017
05:41 PM
Thanks for your message. Ambari version : 2.2.0.0. I have removed the rpcbind-0.2.0-38.el7.x86_64 package and tried to upgrade the HDP and it worked for me.
... View more
07-16-2018
11:34 AM
There is challenge I am facing .. when I am running the snapshotdiff from a remote cluster it is failing with snapshot not found error even though it is available .. do we have any solution for this .. we built a DR cluster and running distcp from DR to utilize the DR resources instead of overloading the PROD .. any solution how this can be achived..
... View more
11-29-2017
04:15 PM
For those who configure a connection to Hive the first time the Database Connection URL is available in Ambari> Service Hive> Sammary
... View more
11-24-2015
06:20 AM
1 Kudo
Well, it seems 600 characters wasn't enough for to supply enough of the back story @Peter Coates, so I added some high-level details in my original answer. I'd be glad to share more fine-grained info via email if you'd like.
... View more