Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to redirect the access logs and other logs to external file system?

avatar
Rising Star

Hi folks,

One of the requirements is to redirect the all kind of logs/ ranger logs/access logs and also other component's logs to external file system. Probably on NFS and not on HDFS or in the DB. Does HDP provide out of the box solution for this?

One workaround I can think of using Flume but wanted to know other approaches?

Regards,

DP

1 ACCEPTED SOLUTION

avatar
Master Mentor

@Darpan Patel

1) https://github.com/abajwa-hw/logsearch-service unofficial

2) You can use NFS to copy logs

3) Ranger and Solr works together for auditing

View solution in original post

2 REPLIES 2

avatar
Master Mentor

@Darpan Patel

1) https://github.com/abajwa-hw/logsearch-service unofficial

2) You can use NFS to copy logs

3) Ranger and Solr works together for auditing

avatar
Master Mentor

@Darpan Patel flume is one directional, you can't push data out of hdfs to fs. We have Apache NiFi that you can leverage for moving logs to and from hdfs.