question Re: How to redirect the access logs and other logs to external file system? in Archives of Support Questions (Read Only)

How to redirect the access logs and other logs to external file system?

darpan_be — Fri, 12 Feb 2016 19:31:08 GMT

Hi folks,

One of the requirements is to redirect the all kind of logs/ ranger logs/access logs and also other component's logs to external file system. Probably on NFS and not on HDFS or in the DB. Does HDP provide out of the box solution for this?

One workaround I can think of using Flume but wanted to know other approaches?

Regards,

Re: How to redirect the access logs and other logs to external file system?

nsabharwal — Fri, 12 Feb 2016 19:35:38 GMT

@Darpan Patel

1) https://github.com/abajwa-hw/logsearch-service unofficial

2) You can use NFS to copy logs

3) Ranger and Solr works together for auditing

Re: How to redirect the access logs and other logs to external file system?

aervits — Fri, 12 Feb 2016 20:30:11 GMT

@Darpan Patel flume is one directional, you can't push data out of hdfs to fs. We have Apache NiFi that you can leverage for moving logs to and from hdfs.