Note: Cloudera does not support antivirus software of any kind.   This article contains general recommendations for excluding ZooKeeper components and directories from antivirus scans and monitoring.   The three primary locations you will want to exclude from antivirus are: Data directories: These can be very large, and therefore take a long time to scan; they can also be very write-heavy, and therefore suffer performance impacts or failures if the antivirus holds up writes. Log directories: These are write-heavy. Scratch directories: These are internal locations used by some services for writing temporary data, and can also cause performance impacts or failures if the antivirus holds up writes.   Note: ZooKeeper has a user-configurable data directory. I recommend you exclude it. This directory can be found by running the following command: # grep dataDir /etc/zookeeper/conf/zoo.cfg Consider excluding the following directories and all of their subdirectories:   Installation, Configuration, and Libraries     /usr/hdp /etc/hadoop     Runtime and Logging     /var/run/zookeeper /var/log/zookeeper     Note: HDFS, YARN, MapReduce and ZooKeeper are mutually interdependent and you are likely to experience unsatisfactory results if you fail to also exclude the other components. ... View more

Note: Cloudera does not support antivirus software of any kind.   This article contains general recommendations for excluding MapReduce components and directories from antivirus scans and monitoring.   The three primary locations you will want to exclude from antivirus are: Data directories: These can be very large, and therefore take a long time to scan; they can also be very write-heavy, and therefore suffer performance impacts or failures if the antivirus holds up writes. Log directories: These are write-heavy. Scratch directories: These are internal locations used by some services for writing temporary data, and can also cause performance impacts or failures if the antivirus holds up writes. Note: Some directories in MapReduce are user-configurable. I recommend you exclude them. These properties can be found in Ambari > YARN > Configs > Advanced, and this one in particular should be excluded:     mapreduce.jobhistory.recovery.store.leveldb.path     Consider excluding the following directories and all of their subdirectories:   Installation, Configuration, and Libraries     /usr/hdp /etc/hadoop /var/lib/hadoop-mapreduce     Runtime and Logging     /var/run/hadoop-mapreduce /var/log/hadoop-mapreduce     Note: HDFS, YARN, MapReduce, and ZooKeeper are mutually interdependent and you are likely to experience unsatisfactory results if you fail to also exclude the other components. ... View more

Note: Cloudera does not support antivirus software of any kind.   This article contains general recommendations for excluding YARN components and directories from antivirus scans and monitoring.   The three primary locations you will want to exclude from antivirus are: Data directories: These can be very large, and therefore take a long time to scan; they can also be very write-heavy, and therefore suffer performance impacts or failures if the antivirus holds up writes. Log directories: These are write-heavy. Scratch directories: These are internal locations used by some services for writing temporary data, and can also cause performance impacts or failures if the antivirus holds up writes. Note: The directories YARN uses are user-configurable. I recommend you exclude them. These properties can be found in Ambari > YARN > Configs > Advanced:   yarn.nodemanager.local-dirs yarn.nodemanager.log-dirs yarn.nodemanager.recovery.dir yarn.timeline-service.leveldb-state-store.path yarn.timeline-service.leveldb-timeline-store.path   Consider excluding the following directories and all of their subdirectories:   Installation, Configuration, and Libraries       /usr/hdp /etc/hadoop /var/lib/hadoop-yarn       Runtime and Logging       /var/run/hadoop-yarn /var/log/hadoop-yarn       Note: HDFS, YARN, MapReduce, and ZooKeeper are mutually interdependent and you are likely to experience unsatisfactory results if you fail to also exclude the other components. ... View more

Note: Cloudera does not support antivirus software of any kind.   This article contains general recommendations for excluding HDFS components and directories from antivirus scans and monitoring.   The three primary locations you will want to exclude from antivirus are: Data directories: These can be very large, and therefore take a long time to scan; they can also be very write-heavy, and therefore suffer performance impacts or failures if the antivirus holds up writes. Log directories: These are write-heavy. Scratch directories: These are internal locations used by some services for writing temporary data, and can also cause performance impacts or failures if the antivirus holds up writes. Note: The directories in HDFS are user-configurable. I recommend you exclude these, especially the data directory for the DataNode and the meta directories for the NameNode and JournalNode. These details can be found in the “hdfs-site.xml” file:       # grep -A1 "dir" /etc/hadoop/conf/hdfs-site.xml       Consider excluding the following directories and all of their subdirectories:   Installation, Configuration, and Libraries     /usr/hdp /etc/hadoop /var/lib/hadoop-hdfs     Runtime and Logging     /var/run/hadoop /var/log/hadoop     Scratch and Temp     /tmp/hadoop-hdfs     Note: HDFS, YARN, MapReduce and ZooKeeper are mutually interdependent and you are likely to experience unsatisfactory results if you fail to exclude the other components. ... View more

Note: Cloudera does not support antivirus software of any kind.   This article contains general recommendations for excluding Ambari components and directories from antivirus scans and monitoring.   The three primary locations you will want to exclude from antivirus are: Data directories: These can be very large, and therefore take a long time to scan; they can also be very write-heavy, and therefore suffer performance impacts or failures if the AV holds up writes. Log directories: These are write-heavy. Scratch directories: These are internal locations used by some services for writing temporary data, and can also cause performance impacts or failures if the AV holds up writes.   Note: Ambari has a special requirement in the form of a user-configurable database. I recommend you exclude this database. However, the details of this database are set on installation; the database may be colocated with ambari-server, or on a remote host. Consult with your database administrators for details on the path where the database information is stored; Ambari does not keep this information anywhere in its configuration. If you need details about which database Ambari is using, search for JDBC in the “amber.properties” file.   # grep 'jdbc' /etc/ambari-server/conf/ambari.properties   Consider excluding the following directories and all of their subdirectories:   Installation, Configuration, and Libraries   /usr/hdp /usr/lib/ambari-agent /usr/lib/ambari-server /etc/hadoop /etc/ambari-agent /etc/ambari-server /var/lib/ambari-agent /var/lib/ambari-server   Runtime and Logging   /var/run/ambari-agent /var/run/ambari-server /var/log/ambari-agent /var/log/ambari-server   ... View more

Note: Cloudera does not support antivirus software of any kind.   This article contains generic recommendations for excluding HDP components and directories from AV scans and monitoring. It is important to note that these recommendations do not apply to each service, and further, some services will have additional items to exclude which are unique to them. These details will be addressed in individual articles dedicated to the service in question.   The three primary locations you will want to exclude from antivirus are: Data directories: These can be very large, and therefore take a long time to scan; they can also be very write-heavy, and therefore suffer performance impacts or failures if the AV holds up writes. Log directories: These are write-heavy. Scratch directories: These are internal locations used by some services for writing temporary data, and can also cause performance impacts or failures if the AV holds up writes. Consider excluding the following directories and all of their subdirectories: Installation, Configuration, and Libraries /hadoop /usr/hdp /etc/hadoop /etc/<component> /var/lib/<component> Runtime and Logging /var/run/<component> /var/log/<component>   Scratch and Temp /var/tmp/<component> /tmp/<component>   Note: The <component> does not only refer to the service name, as a given service may have multiple daemons with their own directories. Example: ambari-agent and ambari-server.   Across HDP services there are also many user-configurable locations. Most of these can be found in Ambari properties with names like "service.scratch.dir" and "service.data.dir"; go to Ambari > Service > Configs > Advanced and search for any property containing "dir", all of which may be considered for exclusion. ... View more