Help with Hive Regex extract.

saikrishna_tara — Thu, 06 Apr 2017 23:38:35 GMT

Hi,

i have a firewall log with entries like this..

Mar 12 04:03:01 172.16.3.1 %ASA-6-106100 access-list FW-DATA permitted tcp FW-DATA 172.16.1.4 59289 OUTSIDE 52.87.195.145 22 hit-cnt 1 first hit

i created an external table in hive for this log file and i am trying to use HIVE SQL and regexp_extract to extract column out of this lines by using this regular expression.

([A-Z][a-z]+ \d+\s\d+:\d+:\d+)\s(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s+(\%ASA-6-106100):\s+(\w+-\w+\s+\w+-\w+)\s+(\w+)\s+(\w+)\s+(\w+-\w+)\/?(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})\((\d+)\)\s+->\s+(\w+-?\w+?)\/?(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})\((\d+)\)\s+(\w+-\w+\s+\d+\s+\w+\s+\w+)

when i tested this at different sites (like https://regex101.com/) ..it is working fine..breaking this line into many different groups shown below. but when i try to use that in HIVE its not working..any help is appriciated.

Mar 12 04:03:01

172.16.3.1

%ASA-6-106100

access-list FW-DATA

permitted

tcp

FW-DATA

172.16.1.4

59289

OUTSIDE

52.87.195.145

hit-cnt 1 first hit

Re: Help with Hive Regex extract.

namaheshwari — Fri, 07 Apr 2017 00:50:53 GMT

@Saikrishna Tarapareddy - I think you accidentally asked the same question twice:

https://community.hortonworks.com/questions/93488/help-with-hive-regex-extract.html

Can you please close this one.

question Help with Hive Regex extract. in Archives of Support Questions (Read Only)

Help with Hive Regex extract.

Re: Help with Hive Regex extract.