Using Command-Line Security Tools from Apache NiFi

[#1]

Download URLCrazy (http://www.morningstarsecurity.com/downloads/urlcrazy-0.5.tar.gz)

An Example Command Line Run for URLCrazy

[root@tspanndev13 security]# ./url.sh dataflowdeveloper.com
Typo Type,Typo,Valid,Pop,DNS-A,CC-A,Country-A,DNS-MX,Extn
Character Omission,daaflowdeveloper.com,true,,,?,,com,
Character Omission,dataflodeveloper.com,true,,,?,,com,
Character Omission,dataflowdeeloper.com,true,,,?,,com,
Character Omission,dataflowdeveloer.com,true,,,?,,com,
Character Omission,dataflowdevelope.com,true,,,?,,com,
Character Omission,dataflowdeveloper.cm,true,,,?,,cm,
Character Omission,dataflowdeveloper.co,false,,,?,,,
Character Omission,dataflowdeveloper.om,false,,,?,,,
Character Omission,dataflowdevelopercom,false,,,?,,,
...

Shell Script to Call From Apache NiFi

/opt/demo/security/urlcrazy-0.5/urlcrazy -i -f csv -p $@

An Example Command Line Run for NSLookup

Non-authoritative answer:
sparkdeveloper.com text = "v=spf1 ip4:00.000.0.0/24 ip4:00.000.00.0/24 ip4:11.111.111.0/19 ?all"

The Final JSON Output:

{
  "path" : "./",
  "execution.command" : "/opt/demo/security/url.sh",
  "urlcrazy" : "Typo Type,Typo,Valid,Pop,DNS-A,CC-A,Country-A,DNS-MX,Extn\nCharacter Omission,sarkdeveloper.com,true,,,?,,com,\nCharacter Omission,spakdeveloper.com,true,,,?,,com,\nCharacter Omission,spardeveloper.com,true,,,?,,com,\nCharacter Omission,sparkdeeloper.com,true,,,?,,com,\nCharacter Omission,sparkdeveloer.com,true,,,?,,com,\nCharacter Omission,sparkdevelope.com,true,543,,?,,com,\nCharacter Omission,sparkdeveloper.cm,true,214000,,?,,cm,\nCharacter Omission,sparkdeveloper.co,false,,,?,,,\nCharacter Omission,sparkdeveloper.om,false,,,?,,,\nCharacter Omission,sparkdevelopercom,false,,,?,,,\nCharacter Omission,sparkdevelopr.com,true,,,?,,com,\nCharacter Omission,sparkdevelper.com,true,2190,,?,,com,\nCharacter Omission,sparkdeveoper.com,true,,,?,,com,\nCharacter Omission,sparkdevloper.com,true,2230,,?,,com,\nCharacter Omission,sparkdveloper.com,true,,,?,,com,\nCharacter Omission,sparkeveloper.com,true,,,?,,com,\nCharacter Omission,sprkdeveloper.com,true,,,?,,com,\nCharacter Repeat,spaarkdeveloper.com,true,,,?,,com,\nCharacter Repeat,sparkddeveloper.com,true,,,?,,com,\nCharacter Repeat,sparkdeeveloper.com,true,,,?,,com,\nCharacter Repeat,sparkdeveeloper.com,true,,,?,,com,\nCharacter Repeat,sparkdevelloper.com,true,,,?,,com,\nCharacter Repeat,sparkdevelooper.com,true,,,?,,com,\nCharacter Repeat,sparkdevelopeer.com,true,,,?,,com,\nCharacter Repeat,sparkdeveloper..com,false,,,?,,com,\nCharacter Repeat,sparkdeveloper.ccom,false,,,?,,,\nCharacter Repeat,sparkdeveloper.comm,false,,,?,,,\nCharacter Repeat,sparkdeveloper.coom,false,,,?,,,\nCharacter Repeat,sparkdeveloperr.com,true,2120,,?,,com,\nCharacter Repeat,sparkdevelopper.com,true,203,,?,,com,\nCharacter Repeat,sparkdevveloper.com,true,,,?,,com,\nCharacter Repeat,sparkkdeveloper.com,true,,,?,,com,\nCharacter Repeat,sparrkdeveloper.com,true,,,?,,com,\nCharacter Repeat,spparkdeveloper.com,true,,,?,,com,\nCharacter Repeat,ssparkdeveloper.com,true,,,?,,com,\nCharacter Swap,psarkdeveloper.com,true,,,?,,com,\nCharacter Swap,saprkdeveloper.com,true,,,?,,com,\nCharacter Swap,spakrdeveloper.com,true,,,?,,com,\nCharacter Swap,spardkeveloper.com,true,,,?,,com,\nCharacter Swap,sparkdeevloper.com,true,,,?,,com,\nCharacter Swap,sparkdeveloepr.com,true,,,?,,com,\nCharacter Swap,sparkdevelope.rcom,false,,,?,,,\nCharacter Swap,sparkdeveloper.cmo,false,,,?,,,\nCharacter Swap,sparkdeveloper.ocm,false,,,?,,,\nCharacter Swap,sparkdeveloperc.om,false,,,?,,,\nCharacter Swap,sparkdevelopre.com,true,,,?,,com,\nCharacter Swap,sparkdevelpoer.com,true,,,?,,com,\nCharacter Swap,sparkdeveolper.com,true,,,?,,com,\nCharacter Swap,sparkdevleoper.com,true,,,?,,com,\nCharacter Swap,sparkdveeloper.com,true,,,?,,com,\nCharacter Swap,sparkedveloper.com,true,,,?,,com,\nCharacter Swap,sprakdeveloper.com,true,18,,?,,com,\nCharacter Replacement,aparkdeveloper.com,true,129,,?,,com,\nCharacter Replacement,dparkdeveloper.com,true,,,?,,com,\nCharacter Replacement,soarkdeveloper.com,true,,,?,,com,\nCharacter Replacement,spaekdeveloper.com,true,,,?,,com,\nCharacter Replacement,sparjdeveloper.com,true,,,?,,com,\nCharacter Replacement,sparkdebeloper.com,true,,,?,,com,\nCharacter Replacement,sparkdeceloper.com,true,,,?,,com,\nCharacter Replacement,sparkdevekoper.com,true,,,?,,com,\nCharacter Replacement,sparkdeveliper.com,true,,,?,,com,\nCharacter Replacement,sparkdevelooer.com,true,92,,?,,com,\nCharacter Replacement,sparkdevelopee.com,true,,,?,,com,\nCharacter Replacement,sparkdeveloper.cim,false,,,?,,,\nCharacter Replacement,sparkdeveloper.con,false,,,?,,,\nCharacter Replacement,sparkdeveloper.cpm,false,,,?,,,\nCharacter Replacement,sparkdeveloper.vom,false,,,?,,,\nCharacter Replacement,sparkdeveloper.xom,false,,,?,,,\nCharacter Replacement,sparkdevelopet.com,true,,,?,,com,\nCharacter Replacement,sparkdeveloprr.com,true,,,?,,com,\nCharacter Replacement,sparkdevelopwr.com,true,,,?,,com,\nCharacter Replacement,sparkdevelpper.com,true,,,?,,com,\nCharacter Replacement,sparkdevrloper.com,true,,,?,,com,\nCharacter Replacement,sparkdevwloper.com,true,,,?,,com,\nCharacter Replacement,sparkdrveloper.com,true,,,?,,com,\nCharacter Replacement,sparkdwveloper.com,true,,,?,,com,\nCharacter Replacement,sparkfeveloper.com",
  "filename" : "4963644600105857",
  "execution.command.args" : "sparkdeveloper.com",
  "execution.status" : "0",
  "spf" : "Server:\t\t10.42.1.20\nAddress:\t10.42.1.20#53\n\nNon-authoritative answer:\nsparkdeveloper.com\ttext = \"v=spf1 ip4:38.113.1.0/24 ip4:38.113.20.0/24 ip4:65.254.224.0/19 ?all\"\n\nAuthoritative answers can be found from:\n\n",
  "execution.error" : "",
  "uuid" : "f13ca0f5-bac7-4da7-b5c3-8b1c145591bf",
  "url" : "sparkdeveloper.com",
  "enrich.dns.record0.group0" : "\"v=spf1 ip4:00.000.0.0/24 ip4:00.000.00.0/24 ip4:11.111.111.0/19 ?all\""
}

You can grab lots of different command line and REST results for augmenting existing data, tools and feeds.

An URL Crazy report is useful for intelligence on what other domains people may be squatting on that are close to yours. Often these can be used by spammers, malware and for other nefarious purposes.