Skip to main content
TimothySpann
Platinum ⭐️⭐️⭐️
Platinum ⭐️⭐️⭐️
Nov 7, 2016 12:36 PM UTC

NIFI 1.x For Automatic Music Playing Pipelines

  • Oct 5, 2026 9:34 AM UTC
  • 0 replies
  • 1612 views
  • Translate All

Use Case

Before meetups, I wanted to play some music. NiFi seemed like a great choice for streaming free music through my Mac.

9180-dataflow.png

MIDI Command Line Player For OSX

brew install timidity

Then you can simply play MIDI files with timidity file.mid.

Microservice to Extract Links from Web Pages

Java 8 Source Code: https://github.com/tspannhw/linkextractor

The Spring Boot REST API accepts a URL, parses out mid files and returns JSON containing linking and descriptions.

Example REST Call to Service

curl -G -v "http://<urL>:8080/extract/url?url=http://www.midiworld.com/classic.htm/&type=mid"

Run the Microservice

java -Xms512m -Xmx2048m -Djava.net.preferIPv4Stack=true -jar target/linkextractor-0.0.1-SNAPSHOT.jar

Java Snippet Using JSoup to extract links from URL (HTML)

pLink = new PrintableLink();
pLink.setLink(link.attr("abs:href"));
pLink.setDescr(trim(link.text(), 100));
linksReturned.add(pLink);

Output

hdfs dfs -ls /music/*.mid
-rw-r--r-- 3 tspann hdfs 87 2016-11-07 03:51 /music/2_ase.mid
-rw-r--r-- 3 tspann hdfs 99 2016-11-07 03:51 /music/4_mtking.mid
-rw-r--r-- 3 tspann hdfs 105 2016-11-07 03:50 /music/EspanjaCaphriccoCatalan.mid
-rw-r--r-- 3 tspann hdfs 87 2016-11-07 03:50 /music/EspanjaPrelude.mid
-rw-r--r-- 3 tspann hdfs 162 2016-11-07 03:50 /music/J_M_Bach_Auf_lasst_uns_den_Herren_loben.mid
-rw-r--r-- 3 tspann hdfs 93 2016-11-07 03:52 /music/adelina.mid
-rw-r--r-- 3 tspann hdfs 95 2016-11-07 03:52 /music/aida_ii2.mid
-rw-r--r-- 3 tspann hdfs 89 2016-11-07 03:50 /music/al_adagi.mid
-rw-r--r-- 3 tspann hdfs 95 2016-11-07 03:52 /music/alborada.mid
-rw-r--r-- 3 tspann hdfs 82 2016-11-07 03:52 /music/aquarium.mid
-rw-r--r-- 3 tspann hdfs 105 2016-11-07 03:52 /music/barbero.mid
-rw-r--r-- 3 tspann hdfs 101 2016-11-07 03:51 /music/barimyst.mid
-rw-r--r-- 3 tspann hdfs 80 2016-11-07 03:52 /music/beevar2.mid
-rw-r--r-- 3 tspann hdfs 111 2016-11-07 03:50 /music/biz_arls.mid
-rw-r--r-- 3 tspann hdfs 94 2016-11-07 03:51 /music/blas1.mid
-rw-r--r-- 3 tspann hdfs 114 2016-11-07 03:50 /music/boccher.mid
-rw-r--r-- 3 tspann hdfs 78 2016-11-07 03:52 /music/bolero.mid
-rw-r--r-- 3 tspann hdfs 100 2016-11-07 03:51 /music/cantique.mid
-rw-r--r-- 3 tspann hdfs 88 2016-11-07 03:51 /music/carminab.mid
-rw-r--r-- 3 tspann hdfs 96 2016-11-07 03:51 /music/clairdelune.mid
-rw-r--r-- 3 tspann hdfs 87 2016-11-07 03:51 /music/cmveder.mid
-rw-r--r-- 3 tspann hdfs 108 2016-11-07 03:52 /music/coucou.mid
-rw-r--r-- 3 tspann hdfs 99 2016-11-07 03:51 /music/coup8a.mid
-rw-r--r-- 3 tspann hdfs 97 2016-11-07 03:51 /music/cpf-bird.mid


2016-11-06 22:57:20.095 ERROR 28694 --- [nio-8080-exec-1] com.dataflowdeveloper.DataController : Query:http://www.midiworld.com/classic.htm/ mid,IP:192.168.1.2 Browser:nifi-agent
2016-11-06 22:57:20.313 ERROR 28694 --- [nio-8080-exec-3] com.dataflowdeveloper.DataController : Query:http://www.midiworld.com/classic.htm/ mid,IP:192.168.1.2 Browser:nifi-agent
2016-11-06 22:57:20.500 ERROR 28694 --- [nio-8080-exec-5] com.dataflowdeveloper.DataController : Query:http://www.midiworld.com/classic.htm/ mid,IP:192.168.1.2 Browser:nifi-agent


ls -lt /opt/demo/midi | more
total 20456
-rw-r--r-- 1 tspann staff 117731 Nov 6 22:58 appspg13.mid
-rw-r--r-- 1 tspann staff 13449 Nov 6 22:58 intrlude.mid
-rw-r--r-- 1 tspann staff 8777 Nov 6 22:58 latalant.mid
-rw-r--r-- 1 tspann staff 1911 Nov 6 22:58 lbvar2.mid
-rw-r--r-- 1 tspann staff 2230 Nov 6 22:58 lbvar4.mid
-rw-r--r-- 1 tspann staff 1458 Nov 6 22:58 lbvar6ep.mid

NiFi Flow

GetHTTP: Call JSoup Microservice that converts HTML page full of MIDI links into JSON file of links and descriptions of MIDI files.

9174-gethttp.png

SplitJSON: Split that big JSON file into individual link, description pairs for working with individual songs.

9175-splitjson.png

EvaluateJSONPath: Use JSONPATH to pull out link and description as attributes.

9176-evaluatejsonpath.png

InvokeHTTP: Download the MIDI file from the link.

9177-invokehttp.png

UpdateAttribute: Give it a good file name.

9178-updateattribute.png

I just want the file name from the link (example http://sdfsdf:8080/test.mid

PutFile: Store the MIDI on the OSX filesystem.

ExecuteStreamCommand: Run the timidity CLI to play the MIDI file.

9179-executestreamcommand.png

Pass a link to the stored MIDI file to timidity player.

PutHDFS: Store the MIDI on HDP 2.5 HDFS.

References

http://jsonpath.com/

http://macappstore.org/timidity/