Custom Zeppelin interpreter not found (HDP Sandbox 2.4)

[#1]

Hello,

Context:

• HDP Sandbox 2.4
• Zeppelin 0.6.0 (pre-installed) -- zeppelin-server-0.6.0.2.4.0.0-169.jar specifically, although the Ambari UI reports version 0.0.5 (!).

I'm writing a custom interpreter for Zeppelin called "dummy" (does nothing interesting at the moment, hence the name). My com.dummy.DummyInterpreter class extends org.apache.zeppelin.interpreter.Interpreter, as it should. I've built it against package org.apache.zeppelin:zeppelin-interpreter:0.6.0 and installed its jar in directory interpreter/dummy.

The interpreter jar does contain a file called interpreter-setting.json at its "root" with the following contents:

[
    {
        "group": "dummy",
        "name": "dummy",
        "className": "com.dummy.DummyInterpreter",
        "properties": {
            "boolProp": {
                "envName": null,
                "propertyName": "dummy.boolProp",
                "defaultValue": "true",
                "description": "Boolean property"
            },
            "intProp": {
                "envName": null,
                "propertyName": "dummy.intProp",
                "defaultValue": "0",
                "description": "Integer property"
            }
        }
    }
]

I also added "com.dummy.DummyInterpreter" to setting "zeppelin.interpreters" in zeppelin-site.xml.

However, I can't get Zeppelin to load it: Zeppelin does pick the "dummy" folder when restarted but it doesn't find the interpreter class, as evidenced by the following log excerpt:

INFO - Reading /usr/hdp/current/zeppelin-server/lib/interpreter/ignite
INFO - Interpreter ignite.ignite found. class=org.apache.zeppelin.ignite.IgniteInterpreter
INFO - Interpreter ignite.ignitesql found. class=org.apache.zeppelin.ignite.IgniteSqlInterpreter
INFO - Reading /usr/hdp/current/zeppelin-server/lib/interpreter/dummy
INFO - Reading /usr/hdp/current/zeppelin-server/lib/interpreter/md
INFO - Interpreter md.md found. class=org.apache.zeppelin.markdown.Markdown

Unfortunately, there's no error reported...

And strangely enough, this output looks more like what Zeppelin 0.5.0 would print than 0.6.0 would, based on my reading the source of Zeppelin's InterpreterFactory...

What am I missing?

Any help would be greatly appreciated.

[#2 · Accepted solution]

The official 0.6.0 is 16 days old (https://github.com/apache/zeppelin/releases) so I expect that the version in the Sandbox is a "0.6.0-preview" maybe more a 0.5.6 (I actually don't know)

Did you try to add

  static {
    Interpreter.register("dummy", DummyInterpreter.class.getName());
  }

to your DummyInterpreter? I understand this was necessary before 0.6.0 (now deprecated but still supported in 0.6.0)

[#3]

Thanks for your reply.

Yes, I tried the static registration method as follows (in the DummyInterpreter class):

static {
    Interpreter.register("dummy", "dummy", DummyInterpreter.class.getName(),
        new InterpreterPropertyBuilder()
            .add("dummy.boolProp", "true", "Boolean property")
            .add("dummy.intProp", "0", "Integer property")
            .build()
    );
}

I also tried building my interpreter against version 0.5.0-incubating (with the above static registration) but there was no difference whatsoever: Zeppelin wouldn't load the interpreter.

I guess I'll have to do what I wanted to avoid: install Zeppelin "manually" on an HDP 2.4... ?

[#4]

I installed it manually, it was quite straightforward. However you need maven 3.3, else some npm stuff will fail.

I just did "mvn clean package -DskipTests"

I then copied conf/zeppelin-env.sh.template to conf/zeppelin-env.sh and added

export JAVA_HOME=/usr/jdk64/jdk1.8.0_60/
export SPARK_HOME=/usr/hdp/current/spark-client
export HADOOP_HOME=/usr/hdp/current/hadoop-client

and copied zeppelin-site.xml.template to zeppelin-site.xml and changed port to 9995

Plus in Zeppelin for the Spark interpreter I changed the "master" property to yarn-client.

Seems to work for me on a HDP 2.4.2 cluster

[#5]

Thank you for the installation details. I'm currently setting up a new VM on which I'll perform a manual install of Zeppelin; I'll post the results here.

[#6]

(Back from vacation.)

Installed Zeppelin 0.6.0 from source in a CentOS VM, and it does see my custom interpreter!

Thanks.