Re: How to export Atlas metadata (not stored on HD...

saba_baig · ‎08-04-2017

@Sharmadha Sainath Can you please guide me on this problem.

saba_baig · ‎08-03-2017

Hi, I know that using sqoop-bridge of Atlas we can export metadata from some hive tables stored on hdfs to any relational database e.g MySQL. But how can we export entities that have nothing to do with Hive? e.g we wrote a custom connector and imported all the metadata from a non-hdfs data source into Atlas as Entitites. Now, if we want to export that data into MySQL what can we do to achieve the job? How can we use sqoop-bridge implemented so far in Atlas for us?

saba_baig · ‎07-27-2017

imgpsh-fullsize.pngscreenshot-from-2017-07-26-11-33-17.png Hi, I'm getting this error in Apache atlas. I basically wrote a piece of code to fetch metadata from a database server such as mysql. when I tried to merge my changes into the code from Atlas's latest repository. I got following errors org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'services': Invocation of init method failed; nested exception is java.lang.RuntimeException: org.apache.atlas.AtlasException: Failed to start embedded kafka Can anyone please guide me? what did I missed? Though, if I do telnet localhost 9026.. I see some valid response...

saba_baig · ‎07-27-2017

screenshot-from-2017-07-26-11-33-17.png Hi, I'm getting this error in Apache atlas. I basically wrote a piece of code to fetch metadata from a database server such as mysql. when I tried to merge my changes into the code from Atlas's latest repository. I got following errors org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'services': Invocation of init method failed; nested exception is java.lang.RuntimeException: org.apache.atlas.AtlasException: Failed to start embedded kafka Can anyone please guide me? what did I missed?

saba_baig · ‎07-11-2017

Thanks @Sharmadha Sainath

saba_baig · ‎07-11-2017

@Sharmadha Sainath I said that I got invalid expression when fired the same query in the Atlas UI for basic search using query. Here it gave me Invalid Expression for the same query (sales_fa?t) that worked via api as mentioned earlier.

saba_baig · ‎07-11-2017

@Sharmadha Sainath You also mentioned that if we write anything in single quotes it gives you all the entities with that tag. I found this behaving the same in case of DSL (Advanced Search) Search for `Fact` in Basic Search But, searching for `Fact` in Basic (Search by Query) behaved differently... Question 1: If we follow what you just said then why am I seeing Log Data and Metric appearing there? Question 2: Secondly, you said "Only classification name is supported in single quotes." then, 2.1) What is a classification name? 2.2) Difference between tag and classification name? 2.3) What to mention in single quotes... Classification name or tag?

saba_baig · ‎07-11-2017

@Sharmadha Sainath I have an entity of type "Table" named as "sales_fact". I tried to get that using api http://localhost:21000/api/atlas/v2/search/basic?limit=25&includeDeletedEntities=true&query=sales_fa?t&typeName=Table =====Response ===== { "queryType": "BASIC", "type": "Table", "entities": [ { "typeName": "Table", "attributes": { "owner": "Joe", "qualifiedName": "sales_fact", "name": "sales_fact", "description": "sales fact table" }, "guid": "c9838518-041f-4759-b3e5-01147b1526d1", "status": "ACTIVE", "displayText": "sales_fact", "classificationNames": [ "Fact" ] } ]} But when I tried to the same search (Basic using query) I typed sales_fa?t and I got invalid Expression. I added a breakpoint in the API but it was never called. I think there is some validation applied on that field that omits any character like ? from the query before submitting it to API. Please guide. I think, this could be the reason Apache Lucene Analysis Pipeline

saba_baig · ‎07-11-2017

@Sharmadha Sainath could you please also tell me how does this works `Log Data` ??? what am I supposed to write in single quotes and how do you explain the results?

saba_baig · ‎07-11-2017

Queries like this are also not returning anything e.g using groupby with select

saba_baig · ‎07-11-2017

@Sharmadha Sainath Could you please also guide me how is it using groupby and other functions liek max, min with some example? Does their result reflects on UI or not? Also, what features e.g wildcards (? . * etc) are available in full-text provided by Atlas? and how can we use them (examples)?

saba_baig · ‎07-11-2017

@Sharmadha Sainath can you please guide me on this? as the method "searchUsingDslQuery" from EntityDiscoveryService.java is getting the result of the fetched attributes but then it only adds the attributes to the object of AtlasSearchResult...neither an AtlasVertex nor contains theInstance or theTrait.... Could you please guide me on this?

saba_baig · ‎07-10-2017

Hi, Can anyone please help me in understanding SELECT clause in DSL? Following mentioned query returns no rows though there is an entity named as "Reporting" of type "DB" having attributes as name and owner. DB where name="Reporting" select name, owner

saba_baig · ‎06-21-2017

Hi, I want to add one more column on the SearchResult page. I've seen from the router that route '!/search/searchResult': 'searchResult' this searchResult uses SearchDetailLayout (view) and it finally uses SearchResultLayoutView to render all of the columns on the page. Now, I want to add one more column here.. say "testCol". Could anyone please guide me on 1) How can I make my changes reflect on UI? 2) How is the information to be displayed here is fetched on UI?

saba_baig · ‎06-16-2017

@Sharmadha Sainath , @Ashutosh Mestry Could you please share your thoughts on this? Is there any hook available for MySQL or Oracle? If not then how can we make a Hook for MySQL or Oracle? What can be the process involved for creating it?

saba_baig · ‎06-15-2017

Thanks @Sharmadha Sainath for the quick and brief response :)Is there any hook available for MySQL or Oracle? If not then how can we make a Hook for MySQL or Oracle? What can be the process involved for creating it?

saba_baig · ‎06-15-2017

I want to know that if any change in Hive database occurs e.g we create table B from table A then is Hive DB going to notify Atlas Server on its own or HiveHook is going to check constantly in the Hive DB and pull the changes?

saba_baig · ‎06-14-2017

How Hive notifies Atlas about any DML/DDL operation in Atlas against which Atlas generates lineage? What is the whole flow? and what is the information that Hive sends to Atlas?

saba_baig · ‎05-12-2017

I've imported Apache Atlas code into Eclipse successfully but I don't know how can I run its test cases in order to understand its behaviour. e.g testRequestUser() in EntityJerseyResourceIT.java class

saba_baig · ‎05-08-2017

I'm also not able enough to see any lineage for insert query. e.g if Create table test(id int, name string); and I do insert into table test (id,name) values(1,'a'); I don't see any lineage against this.

saba_baig · ‎05-08-2017

saba_baig · ‎05-08-2017

@anaik Yeah I verified from hive-site.xml I've added their following property <property> <name>hive.exec.post.hooks</name> <value>org.apache.atlas.hive.hook.HiveHook</value> </property>

saba_baig · ‎05-05-2017

Hi, I imported my tables from hive source but now if I create any table using CTAS from an existing table I can't see any lineage showing that it has been created from some source table. also if I create any table and alter its schema then it does not reflect on atlas. e.g a newly added column via hiveQL does not appear in atlas even if I run import-hive command again.

saba_baig · ‎05-05-2017

This happens everytime

saba_baig · ‎05-05-2017

I'm able to see existing tables in atlas gui but unable to see any lineage for tables created via CREATE TABLE AS SELECT.

saba_baig · ‎05-05-2017

Hi, I've set up Apache atlas server (incubating -0.9) on my local ubuntu (16.04 LTS). I also did set up of Hadoop (2.8) single node with Apache Hive(1.2.2). I tried to follow steps written in Configure Hive Hook. But when I create another table from existing table li ke CTAS. Not only my hadoop goes away (i.e when I type jps it doesn't show me anything except jps running) but I'm also struck out of the user in which I performed all setup of Hadoop. Please help me on this...

saba_baig · ‎05-05-2017

@Vinod Bonthu This is my bashrc file, here I tried to set CLASSPATH=$CLASSPATH:/usr/local/hadoop/lib/*:. but it says that it's a non-existing directory. whereas in my /usr/local/hadoop folder their is a folder named as native which has files with .so extension. Please guide me how can I resolve my HADOOP_CLASSPATH variable? Any help on setting up Hive-client on locally installed Apache Atlas Server would be highly appreciated.

saba_baig · ‎05-04-2017

Hi, I've setup apache-atlas-0.9-incubating locally on Ubuntu 16.04 LTS. In order to import Hive metadata into atlas, I've setup a cluster of Hadoop 2.7 with Apache Hive 1.2.2 now when I execute shell script (import-hive.sh) from atlas/distro/target/apache-atlas-0.9-incubating-SNAPSHOT/bin. It gives me different exceptions for "(MIssing jars) ClassNotFound". I've tried to fix them by manually placing the jar under folder atlas/distro/target/apache-atlas-0.9-incubating-SNAPSHOT/hook/hive/atlas-hive-plugin-impl/ but every time there is a new missing jar. So, I think this is not the right solution. Please guide me on this urgently with very basic level guide as I'm a newbee... 🙂 Thanks

saba_baig · ‎04-26-2017

@Michael Young It would be really kind of you if you could also tell me that do I need to create new types for them as well? or I can use the existing ones? like the ones we get pre-loaded (Table, Column from quick_start.py script)

saba_baig · ‎04-26-2017

HI, It would be really if someone could just point me out to the location in code from where Atlas does the job of rendering data from BerekeleyDB that was loaded initially using quick_start.py Thanks

Online	Offline
Last Visited	‎08-04-2017 11:03 AM

Date Registered	‎04-17-2017 06:03 AM
Last Visited	‎08-04-2017 11:03 AM
Total Messages Posted	34
Total Kudos Received	3

Re: How to export Atlas metadata (not stored on HD...

How to export Atlas metadata (not stored on HDFS) ...

Could not start embedded kafka service in atlas

Could not start serviice "Embedded Kafka" in Apach...

Re: Select clause is not working in Atlas DSL

Re: Select clause is not working in Atlas DSL

Re: Select clause is not working in Atlas DSL

Re: Select clause is not working in Atlas DSL

Re: Select clause is not working in Atlas DSL

Re: Select clause is not working in Atlas DSL

Re: Select clause is not working in Atlas DSL

Re: Select clause is not working in Atlas DSL

Select clause is not working in Atlas DSL

How can we make any change in Apache Atlas views a...

Re: How Atlas is notified when there is a change i...

Re: How Atlas is notified when there is a change i...

Re: How Atlas is notified when there is a change i...

How Atlas is notified when there is a change in an...

WHich testing framework is used in Apache Atlas

Re: Unable to see Lineage for Hive tables created ...

How can we capture DML(insert) and alter operation...

Re: Unable to see Lineage for Hive tables created ...

Unable to see Lineage for Hive tables created via ...

Re: Unable to create Table as select from another ...

Re: Unable to create Table as select from another ...

Unable to create Table as select from another tabl...

Re: import-hive.sh gives classnotfound exceptions

import-hive.sh gives classnotfound exceptions

Re: How to import AWS Redshift metadata into Atlas...

How Atlas fetches data from BerkeleyDB and display...