Member since
12-26-2018
3
Posts
0
Kudos Received
0
Solutions
04-10-2020
10:56 PM
Hi All,
When I am trying sample code provided on Cloudera documentation here. It gives me error message saying that Attributes error.
Error:
AttributeError: type object 'PandasUDFType' has no attribute 'GROUPED_AGG'
AttributeError Traceback (most recent call last)
in engine
----> 1 import ibis
/home/cdsw/.local/lib/python3.6/site-packages/ibis/__init__.py in <module>()
58 with suppress(ImportError):
59 # pip install ibis-framework[spark]
---> 60 import ibis.spark.api as spark # noqa: F401
61
62 with suppress(ImportError):
/home/cdsw/.local/lib/python3.6/site-packages/ibis/spark/api.py in <module>()
2 from ibis.spark.client import SparkClient
3 from ibis.spark.compiler import dialect # noqa: F401
----> 4 from ibis.spark.udf import udf # noqa: F401
5
6
/home/cdsw/.local/lib/python3.6/site-packages/ibis/spark/udf.py in <module>()
123
124
--> 125 class SparkPandasAggregateUDF(SparkPandasUDF):
126 base_class = SparkUDAFNode
127 pandas_udf_type = f.PandasUDFType.GROUPED_AGG
/home/cdsw/.local/lib/python3.6/site-packages/ibis/spark/udf.py in SparkPandasAggregateUDF()
125 class SparkPandasAggregateUDF(SparkPandasUDF):
126 base_class = SparkUDAFNode
--> 127 pandas_udf_type = f.PandasUDFType.GROUPED_AGG
128
129
AttributeError: type object 'PandasUDFType' has no attribute 'GROUPED_AGG'
Any help to resolve this issue?
Thanks,
CRP
... View more
Labels:
12-28-2018
05:48 AM
I don't think that it (Cloudera ODBC driver doesn't support insert) is true. By defining table as transcational table, you can insert data. CREATE TABLE insert_test( column1 string, column2 string) clustered by (column1) into 3 buckets stored as orcfile TBLPROPERTIES ('transactional'='true'); insert into table efvci_lnd_edw_dev.insert_test values('1', 'One'); insert into table efvci_lnd_edw_dev.insert_test values('2', 'Two'); Thanks, Chirag Patel
... View more