Conditional Split on String Variable condition

[#1]

I have a requirement to filter out specific string values like AB01, BC01,CD05 stored in a variable which can change later and I don't want to hard code this values in my conditional split comparison. Also I have a limitation not to filter these values in the source so I want to control it in Conditional split.

My condition is COMP_Name != (AB01, BC01,CD05)

Any help is highly appreciated.

Thank you

Saanvi S

[#2]

@Saanvi Sharma, you're doing this in Spark?

You can declare the variable as a list, for exampe - comp_list=['AB01','BC01']

Then you can filter it with rdd.filter(lambda x:not(x in comp_list)), if you're filtering on this string directly,

Otherwise you can put your field as an argument to lambda expression.

Similar concept may be applied if you're doing this any other way.