1973
Posts
1225
Kudos Received
124
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
772 | 04-03-2024 06:39 AM | |
1424 | 01-12-2024 08:19 AM | |
771 | 12-07-2023 01:49 PM | |
1326 | 08-02-2023 07:30 AM | |
1920 | 03-29-2023 01:22 PM |
07-12-2023
08:38 AM
Definitely upgrade to the latest NiFI which is 1.22.
... View more
06-29-2023
11:47 AM
https://medium.com/@tspann/ingesting-events-into-dockerized-ibm-db2-jdbc-with-apache-nifi-f0ca452d1351 It works fine with QueryDatabaseTableRecord and PutDatabaseRecord
... View more
06-12-2023
05:06 PM
@Motimot Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.
... View more
05-30-2023
06:13 AM
https://dev.to/tspannhw/simple-change-data-capture-cdc-with-sql-selects-via-apache-nifi-flank-19m4 You can use the metadata database processors to list all tables in a database and then read all values from tables
... View more
04-27-2023
10:03 AM
Also I had one with a Kudu cache for calling Daily Med https://github.com/tspannhw/ApacheConAtHome2020/tree/main/flows/DailyMed https://www.datainmotion.dev/2021/01/flank-using-apache-kudu-as-cache-for.html
... View more
04-03-2023
08:12 AM
1 Kudo
CODE + COMMUNITY This week in FLaNK Stack Weekly, we have some events going on, a meetup in preparation and a lot of interesting new tools to explore. Please Join my meetup groups. We will be hybrid so if you are remote you can still see via zoom or Youtube. For those in the Princeton, New York City or Philadelphia are we will be in person as well. https://www.meetup.com/futureofdata-princeton/ https://www.meetup.com/futureofdata-newyork/ https://www.meetup.com/futureofdata-philadelphia I have a meetup in person at our San Francisco office. I will also be speaking at the Real-Time Analytics Summit that week. https://www.meetup.com/futureofdata-sanfrancisco/events/292453316/ This is Issue #77 and if you wish to look at all of our back issues, check them out in github. They are in a few different formats. https://github.com/tspannhw/FLiPStackWeekly I travel the world spreading the word of streaming, please join us. https://www.linkedin.com/pulse/schedule-2023-tim-spann-/ Videos These were the most interesting streaming videos of the week, check them out on Youtube. https://www.youtube.com/watch?v=iT60STl-Wuk https://www.youtube.com/watch?v=4X5Yky3CT6I&t=13s https://www.youtube.com/watch?v=V_DpqTo4bQ0 https://www.youtube.com/watch?v=p9-Y1PRYDn4&t=2s https://www.youtube.com/watch?v=s80sz3NWwHo Articles These were the most interesting articles of the week. https://community.cloudera.com/t5/What-s-New-Cloudera/Cloudera-DataFlow-Designer-for-self-service-data-flow/ba-p/366039 https://posthog.com/blog/dev-marketing-for-startups#its-ok-for-other-companies-to-be-much-better-than-you-at-social-media https://developerrelations.com/devrel-roundtable/looking-ahead-to-conference-season https://ossinsight.io/collections/chat-gpt-alternatives/ https://robertsahlin.substack.com/p/the-data-engineer-is-dead-long-live https://blogs.oracle.com/javamagazine/ https://www.infoq.com/articles/billions-messages-minute/? https://technology.amis.nl/big-data-database/apache-nifi-automating-tasks-using-nipyapi/ https://blog.twitter.com/engineering/en_us/topics/open-source/2023/twitter-recommendation-algorithm https://www.linkedin.com/posts/michael-kohs-27a17525_snowpipe-snowflake-nifi-activity-7047694779786084352-ArdL/ https://thenewstack.io/linkedin-unifies-stream-and-batch-processing-with-apache-beam/ Recent Talks This is my most recent talk at the Trenton Computer Festival, I spoke on Streaming. Trenton Computer Festival Pro https://www.slideshare.net/bunkertor/itpc-building-modern-data-streaming-apps https://www.youtube.com/watch?v=iT60STl-Wuk&list=PLIJGKvnQWB-u0SPXIwozegOWCG2V85WGe&index=12 Events I have a number of events coming up soon, check them out if you can. https://www.cloudera.com/about/events/evolve.html https://web.cvent.com/event/7598f981-2f7e-4915-b662-bd7be9b5f48d/summary?RefId=homepage_impact24 April 4-6, 2023: DevNexus: Atlanta, GA. In-Person. https://devnexus.com/ April 24-26, 2023: Real-Time Analytics Summit: San Francisco, CA. In-Person. https://rtasummit.com/ April 25, 2023: Future of Data Meetup: San Francisco, CA. In-Person. https://www.meetup.com/futureofdata-princeton/ https://www.meetup.com/futureofdata-sanfrancisco/events/292453316/ May 9, 2023: Garden State Java User Group. In-Person. New Jersey https://gsjug.org/ May 10-12, 2023: Open Source Summit North America. Virtual https://events.linuxfoundation.org/open-source-summit-north-america/ May 23, 2023: Pulsar Summit Europe. Virtual https://pulsar-summit.org/ Cloudera Events https://www.cloudera.com/about/events.html More Events: https://www.linkedin.com/pulse/schedule-2023-tim-spann-/ Code There are a couple of good demos with source code available, check them out. https://github.com/pdefusco/Oozie2CDE_Migration https://github.com/SuperEllipse/edge2ai_pred_maint https://github.com/tspannhw/FLaNK-AllTheStreams https://github.com/tspannhw/CloudDemo2023 Tools There are a lot of tools I have found in the open source to be very helpful. https://github.com/bencgreenberg/stackexchange-tutorial-themes https://github.com/jaymody/picoGPT https://regex.ai/ https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.20.0/org.apache.nifi.processors.standard.JoinEnrichment/additionalDetails.html https://clickhouse.com/docs/en/integrations/nifi https://github.com/TheoKanning/openai-java https://pyscript.net/ https://tmate.io/ https://github.com/dylanaraps/neofetch https://github.com/jesseduffield/lazydocker https://github.com/httpie/httpie https://github.com/GothenburgBitFactory/taskwarrior https://github.com/newsboat/newsboat https://github.com/jarun/ddgr https://github.com/cointop-sh/cointop https://github.com/Byron/dua-cli https://nicolargo.github.io/glances/ https://github.com/aristocratos/bpytop https://github.com/hacker1024/coretemp https://github.com/bcicen/ctop https://github.com/imsnif/bandwhich https://github.com/jbruchon/jdupes https://exiftool.org/ https://github.com/aria2/aria2 https://github.com/muesli/duf https://github.com/ajeetdsouza/zoxide https://github.com/PrefectHQ/marvin https://github.com/libAudioFlux/audioFlux https://github.com/jamesturk/scrapeghost/ https://gut-cli.dev/ https://yakgpt.vercel.app/ https://github.com/HamburgChimps/apple-notes-liberator https://www.cursor.so/ https://orbstack.dev/ https://a16z.com/2023/03/30/b2b-generative-ai-synthai/ https://github.com/twitter/the-algorithm https://github.com/twitter/the-algorithm-ml https://github.com/fipso/ccurl.sh https://donuts-are-good.github.io/shhhbb/ Thanks for reading, same time next week! © 2023 Tim Spann
... View more
Labels:
03-13-2023
08:42 AM
I am trying to use Nifi to send data via tcp to relevant filebeat modules, but I am finding that Nifi is adding unwanted data to the logs. Has anyone come across this problem if using Nifi to send to Elasticsearch before and if so, what might be a good solution to preventing Nifi from doing this? (separate pipeline in Elastic to drop/rename/add etc or to use Logstash from Nifi or add a new module to not pick up Nifi data...?)
... View more
12-07-2022
09:46 PM
While converting from json to avro format,how to get logicaltype in avro format. And to get logicaltype in avro format,what we need to add in json data .
... View more
09-27-2022
08:26 AM
@arkumari As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.
... View more