NiFi read and write avro files with groovy
Avro is a very commonly used binary row oriented file format, it has a very small footprint compared to text formats like CSV. Many processors like ExecuteSql with reads data from a database are returning the result in avro format. In this article I share a small groovy template for reading and writing avro files […]
Continue ReadingXML to tables (xml2csv) with NiFi and Groovy Part 2 of 2
All code used in this article can be downloaded from https://github.com/maxbback/nifi-xml/ Problem with XML and design for converting XML to CSV XML is a common format used by many applications, so it will be one of the formats you need to support in your Big Data platform. XML is a structured format that is good […]
Continue ReadingXML to tables (csv) with NiFi and Groovy Part 1 of 2
XMl is a common format used by many applications, so it will be one of the formats you need to support in your Big Data platform. I have divided up this article in two part Part 1 Preparation Part 2 Coding and a complete working setup in NiFi with groovy All code used in this […]
Continue ReadingLookup table to mask or extend a feed in NiFi with Groovy
If you want to use a lookup table in NiFi to mask or complement the data in a feed you can build a simple processor with Groovy. The groovy code can also be found here https://github.com/maxbback/nifi_lookuptable In this processor I use a DB pool service for looking up addresses to mask in my text. For each […]
Continue Readingcsv analyser with NiFi and groovy
When you want your users to bring their own data, you soon realize that they will bring any kind of data and you need to figure out what they want to load. As an analyst you like to work with structured data if you can and many times you have some ind of csv file […]
Continue ReadingJDBC connection with NiF and groovy
I am found of quick solutions and groovy is a convenient way of developing small scripts for extending NiFi when you need it. NiFi has good processors for extracting data from a database, but sometimes you need your own, which I will show in another post where I will use this code to make lookups […]
Continue ReadingExcel to csv converter processor in Nifi/Kylo
NiFi is a fantastic tool for moving data from one system to another, and in combination with Kylo self service front end it makes it easy to move data. Process data in NiFi is easy as it comes bundled with a lot of processors and the ability to extend with your own processors written in […]
Continue ReadingThe one day executive training, “Unifying Your Organization with a Goal”
The one day executive training, “Unifying Your Organization with a Goal”, with reference to the Goal Tree which is a central part of Logical Thinking Process. This unique training day will be Tuesday 5th of June, 2018 in our offices, the day before the regular LTP training. You may be interested to check the […]
Continue ReadingBig Data enabled AI possibilities in your company.
Many companies are building a Data Lake based on hadoop with streaming capabilities with spark, flink and storm this enables realtime analyzing and enrichment of the data to provide better customer relationship by showing them what they need right now or making offers that will support what they are looking for. Self driving cars are […]
Continue ReadingA New book THE LOGICAL THINKING PROCESS – AN EXECUTIVE SUMMARY
Are you struggling with solving a problem? Logical Thinking is a good starting point why not do as I am doing right now read the THE LOGICAL THINKING PROCESS – AN EXECUTIVE SUMMARY Get the book here Author: H. William Dettmer ISBN 978-1-947532-49-6 (softcover) 978-1-947532-67-0 (eBook) 74pages The Logical Thinking Process – An Executive Summary is […]
Continue Reading