Max Börebäck
Data mining and analytic is one of the hottest topics in the market, and I have the fortune to work in this sector. We can divide analytics into 3 main categories; Descriptive Predictive Prescriptive Descriptive is describing the past, what has happened and why, and also describing known facts. This is used in all reports, […]
Continue Reading
I often here phrases like, now when we are agile, can you help me with this now? Agile do not mean that we shall work ad hock and take in all request that is coming our way, its about; Being able to change direction faster Delivering increments to faster get response back of if we […]
Continue Reading
Avro is a very good record oriented compact format and is easy to work with, this processor is a version of the xml2csv processor that I published a few weeks ago, but is improved and is now generating avro files instead of csv files. All code used in this article can be downloaded from https://github.com/maxbback/xml2avro […]
Continue Reading
Upgrade all instances of your NiFi process groups with automation script NiFi has a nice registry function to manage versioning of process groups it is called nifi-registry https://nifi.apache.org/registry.html In this article I will show how you can maintain versions with nifi-registry and how you can upgrade or downgrade all your instances of your process groups. […]
Continue Reading
In GDPR Article 32 and Article 4 anonymization & pseudonymisation is mentioned as methods of securing personal information. http://www.privacy-regulation.eu/en/article-32-security-of-processing-GDPR.htm http://www.privacy-regulation.eu/en/article-4-definitions-GDPR.htm Anonymization of the data secures that the data cannot be used to identify an individual by masking or encrypting the data in a way that it cannot be reversed back to its origin. There is […]
Continue Reading
Avro is a very commonly used binary row oriented file format, it has a very small footprint compared to text formats like CSV. Many processors like ExecuteSql with reads data from a database are returning the result in avro format. In this article I share a small groovy template for reading and writing avro files […]
Continue Reading
All code used in this article can be downloaded from https://github.com/maxbback/nifi-xml/ Problem with XML and design for converting XML to CSV XML is a common format used by many applications, so it will be one of the formats you need to support in your Big Data platform. XML is a structured format that is good […]
Continue ReadingXMl is a common format used by many applications, so it will be one of the formats you need to support in your Big Data platform. I have divided up this article in two part Part 1 Preparation Part 2 Coding and a complete working setup in NiFi with groovy All code used in this […]
Continue Reading
If you want to use a lookup table in NiFi to mask or complement the data in a feed you can build a simple processor with Groovy. The groovy code can also be found here https://github.com/maxbback/nifi_lookuptable In this processor I use a DB pool service for looking up addresses to mask in my text. For each […]
Continue ReadingWhen you want your users to bring their own data, you soon realize that they will bring any kind of data and you need to figure out what they want to load. As an analyst you like to work with structured data if you can and many times you have some ind of csv file […]
Continue Reading