Category:

Data Warehouse benchmark in the cloud

Are you looking into moving your DataLake or Data Warehouse to the cloud? I have collected a few articles that would be interesting for you, but do not trust that they bring the whole trough, make sure you investigate and take a sane decision. One of the key things I recommend is to secure that […]

Continue Reading
Posted On :
Category:

Simplify service integration

Point to point integration is hard to maintain when consumers increase and have slightly different demands. And how can we keep a fast delivery of our own service without the synchronized upgrades of multiple services at the same time to solve point to point integration dependency. One way is to use a streaming platform for […]

Continue Reading
Posted On :
xml2avro processor

XML2Avro NiFi processor

Avro is a very good record oriented compact format and is easy to work with, this processor is a version of the xml2csv processor that I published a few weeks ago, but is improved and is now generating avro files instead of csv files. All code used in this article can be downloaded from https://github.com/maxbback/xml2avro […]

Continue Reading
Posted On :

NiFi version control & deployment automation

Upgrade all instances of your NiFi process groups with automation script NiFi has a nice registry function to manage versioning of process groups it is called nifi-registry https://nifi.apache.org/registry.html In this article I will show how you can maintain versions with nifi-registry and how you can upgrade or downgrade all your instances of your process groups. […]

Continue Reading
Posted On :

BigData GDPR, PII Anonymization, pseudonymisation & FPE of data

In GDPR Article 32 and Article 4 anonymization & pseudonymisation is mentioned as methods of securing personal information. http://www.privacy-regulation.eu/en/article-32-security-of-processing-GDPR.htm http://www.privacy-regulation.eu/en/article-4-definitions-GDPR.htm Anonymization of the data secures that the data cannot be used to identify an individual by masking or encrypting the data in a way that it cannot be reversed back to its origin. There is […]

Continue Reading
Posted On :