logo
Tags down

shadow

value manipulation in nifi flow


By : 吳柏瑜
Date : October 18 2020, 06:10 AM
wish of those help Once you capture the value then use Nifi expression language to add 5.
code :
${'$1':toNumber():plus(5)}


Share : facebook icon twitter icon

JSON Transformation with data manipulation using Apache NiFi


By : Joshua Cook
Date : March 29 2020, 07:55 AM
I think the issue was by ths following , I want to do some basic transformation to my below sample JSON, I want to change the value of the timeStamp tag to date format and want to add a new tag created_ts with value of current_timestamp to the my expected JSON output using NiFi. , data transformation not implemented.
check the official doc:
code :
shift       : copy data from the input tree and put it the output tree
default     : apply default values to the tree
remove      : remove data from the tree
sort        : sort the Map key values alphabetically ( for debugging and human readability )
cardinality : "fix" the cardinality of input data.  Eg, the "urls" element is usually a List, 
                    but if there is only one, then it is a String
attribute  |   value/expression
-----------------------------------------------------------
timeStamp  |   timeStamp:format('yyyy-MM-dd HH:mm:ss.SSS')
created_ts |   now():format('yyyy-MM-dd HH:mm:ss.SSS')
import groovy.json.JsonSlurper
import groovy.json.JsonBuilder

def ff = session.get()
if(!ff)return
ff = session.write(ff, {rawIn, rawOut->
    // transform streams into reader and writer
    rawIn.withReader("UTF-8"){reader->
        rawOut.withWriter("UTF-8"){writer->
            //parse reader into Map
            def json = new JsonSlurper().parse(reader)
            //change/set values
            json.timeStamp = new Date(json.timeStamp as Long).format('yyyy-MM-dd HH:mm:ss.SSS')
            json.created_ts = new Date().format('yyyy-MM-dd HH:mm:ss.SSS')
            //write changed object to writer
            new JsonBuilder(json).writeTo(writer)
        }
    }
} as StreamCallback)
session.transfer(ff, REL_SUCCESS)

Nifi PutMongo - How to insert document into MongoDB using attribute values of Flow File but not Flow File content?


By : Jeferson Camilo Jime
Date : March 29 2020, 07:55 AM
I wish this help you You can use a processor like AttributesToJson, or ReplaceText, to overwrite the flow file content with the attributes that you want to use for the Mongo document.

NiFi: Failed to connect node to cluster because local flow is different than cluster flow


By : user3478091
Date : March 29 2020, 07:55 AM
will help you This means there is a difference in authorizations.xml or users.xml, most likely authorizations.xml. I would try copying those two files from one of the other nodes over to the node that is having the problem, this will ensure they are exactly the same.

How to remove the flow file attributes in Nifi flow file?


By : Wesley Peter
Date : March 29 2020, 07:55 AM
hop of those help? It is hard to tell from the screenshots provided, but it looks like the fields you want to remove are part of the content of the flow file, which is different then the attributes of the flow file. UpdateAttribute can only remove attributes, not anything in the content.
In order to modify the content you would need to use a processor specific to the type of content being processed. In your case it looks like JSON, so you could use a ConvertRecord processor with a JsonTreeReader and JsonRecordSetWriter, and configure the writer to have a different schema then the reader. Basically read in all the fields, but only write out the fields you want.

NiFi How to query Content Repository from NiFi Flow


By : Lars Olesen
Date : March 29 2020, 07:55 AM
will be helpful for those in need You are mistaken, the content of the flow files is saved into the Content Repository. You don't need to query the Content Repository to use those FlowFiles.. I assume that you are using PutHiveQL, so just use the success relationship for your next flow.
Example: ExecuteSQL ->(success) Transform Process Group ->(output) PutHiveQL ->(success) -> Next Flow
shadow
Privacy Policy - Terms - Contact Us © 35dp-dentalpractice.co.uk