Building schemas is tedious work and is often fraught with errors. The InferAvroSchema processor can get you started. It generates a compliant schema for use. There is one caveat: you have to make sure you are using Apache Avro-safe field names. I have a custom processor that will clean your attributes if you need them to be Avro-safe. See the processor listed below.

Example flow utilizing InferAvroSchema:

InferAvroSchema details:

The steps are as follows:

1. Use Apache NiFi to convert data to JSON or CSV.

2. Send JSON or CSV data to InferAvroSchema. I recommend setting the output destination to flowfile-attribute, input content type to json, and the pretty Avro output to true.

3. The new schema is now in the following attribute format: inferred.avro.schema.

inferred.avro.schema    
{ "type" : "record", "name" : "schema1", 
 "fields" : [ { 
   "name" : "table", "type" : "string", 
   "doc" : "Type inferred from '\"schema1.tableName\"'" } ] 
}   

This schema can then be used for conversions directly or can be stored in Hortonworks Schema Registry or Apache NiFi's built-in Avro Registry.

Now, you can use it for ConvertRecord, QueryRecord, and other Record processing.

Example generated schema in Avro-JSON format stored in Hortonworks Schema Registry:

Source

And that's it!