Data Formats in ETL pipeline

• A data format is how that data is structured; for example, the data format can be a CSV format, Fixed width text format, MySQL table record, or a JSON Format.

• Depending on the Data Format, other settings may need to be specified for reading and writing data to relevant fields.

• The Data Format tab consists of three (3) sections:

  • Reader: This section contains details related to the data source, i.e. where the data is read from. This tab is selected by default.

  • Writer: This section includes details about the data destination, i.e., where the data is written to.

  • Field Mapping: This section maps the data fields from the source to the destination.