SmartConnect preview shows extra string characters in the header row.

Published: Feb 03, 2023
Post Author Written by eOne Solutions

Using a map with either a text or csv file, when I preview the map I see extra characters at the beginning of the first column name.   If I open the file it looks completely normal with no extra characters. 

Here is my preview from SmartConnect:

However this is my file when I open it in Notepad, where are these characters coming from?

These are hidden characters in a UTF-8-BOM file. The UTF-8-BOM is a sequence of bytes at the start of a text stream (0xEF, 0xBB, 0xBF) that allows the reader to more reliably guess a file as being encoded in UTF-8.

Normally, the BOM is used to signal the endianness of an encoding, but since endianness is irrelevant to UTF-8, the BOM is unnecessary.

According to the Unicode standard section 2.6, the use of a BOM is neither required nor recommended for UTF-8, but may be encountered in contexts where UTF-8 data is converted from other encoding forms that use a BOM or where the BOM is used as a UTF-8 signature.  

Link to the UNICODE standard document

SmartConnect will not work with UTF-8-BOM files, they need to be converted to standard UTF-8 encoding.

Feeling stuck? Get the support and guidance you need to help you power through any data challenge

We're on your integration team. Connect with our people and let us know how we can help you.