Foreign Unicode Characters Not Displaying in UI and Database in RSA Identity Governance and Lifecycle
Originally Published: 2017-06-13
Last Modified: 2023-10-06
Article Number
000065058
Applies To
RSA Product Set: RSA Identity Governance and Lifecycle
RSA Version/Condition: All
Issue
Some accounts or entitlements with foreign characters are not displaying in the user interface (UI). They look fine in the input file but end up corrupted in the database after being collected.

Here is an example added to a CSV file t_sample_perforce_accounts.csv. Notice the last entry has a French character 'ç': 
accountname,userid,accusermapid
Admin,bbyragani,1223
BLR-Intern,svejandla,122
BLR-Intern,cchennagiri,5666
P4-Guest,cccchennagiri,5666
Testç1,Sioned,5669
After collection:
Notice the question mark with black background in the account column.

It's also not getting loaded correctly in the database:
Looking at the entry in PV_ACCOUNT view

There are no related errors in the collection run but the account does not display properly in the Accounts Name column.
Raw Data from the Collection Run
Cause
The text file is not in a format that can handle the foreign characters. The format needs to be changed to an encoding that can handle it properly.
Resolution
In the example above the 'file' command can be used to display the current encoding  of the CSV file: 
$  file t_sample_perforce_accounts.csv
t_sample_perforce_accounts.csv: ISO-8859 text
In this case the ISO-8859 encoding does not handle all the French characters. Most text editors have an option for changing the encoding to another character set. You'll need to edit the file, change the encoding, then save it. In this example UTF-8 encoding will resolve the problem.

In VIM you can set the encoding as follows: 
accountname,userid,accusermapid
Admin,bbyragani,1223
BLR-Intern,svejandla,122
BLR-Intern,cchennagiri,5666
P4-Guest,cccchennagiri,5666
Testç1,Sioned,5669

:set fileencoding=utf8
Now the file command displays: 
$  file t_sample_perforce_accounts.csv
t_sample_perforce_accounts.csv: UTF-8 Unicode text
Now rerun the collector and it will be stored and displayed properly. The old version of the account is deleted.:
the account now loads properly. The old version shows as deleted.

Related Articles

Trending Articles

Don't see what you're looking for?