PR# 19385 Unicode symbols are not rendered properly by various tools

Problem Report Summary
Submitter: rosivaldo
Category: EiffelStudio
Priority: Low
Date: 2017/08/31
Class: Bug
Severity: Non-critical
Number: 19385
Release: GPL Edition - win64
Confidential: No
Status: Analyzed
Environment: win64
Synopsis: Unicode symbols are not rendered properly by various tools

Please have a look on this class.


	unicode_feature alias "?": like Current
			Result := Current

It compiles well, but the symbol ? of unicode_feature's alias faces several problems (the mentioned files are attached in
1) it disappears from "Basic text view" of "Feature" tool (Basic-text-view.png; indeed it is replaced by a null character);
2) it disappears from any printing (APPLICATION.pdf);
3) it is replaced by a question mark in any documentation made by "Generate documentation..." menu command (application.html and application_short.html);
4) if I invoke the "Search" tool (Ctrl+Alt+f) and do a search and replace of *anything*, the files altered on disk (i.e. those which are not open in "Editor" tool) have any unicode character like ? replaced by a null character (APPLICATION2.e).

Thanks in advance.

To Reproduce

Problem Report Interactions
From:alexk_es    Date:2017/09/26    Status: Analyzed    Download   
Printing with Unicode characters is supported in rev#100793 of EiffelStudio 17.11 intermediate release. To achieve that you need to
- use RTF backend (i.e. do not set "Use Postscript" in the preferences)
- select a font that supports all used character glyphs (e.g., it could be "Segoe UI Symbol" for non-keyword text)

From:alexk_es    Date:2017/09/25    Status: Analyzed    Download   
Thank you for your report.

The bug with the basic text view for a feature is fixed in rev#100788 of EiffelStudio 17.11 intermediate release.

We were able to reproduce the issue with the Search-Replace tool and printing, but there is no fix yet.

As to documentation generation, there is no quick fix because it requires changes to the current infrastructure. The documentation is generated using so called filters. You can find them in the directory .../studio/filters of you EiffelStudio installation. Everything is plain text and EiffelStudio does not know how to encode non-ASCII characters (if any). At the moment it relies on the current system encoding to encode non-ASCII. Therefore, there is no easy way to improve it under Windows because it does not allow for using UTF-8 as a system code page. But on Linux (and MacOS) it should be possible to specify a suitable locale and to run EiffelStudio with this environment setting. The appropriate locale is set with the commands

   export LC_ALL=en_US.UTF-8
Output truncated, Click download to get the full message

From:rosivaldo    Date:2017/08/31    Status: Open    Download   
After submission, the very text of my report got corrupted due to the presence of unicode characters. Please have a look at the attached class (application.e).

Attachment: application.e     Size:162
From:rosivaldo    Date:2017/08/31    Download   
Attachments for problem report #19385

Attachment:     Size:18555