NAV 2009 RTC + Add-In problem: utf-8 encoded chars mixed-up

staub
Member Posts: 14
Hello,
I create Add-In and configure to use in Navision like in the blog: "Add-ins for the RoleTailored client of Dynamics NAV 2009 SP1 (part2)"
http://blogs.msdn.com/b/cabeln/archive/ ... part2.aspx
I send string from BigText, data prepared by XMLport with UTF-8. After in add-in received data going to save in a file, now that has specific chars mixed-up:
(source char ī (c4 ab) became (c2 b8)+(c2 bd) )
What should I do to get correct charset?
Thank you,
Yuri
I create Add-In and configure to use in Navision like in the blog: "Add-ins for the RoleTailored client of Dynamics NAV 2009 SP1 (part2)"
http://blogs.msdn.com/b/cabeln/archive/ ... part2.aspx
I send string from BigText, data prepared by XMLport with UTF-8. After in add-in received data going to save in a file, now that has specific chars mixed-up:
(source char ī (c4 ab) became (c2 b8)+(c2 bd) )
What should I do to get correct charset?
Thank you,
Yuri
Jurijs Staubs
Navision developer, (3 years in NAV)
Navision developer, (3 years in NAV)
0
Comments
-
I don't know if it's exactly the same problem but i had errors with charsets and bigtext var too.
Here is my blogpost about it:
http://devch.wordpress.com/2011/12/22/clean-workaround-for-special-characters-in-bigtext-variables/
you need 2009 R2 to get it work!0 -
You can encode your string on .NET side to Base64, and decode on NAV side. You can use standard xmldom methods to encode\decode Base64 string. The nature of problem is quite simple - NAV2009 does not support unicode.
BTW - say hello to Elva guys from me:)
edited:
Ok, didn't got, you sending a string from NAV - even simplier. Get rid of bigtext - use xmldom, and send entrie xmldom variable to your .NET application. So, stream out data from xml port, load to xmldom, send to .NET.0 -
Thank you Peter (ppavuk),
It is good to use xmldom object. But I choose other way, because of: no time to test new solution plus minimize changes on .Net part.
So, I use HTMLdecode/encode technique. It is good that I have nonASCII chars only at one direction from Nav to DLL. Algorithm is:
1. string stored in BigText I encode
2. then in .NET dll I decode received text
3. then general XML processing left the same – that mean in c# code only one additional statement: _xml = System.Web.HttpUtility.HtmlDecode(value);
The biggest problem is on Nav side, because as it was mentioned before by ppavuk: Navision works with ANSI charset - not utf-8. So it is challenge to encode utf-8 text (need to know the structure of that unicode), hereby in attachment is code for those who are interested.
P.S. special HELLO to Peter from Raimond.UTF8ToHTML(VAR BTextSrc : BigText;VAR BTextDest : BigText) : Integer ParseStrLen := MAXSTRLEN(ParseStr); DestStrLen := ParseStrLen; CLEAR(BTextDest); BTextSrcLen := BTextSrc.LENGTH; IF BTextSrcLen > 0 THEN BEGIN BTextSrcCurrPos := 1; WHILE BTextSrcCurrPos < BTextSrcLen DO BEGIN BTextSrc.GETSUBTEXT(ParseStr, BTextSrcCurrPos, ParseStrLen); ParseStrCurrLen := STRLEN(ParseStr); FOR I := 1 TO ParseStrCurrLen DO BEGIN CurrChar := ParseStr[I]; DestStrPart := ''; CASE CurrChar OF 0..127: //ASCII BEGIN DestStrPart := COPYSTR(ParseStr, I, 1); BytesUsedForUnicode := 1; BytesUsedForUnicodeCurr := 0; CLEAR(ByteArr); //UnicodeStr := ''; END; 128..191: //LAST BYTE for unicode BEGIN //UnicodeStr += Char2Hex(CurrChar); BytesUsedForUnicodeCurr += 1; Char2Byte(CurrChar, ByteArr[BytesUsedForUnicodeCurr]); IF BytesUsedForUnicodeCurr >= BytesUsedForUnicode THEN BEGIN UnicodeInt := 0; FOR j := BytesUsedForUnicode DOWNTO 1 DO BEGIN ByteArr[1, 8 - (BytesUsedForUnicode-j)] := FALSE; ByteArr[j, 8] := FALSE; UnicodeInt += Byte2BigInt(ByteArr[j], ((BytesUsedForUnicode-j)*64)); END; DestStrPart := '&#' + FORMAT(UnicodeInt) + ';'; END; BytesUsedForUnicode := 1; BytesUsedForUnicodeCurr := 0; CLEAR(ByteArr); //DestStrPart := '&#x' + FORMAT(UnicodeStr) + ';'; //UnicodeStr := ''; END; ELSE BEGIN //first byte for unicode //UnicodeStr += Hex(); CLEAR(ByteArr); BytesUsedForUnicodeCurr := 1; Char2Byte(CurrChar, ByteArr[BytesUsedForUnicodeCurr]); CASE CurrChar OF 192..223: BytesUsedForUnicode := 2; 224..239: BytesUsedForUnicode := 3; 240..247: BytesUsedForUnicode := 4; 248..251: BytesUsedForUnicode := 5; 252..253: BytesUsedForUnicode := 6; ELSE BytesUsedForUnicode := 1; END; END; END; DestStrPartCurrLen := STRLEN(DestStrPart); IF CurrChar = '9' THEN CurrChar := '9'; IF DestStrPartCurrLen > 0 THEN BEGIN IF (DestStrPartCurrLen + DestStrCurrLen) > DestStrLen THEN BEGIN BTextDest.ADDTEXT(DestStr); DestStr := DestStrPart; DestStrCurrLen := DestStrPartCurrLen; END ELSE BEGIN DestStr += DestStrPart; DestStrCurrLen += DestStrPartCurrLen; END; END; END; BTextSrcCurrPos += ParseStrLen; END; BTextDest.ADDTEXT(DestStr); END ELSE EXIT(1); EXIT(0); Char2Byte(VAR CharPar : Char;VAR Byte : ARRAY [8] OF Boolean) //1st bit (index 1) means 1st bit from the right CharInt := CharPar; Oper := 128; FOR i := 8 DOWNTO 1 DO BEGIN IF CharInt >= Oper THEN BEGIN Byte[i] := TRUE; CharInt -= Oper; END; Oper := Oper DIV 2; END; Char2Hex(VAR CharPar : Char) ResultHex : Text[2] //In Navision Char is coded by one byte so return value should be 2char only Char2Byte(CharPar, Byte); CLEAR(ByteHalf); FOR i := 1 TO 4 DO ByteHalf[i] := Byte[i+4]; Int := Byte2Int(ByteHalf); ResultHex := TinyInt2Hex(Int); CLEAR(ByteHalf); FOR i := 1 TO 4 DO ByteHalf[i] := Byte[i]; Int := Byte2Int(ByteHalf); ResultHex += TinyInt2Hex(Int); EXIT(ResultHex); Byte2Int(Byte : ARRAY [8] OF Boolean) IntResult : Integer //1st bit (index 0) means 1st bit from the right IntResult := 0; Oper := 1; FOR i := 1 TO 8 DO BEGIN IF Byte[i] THEN IntResult += Oper; Oper := Oper*2; END; EXIT(IntResult); Byte2BigInt(Byte : ARRAY [8] OF Boolean;Oper : BigInteger) IntResult : BigInteger //1st bit (index 0) means 1st bit from the right IntResult := 0; IF Oper = 0 THEN Oper := 1; FOR i := 1 TO 8 DO BEGIN IF Byte[i] THEN IntResult += Oper; Oper := Oper*2; END; EXIT(IntResult); TinyInt2Hex(Int : Integer) Result : Text[1] //Int supposed to be 0..15 CASE Int OF 10: Result := 'A'; 11: Result := 'B'; 12: Result := 'C'; 13: Result := 'D'; 14: Result := 'E'; 15: Result := 'F'; ELSE Result := FORMAT(Int); END;
Jurijs Staubs
Navision developer, (3 years in NAV)0 -
thx of course for the code. Am I correct in how it works.
You only have a problem sending data to C#
NAV
1.
fillup Bigtext with text
2.
encode with UTF8ToHTML
3.
Send result back to webservice
C#
1.
decode to html
(Just edited the steps)0 -
Why not to use 'bin.base64', which is just a 1 line of code? And of course you can do the same on your .NET application.
base64 obviosly do not contain any utf-8 char, and all encode\decode will be transparent, and moreover you can get rid of lots of code lines.
Let's have a look to pseudo-code below.encode := ''; objXmlNode := objXmlDom.createElement("tmp"); objXmlNode.datatype := "bin.base64"; objXmlNode.nodeTypedvalue := 'your value'; encode : = objXmlNode.Text; decode = '' objXmlNode := objXmlDom.createElement("tmp"); objXmlNode.datatype := "bin.base64"; objXmlNode.Text := base64Code; decode := objXmlNode.nodeTypedvalue;
Say hello to Raimonds too0 -
I like the idea
but this way we need to iterate through all the nodes to encode it?
0 -
Actually streamreader is nice for r2, but will not work in older versions. So, xml for old versions, streamreader for R2 +
And yes, all text parsing/converting code in C/AL is extremely slow! We have to avoid this if possible.0 -
mdPartnerNL wrote:I like the idea
but this way we need to iterate through all the nodes to encode it?
Actually not, you already iterating to create nodes. just add base64 property to node type. That's all.0 -
Ok, so to recap,
this problem does not occur in 2013
In NAV 2009 and earlier the problem only exist when sending data From NAV to ?0 -
mdPartnerNL wrote:Ok, so to recap,
this problem does not occur in 2013
In NAV 2009 and earlier the problem only exist when sending data From NAV to ?
to whatever external application, and reading from external apps.
In R2 conversion can be done by .NET stream reader,
In earliest versions - by XmlDom (base64)
There is also third way -CREATE(ADOStreamUTF); ADOStreamUTF.Open; ADOStreamUTF.LoadFromFile(Filename); ADOStreamUTF.LineSeparator := -1; ADOStreamUTF.Charset := 'UTF-8'; CREATE(ADOStreamANSI); ADOStreamANSI.Open; ADOStreamANSI.Type := 2; ADOStreamANSI.LineSeparator := -1; ADOStreamANSI.Charset := 'x-ansi'; StreamLen := ADOStreamUTF.Size; WHILE NOT ADOStreamUTF.EOS DO BEGIN ADOStreamANSI.WriteText(ADOStreamUTF.ReadText(1000)); END; ADOStreamUTF.Close; ADOStreamANSI.SaveToFile(TempFilename, 2); ADOStreamANSI.Close;
Maybe there is more ways, but i think 3 is enough
All this make sense if your locale is not US or not UK (or not whatever English)0 -
Just to clarify:
ADOStreamUTF and ADOStreamANSI are no dotNET variables but Automation variables linked to 'Microsoft ActiveX Data Objects 6.0 Library'.Stream'
And to get a working verion I also had to change the sequence of statements:CLEAR(ADOStreamUTF); CREATE(ADOStreamUTF); ADOStreamUTF.Charset := 'UTF-8'; ADOStreamUTF.Open; ADOStreamUTF.LoadFromFile(FilenameIn); ADOStreamUTF.LineSeparator := -1; CLEAR(ADOStreamANSI); CREATE(ADOStreamANSI); ADOStreamANSI.Charset := 'x-ansi'; ADOStreamANSI.Open; ADOStreamANSI.Type := 2; ADOStreamANSI.LineSeparator := -1; lintStreamLen := ADOStreamUTF.Size; WHILE NOT ADOStreamUTF.EOS DO BEGIN ADOStreamANSI.WriteText(ADOStreamUTF.ReadText(1000)); END; ADOStreamUTF.Close; ADOStreamANSI.SaveToFile(FilenameOut, 2); ADOStreamANSI.Close;
Now, let's see what we can see.
...
Everybody on-line.
...
Looking good!0
Categories
- All Categories
- 73 General
- 73 Announcements
- 66.6K Microsoft Dynamics NAV
- 18.7K NAV Three Tier
- 38.4K NAV/Navision Classic Client
- 3.6K Navision Attain
- 2.4K Navision Financials
- 116 Navision DOS
- 851 Navision e-Commerce
- 1K NAV Tips & Tricks
- 772 NAV Dutch speaking only
- 617 NAV Courses, Exams & Certification
- 2K Microsoft Dynamics-Other
- 1.5K Dynamics AX
- 320 Dynamics CRM
- 111 Dynamics GP
- 10 Dynamics SL
- 1.5K Other
- 990 SQL General
- 383 SQL Performance
- 34 SQL Tips & Tricks
- 35 Design Patterns (General & Best Practices)
- 1 Architectural Patterns
- 10 Design Patterns
- 5 Implementation Patterns
- 53 3rd Party Products, Services & Events
- 1.6K General
- 1.1K General Chat
- 1.6K Website
- 83 Testing
- 1.2K Download section
- 23 How Tos section
- 252 Feedback
- 12 NAV TechDays 2013 Sessions
- 13 NAV TechDays 2012 Sessions