Generated Hash is using UTF-8 Encoding, while XMLDoc will save in UTF-8 BOM Encoding

Falconettifer
Falconettifer Member Posts: 10
in NAV Three Tier
Hi,

I am using Microsoft Dynamics 365 Business Central, version 14.0.29530

I have to send an xml via HttpRequest that should have a validation hash.
This hash is generated based on the whole file to be sent, so these are the seps:

1) Generate XML (without Hash)
2) Write XML to InStream
3) Use EncryptionManagement codeunit to generate Hash (using Stream from step 2).
4) Generate XML again (as in step 1) adding the Hash from step 3 at the end of the xml.
5) Send XML from step 4 via HttpRequest

The idea is that the 3rd party system receiving my request, should read my request, remove the hash, calculate the hash again and check if my hash matches their hash.

The problem is that NAV will create the hash based on an InStream with UTF-8 Encoding, but in order to send via HttpRequest I will use XMLDoc.Load function, which will use UTF-8 BOM encoding. This means the calculation of the hash in the 3rd party system won't match my hash, because UTF-8-BOM encoding will add BOM character at the beginning of the transmissionf.

My questions: how can I generate hash using UTF-8 encoding and also have XMLDoc.Load using UTF-8 Encoding? can I use HttpRequest without usting XMLDoc, only my instream?

Remarks:
1) it is also ok if both hash and Http Request use UTF-8-BOM.
2) All declarations of xml, instreams and outsteams are using UTF-8.
3) I have also tried to generate hash using text
4) I am not using xmlport as this xml is using cdata sections


My Code (Steps described before will be added as comments):

//1) Generate XML (without Hash)
BuildPASPatientPacketFile(PrescriptionOrder,XmlDoc,''); //Generates XML, 3rd parameter is Hash, blank in this step

//2) Write XML to InStream
TempBlob.Blob.CREATEOUTSTREAM(outstr, TEXTENCODING::UTF8);
XmlDoc.Save(outstr);

//Set Instream for hash generation from exported data
TempBlob.Blob.CREATEINSTREAM(instr, TEXTENCODING::UTF8);

//3) Use EncryptionManagement codeunit to generate Hash (using Stream from step 2).
Hash := EncryptionManagement.GenerateHashFromStream(InputStream,4); //2nd Parameter, with value 4 = SHA512

//4) Generate XML again (as in step 1) adding the Hash from step 3 at the end of the xml.
CLEAR(XmlDoc);
BuildPASPatientPacketFile(PrescriptionOrder,XmlDoc,Hash); //3rd parameter is Hash.

TempBlob2.Blob.CREATEOUTSTREAM(ReqOutStream);
XmlDoc.Save(ReqOutStream);

//5) Send XML from step 4 via HttpRequest
TempBlob2.Blob.CREATEINSTREAM(ReqInStream);
END;

SendOrderRequest(ReqInStream,HttpWebRequest); //This function handles the http request, let's have a look:

LOCAL SendOrderRequest(VAR ReqInStream : InStream;VAR HttpWebRequest : DotNet "System.Net.HttpWebRequest")
HttpWebRequest := HttpWebRequest.Create(WarehouseIntegrationSetup."API URL (PAS Patient Package)");
HttpWebRequest.Method := 'POST';
//HttpWebRequest.ContentType := 'text/xml;charset=utf-8';
HttpWebRequest.ContentType('text/xml;charset=utf-8');
HttpWebRequest.KeepAlive := TRUE;
HttpWebRequest.AllowAutoRedirect := TRUE;
HttpWebRequest.UseDefaultCredentials := TRUE;
IF GlobalTimeout <= 0 THEN
GlobalTimeout := 600000;
HttpWebRequest.Timeout := GlobalTimeout;
HttpWebRequest.AutomaticDecompression := DecompressionMethods.GZip;

XmlDoc := XmlDoc.XmlDocument;
XmlDoc.Load(ReqInStream); //This instruction will load xmldoc using UTF-8-BOM encoding.
XmlDoc.Save(HttpWebRequest.GetRequestStream);



Thanks very much in advance!

Best Answer

  • Falconettifer
    Falconettifer Member Posts: 10
    Answer ✓
    Hi @ftornero

    Thanks, you gave to me the clue for this being solved.

    So, final solution consisted in:
    - generating Hash using StreamWriter with MemoryStream.
    - creating Web Request using StreamWriter with HTTPRequest

    This way, both hash and HTTPRequest are using StreamWriter, so none of them have UTF-8 BOM Preamble and are formatted the same way.

    So, this is the final solution (again, Steps described in original posts will be added as comments; I have written changes in bold, and functions signatures also in bold):

    //1) Generate XML (without Hash)
    BuildPASPatientPacketFile(PrescriptionOrder,XmlDoc,''); //Generates XML, 3rd parameter is Hash, blank in this step

    //2) Write XML to InStream
    TempBlob.Blob.CREATEOUTSTREAM(outstr, TEXTENCODING::UTF8);
    XmlDoc.Save(outstr);

    //Set Instream for hash generation from exported data
    TempBlob.Blob.CREATEINSTREAM(instr, TEXTENCODING::UTF8);

    //3) Use EncryptionManagement codeunit to generate Hash (using Stream from step 2).
    Hash := GenerateHashFromStreamWriter(Instr,FALSE);
    //Hash := EncryptionManagement.GenerateHashFromStream(InputStream,4); //2nd Parameter, with value 4 = SHA512

    //4) Generate XML again (as in step 1) adding the Hash from step 3 at the end of the xml.
    CLEAR(XmlDoc);
    BuildPASPatientPacketFile(PrescriptionOrder,XmlDoc,Hash); //3rd parameter is Hash.

    TempBlob2.Blob.CREATEOUTSTREAM(ReqOutStream);
    XmlDoc.Save(ReqOutStream);

    //5) Send XML from step 4 via HttpRequest
    TempBlob2.Blob.CREATEINSTREAM(ReqInStream);
    END;

    SendOrderRequestWithStreamWritter(ReqInStream,HttpWebRequest);
    //SendOrderRequest(ReqInStream,HttpWebRequest);


    //So, let's have a look to the things that changed, 2 functions:
    // - GenerateHashFromStreamWriter
    // - SendOrderRequestWithStreamWritter

    LOCAL GenerateHashFromStreamWriter(InputStream : InStream;WithPreamble : Boolean) Hash : Text
    //WithPreamble - this parameter will add UTF-8 BOM preamble if it is TRUE

    Encoding := Encoding.UTF8Encoding(WithPreamble);
    MemoryStream := MemoryStream.MemoryStream;
    StreamWriter := StreamWriter.StreamWriter(MemoryStream, Encoding);

    WHILE NOT InputStream.EOS DO BEGIN
    InputStream.READTEXT(AuxText);
    StreamWriter.WriteLine(AuxText);
    END;

    StreamWriter.Close;

    //0 = MD5, 1 = SHA1, 2 = SHA256, 3 = SHA384, 4 = SHA512
    Hash := GenerateHashFromMemoryStream(MemoryStream,4); //4 = SHA512


    LOCAL SendOrderRequestWithStreamWritter(VAR ReqInstream : InStream;VAR HttpWebRequest : DotNet "System.Net.HttpWebRequest")

    HttpWebRequest := HttpWebRequest.Create(WarehouseIntegrationSetup."API URL (PAS Patient Package)");
    HttpWebRequest.Method := 'POST';
    HttpWebRequest.ContentType('text/xml;charset=utf-8');
    HttpWebRequest.KeepAlive := TRUE;
    HttpWebRequest.AllowAutoRedirect := TRUE;
    HttpWebRequest.UseDefaultCredentials := TRUE;
    IF GlobalTimeout <= 0 THEN
    GlobalTimeout := 600000;
    HttpWebRequest.Timeout := GlobalTimeout;
    HttpWebRequest.AutomaticDecompression := DecompressionMethods.GZip;

    StreamWriter := StreamWriter.StreamWriter(HttpWebRequest.GetRequestStream);

    WHILE NOT ReqInstream.EOS DO BEGIN
    ReqInstream.READTEXT(AuxText);
    StreamWriter.WriteLine(AuxText);
    END;

    StreamWriter.Close;

Answers

  • ftornero
    ftornero Member Posts: 524
    Hello @Falconettifer,

    You can try to pass the Instream to a Text variable(Data in the code below) and send this last one.
    StreamWriter := StreamWriter.StreamWriter(HttpWebRequest.GetRequestStream);
StreamWriter.WriteLine(Data);
StreamWriter.Close;

    Where StreamWriter is
    Name	DataType	Subtype	Length
StreamWriter	DotNet	System.IO.StreamWriter.'mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

    Regards
  • Falconettifer
    Falconettifer Member Posts: 10
    ftornero wrote: »
    Hello @Falconettifer,

    You can try to pass the Instream to a Text variable(Data in the code below) and send this last one.
    StreamWriter := StreamWriter.StreamWriter(HttpWebRequest.GetRequestStream);
StreamWriter.WriteLine(Data);
StreamWriter.Close;

    Where StreamWriter is
    Name	DataType	Subtype	Length
StreamWriter	DotNet	System.IO.StreamWriter.'mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

    Regards

    Thanks for your reply,

    I'll try this. Do you know if I can set encoding UTF-8 BOM to the streamwriter?

    I've learn that I can set such encoding with this code, but I have done it to XMLWritter:

    XmlWriterSettings := XmlWriterSettings.XmlWriterSettings();
    Encoding := Encoding.UTF8Encoding(TRUE);
    XmlWriterSettings.Encoding := Encoding;
    XmlWriterSettings.Indent := TRUE;
  • Falconettifer
    Falconettifer Member Posts: 10
    Falconettifer wrote: »
    ftornero wrote: »
    Hello @Falconettifer,

    You can try to pass the Instream to a Text variable(Data in the code below) and send this last one.
    StreamWriter := StreamWriter.StreamWriter(HttpWebRequest.GetRequestStream);
StreamWriter.WriteLine(Data);
StreamWriter.Close;

    Where StreamWriter is
    Name	DataType	Subtype	Length
StreamWriter	DotNet	System.IO.StreamWriter.'mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

    Regards

    Thanks for your reply,

    I'll try this. Do you know if I can set encoding UTF-8 BOM to the streamwriter?

    I've learn that I can set such encoding with this code, but I have done it to XMLWritter:

    XmlWriterSettings := XmlWriterSettings.XmlWriterSettings();
    Encoding := Encoding.UTF8Encoding(TRUE);
    XmlWriterSettings.Encoding := Encoding;
    XmlWriterSettings.Indent := TRUE;

    I've seen that streamwriter has this constructor:

    [StreamWriter StreamWriter :=] StreamWriter.StreamWriter(Stream stream, System.Text.Encoding encoding)

    , I will check this way, thanks.
  • Falconettifer
    Falconettifer Member Posts: 10
    ftornero wrote: »
    Hello @Falconettifer,

    You can try to pass the Instream to a Text variable(Data in the code below) and send this last one.
    StreamWriter := StreamWriter.StreamWriter(HttpWebRequest.GetRequestStream);
StreamWriter.WriteLine(Data);
StreamWriter.Close;

    Where StreamWriter is
    Name	DataType	Subtype	Length
StreamWriter	DotNet	System.IO.StreamWriter.'mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

    Regards

    This almost worked.
    I have to read the stream and send call the WriteLine function while reading:

    WHILE NOT InputStream.EOS DO BEGIN
    InputStream.READTEXT(AuxText);
    StreamWriter.WriteLine(AuxText);
    END;

    Receiver has confirmed that they are not receiving the BOM preamble character
    Still something is not fully ok, I will update with new findings.
  • Falconettifer
    Falconettifer Member Posts: 10
    Answer ✓
    Hi @ftornero

    Thanks, you gave to me the clue for this being solved.

    So, final solution consisted in:
    - generating Hash using StreamWriter with MemoryStream.
    - creating Web Request using StreamWriter with HTTPRequest

    This way, both hash and HTTPRequest are using StreamWriter, so none of them have UTF-8 BOM Preamble and are formatted the same way.

    So, this is the final solution (again, Steps described in original posts will be added as comments; I have written changes in bold, and functions signatures also in bold):

    //1) Generate XML (without Hash)
    BuildPASPatientPacketFile(PrescriptionOrder,XmlDoc,''); //Generates XML, 3rd parameter is Hash, blank in this step

    //2) Write XML to InStream
    TempBlob.Blob.CREATEOUTSTREAM(outstr, TEXTENCODING::UTF8);
    XmlDoc.Save(outstr);

    //Set Instream for hash generation from exported data
    TempBlob.Blob.CREATEINSTREAM(instr, TEXTENCODING::UTF8);

    //3) Use EncryptionManagement codeunit to generate Hash (using Stream from step 2).
    Hash := GenerateHashFromStreamWriter(Instr,FALSE);
    //Hash := EncryptionManagement.GenerateHashFromStream(InputStream,4); //2nd Parameter, with value 4 = SHA512

    //4) Generate XML again (as in step 1) adding the Hash from step 3 at the end of the xml.
    CLEAR(XmlDoc);
    BuildPASPatientPacketFile(PrescriptionOrder,XmlDoc,Hash); //3rd parameter is Hash.

    TempBlob2.Blob.CREATEOUTSTREAM(ReqOutStream);
    XmlDoc.Save(ReqOutStream);

    //5) Send XML from step 4 via HttpRequest
    TempBlob2.Blob.CREATEINSTREAM(ReqInStream);
    END;

    SendOrderRequestWithStreamWritter(ReqInStream,HttpWebRequest);
    //SendOrderRequest(ReqInStream,HttpWebRequest);


    //So, let's have a look to the things that changed, 2 functions:
    // - GenerateHashFromStreamWriter
    // - SendOrderRequestWithStreamWritter

    LOCAL GenerateHashFromStreamWriter(InputStream : InStream;WithPreamble : Boolean) Hash : Text
    //WithPreamble - this parameter will add UTF-8 BOM preamble if it is TRUE

    Encoding := Encoding.UTF8Encoding(WithPreamble);
    MemoryStream := MemoryStream.MemoryStream;
    StreamWriter := StreamWriter.StreamWriter(MemoryStream, Encoding);

    WHILE NOT InputStream.EOS DO BEGIN
    InputStream.READTEXT(AuxText);
    StreamWriter.WriteLine(AuxText);
    END;

    StreamWriter.Close;

    //0 = MD5, 1 = SHA1, 2 = SHA256, 3 = SHA384, 4 = SHA512
    Hash := GenerateHashFromMemoryStream(MemoryStream,4); //4 = SHA512


    LOCAL SendOrderRequestWithStreamWritter(VAR ReqInstream : InStream;VAR HttpWebRequest : DotNet "System.Net.HttpWebRequest")

    HttpWebRequest := HttpWebRequest.Create(WarehouseIntegrationSetup."API URL (PAS Patient Package)");
    HttpWebRequest.Method := 'POST';
    HttpWebRequest.ContentType('text/xml;charset=utf-8');
    HttpWebRequest.KeepAlive := TRUE;
    HttpWebRequest.AllowAutoRedirect := TRUE;
    HttpWebRequest.UseDefaultCredentials := TRUE;
    IF GlobalTimeout <= 0 THEN
    GlobalTimeout := 600000;
    HttpWebRequest.Timeout := GlobalTimeout;
    HttpWebRequest.AutomaticDecompression := DecompressionMethods.GZip;

    StreamWriter := StreamWriter.StreamWriter(HttpWebRequest.GetRequestStream);

    WHILE NOT ReqInstream.EOS DO BEGIN
    ReqInstream.READTEXT(AuxText);
    StreamWriter.WriteLine(AuxText);
    END;

    StreamWriter.Close;

Categories