XML Data Enrichment

This Article outlines the purpose and procedure of XML Data Enrichment. From why it may be used to building a custom implementation in trellispark.

What is Data Enrichment?

To understand Data Enrichment, first 3 things must be established.

  1. Instance Data
  2. Deep XML
  3. Enriched XML

Instance Data

Instance Data is the immediate data for an instance including the Instance unique identifier (GUID), the Parent GUID, It’s place in the hierarchy, and all the data that is displayed on the form for that instance. If the instance has a field such as Email Address, the instance data will include the value for that field.

DeepXML

Deep XML includes the Instance Data for the current instance as well as for all instances below the current instance in the hierarchy. Deep XML can be retrieved in any trellispark implementation by referencing the XMLManager class in the GreatIdeaz.trellispark.Rest.Logic namespace in any API project or by calling the FormBuilder class available in the GreatIdeaz.trellispark.UX.WASM.Services namespace in any WASM project. The method name in both cases is ReadDeepXML.

Enriched XML

Enriched XML includes the Instance Data for the current instance and any additional data required for the current data processing task. The task may be building a document, report, or dashboard. XML becomes enriched by adding data from instances that aren’t included in the Instance Data. This data is normally pulled from the database directly by a stored procedure. Because the data required for any operation is usually very specific, write an individual API command for each implementation.

Implementing your own Data Enrichment API

While a generic data enrichment command is a possible future feature, currently it is not implemented, and any Data Enrichment process needs to be manually written.

To begin, either build your own web API project or use the ZEX-ExampleAPI sample project provided in the trellispark release zip file. I recommend you use the sample project as it includes the scaffolding for XML enrichment in trellispark.

When performing Data Enrichment, there are a few steps that should be taken:

  1. Read the current Instance.
    1. Normally done before beginning Data Enrichment.
    2. Used to validate user access to the instance.
    3. Used to ensure that Data Enrichment is performed with the most recent instance data.
  2. (Optional) Read the Deep XML for the instance.
    1. Includes information from all the children of the current instance.
  3. Retrieve the required data for the XML Enrichment.
    1. Usually done through an SQL stored procedure.
    2. Limiting the amount of data as much as possible is recommended to avoid bloating and reduce processing time.

Sample code

ExecuteCommand

public async Task<XMLEnrichment> ExecuteCommand(XMLEnrichment commandData)
{
    try
    {
        // Validate the user's access to the target instance
        InstanceInformation targetInstance = new();
        targetInstance.Initialise(commandData.SessionGUID, commandData.UserGUID);
        targetInstance.InstanceGUID = commandData.InstanceGUID;
        targetInstance.FormGeneration = false;
        targetInstance.RequiredVersion = 0;
        Instance myTargetInstance = new(DB);
        await myTargetInstance.ReadInstance(targetInstance);
        if (targetInstance.ErrorMessage != "")
        {
            commandData.ErrorMessage = targetInstance.ErrorMessage;
            await GreatIdeaz.trellispark.Rest.Logic.AppInsights.Log(0, "ZEX-REST-API.XMLEnrichmentCommand.ExecuteCommand", targetInstance.ErrorMessage, DB);
        }
        else
        {
            // Standard B64 decode of the input XML
            var b64Original = System.Convert.FromBase64String(commandData.B64Original);
            string original = Encoding.UTF8.GetString(b64Original);
            XElement originalXML = XElement.Parse(original);
            XElement enrichedXML = XElement.Parse(original);

            switch (commandData.CommandName)
            {
                case "":
                    break;

                default:
                    commandData.ErrorMessage = "ZEX-REST-API.XMLEnrichmentCommand.ExecuteCommand - Command not found";
                    break;
            }

            // Standard B64 encode of the output XML
            string enriched = enrichedXML.ToString();
            var b64Enriched = System.Text.Encoding.UTF8.GetBytes(enriched);
            commandData.B64Enriched = System.Convert.ToBase64String(b64Enriched);
        }

    }
    catch (Exception ex)
    {
        commandData.ErrorMessage = "ZEX-REST-API.XMLEnrichmentCommand.ExecuteCommand - Unknown exception";
        await GreatIdeaz.trellispark.Rest.Logic.AppInsights.Log(0, "ZEX-REST-API.XMLEnrichmentCommand.ExecuteCommand - Unknown exception", ex.ToString(), DB);
    }
    return commandData;
}


ExecuteCommand is the function that is used to control the XML enrichment process.

The first step of XML Enrichment is to read the current instance to validate the user’s access to the instance data.

Assuming that the user does have access to the underlying instance, then the original base 64 encoded XML should be unpacked and copied to the originalXML and enrichedXML variables.

The enrichedXML variable will ultimately be base64 encoded and returned. If no actual enrichment is required, the returned enrichedXML will be identical to the originalXML

The enrichment process is controlled by the switch statement. Depending upon the commandData.CommandName any set of specific enrichment tasks may be undertaken that convert the originalXML into the enrichedXML.

ExecuteTSQL

private async Task<XElement> ExecuteTSQL(InstanceInformation instanceInfo, string StoredProcedure, IDatabase myDB)
        {
            try
            {
                ExecuteSQLInformation mySQL = new();
                mySQL.Initialise(instanceInfo.SessionGUID, instanceInfo.UserGUID);
                mySQL.CurrentInstanceGUID = instanceInfo.InstanceGUID;
                mySQL.CommandName = StoredProcedure;
                mySQL.NewSQLVersion = false;
                mySQL.ReturnsDataset = true;

                await myDB.ExecuteTSQL(mySQL);

                if (mySQL.ErrorMessage != "")
                {
                    await myDB.LogEvent(1, "Process failed to execute " + StoredProcedure +
                        " InstanceGUID: " + instanceInfo.InstanceGUID.ToString(), mySQL.ErrorMessage);
                    throw new Exception("Failed to execute " + StoredProcedure + " " + mySQL.ErrorMessage);
                }

                return XElement.Parse(mySQL.Result);
            }
            catch (Exception ex)
            {
                await myDB.LogEvent(1, "Process failed to execute " + StoredProcedure +
                    " InstanceGUID: " + instanceInfo.InstanceGUID.ToString() + " Unknown Error: ", ex.ToString());
                throw new Exception("Failed to execute stored procedure" + ex.ToString());
            }
        }

XML enrichment tasks frequently include adding additional XElements to the XML which are retrieved from the DAS-RSS.

Updated on November 2, 2022

Was this article helpful?

Related Articles

Need Support?
Can’t find the answer you’re looking for? Don’t worry we’re here to help!
Contact Support

Leave a Comment