Category Archives: BizTalk - Page 2

System.Transactions.TransactionException: The operation is not valid for the state of the transaction

When running large amounts of data into an MS SQL database via a composite WCF-SQL send port in BizTalk I would eventually get (after about 20mins) the error “System.Transactions.TransactionException: The operation is not valid for the state of the transaction”. Smaller amounts of data ran fine.

The solution was to turn off the transaction in the send port. Of course this means you lose the transactional capabilities but do you really want to lock a table on a live database for 20 mins anyway? It may be good practice to only turn this feature off when you know you are running a long transaction and then turn it back on again when you are finished.

Turn_Off_Transactions_In_Send_Port

Securing a WCF Service in BizTalk by Role/Claim using Windows or Claims Based Authentication

There are loads of articles on the web about how to implement windows or certificate authentication but very few that tell you how to actually use the authentication mechanism to restrict access to a WCF service to a particular group of users. One fantastic exception is this blog by Richard Seroter, on which much of this article is based. The major difference is that I also implement Claims based authentication for use with a federated service end point as well as bring the code samples up to date.

So you’ve got a WCF service working with either Windows or Federated security, but now what? How do we authorize particular users/groups? Unfortunately there is no out of the box solution in BizTalk (at least not up to 2010) so you have to create your own custom behavior extension. The good news is that you only need to write this component once and you can then re-use it across all of your BizTalk WCF services.

In Visual Studio create a new Class called BizTalkCustomServiceAuthManager.cs in an existing or newly created generic project (I call mine <CompanyName>.BizTalk.CommonArtifacts.Components which also holds lots of my other common classes). Add a reference to System.configuration, System.IdentiyModel and System.ServiceModel. Create 3 properties, m_group which will be the AD Group or the Claim Value to search for, the m_claimtype which will be the name of the Claim to search for (Claims only) and m_authenticationType which will be an enum allowing you to select either Windows or Claims based authentication in the Receive Port configuration. This could easily be extended to handle another type of authentication mechanism if you wanted.


        public class BizTalkCustomServiceAuthManager : ServiceAuthorizationManager
    {
        private string m_group;
        private string m_claimtype;
        private CustomServiceAuthenticationType m_authenticationType;

        ///
<summary> /// Claims or Windows Authentication
 /// </summary>
        public enum CustomServiceAuthenticationType
        {
            WINDOWS=1,
            CLAIMS=2
        };

        ///
<summary> /// Initializes a new instance of the class.
 /// </summary>
        ///The group.
        ///Type of the authentication.
        public BizTalkCustomServiceAuthManager(string group, CustomServiceAuthenticationType authenticationType, string claimType)
        {
            this.m_group = group;
            this.m_authenticationType = authenticationType;
            this.m_claimtype = claimType;
        }
  }

Now add the following block of code which will form the basis of the core authorization logic for either Windows or Claims authentication depending on what is chosen. Note that I’ve added a lot of Debug lines so you can view what is going on when we start testing the configuration.

        /// <summary>
        /// Checks authorization for the given operation context based on default policy evaluation.
        /// </summary>
        /// <param name="operationContext">The <see cref="T:System.ServiceModel.OperationContext"/> for the current authorization request.</param>
        /// <returns>
        /// true if access is granted; otherwise, false. The default is true.
        /// </returns>
        protected override bool CheckAccessCore(OperationContext operationContext)
        {
            //check that basic access is ok before checking our custom conditions
            if (!base.CheckAccessCore(operationContext))
            {
                return false;
            }

            if (this.m_authenticationType == CustomServiceAuthenticationType.CLAIMS)
            {
                return performClaimsAuthentication(operationContext);
            }
            if (this.m_authenticationType == CustomServiceAuthenticationType.WINDOWS)
            {
                return performWindowsAuthentication(operationContext);
            }
            else
            {
                System.Diagnostics.Trace.WriteLine("No Authentication Type Selected in BizTalk binding ");
                return false;
            }
        }

        /// <summary>
        /// Performs the claims authentication.
        /// </summary>
        /// <param name="operationContext">The operation context.</param>
        /// <returns></returns>
        public bool performWindowsAuthentication(OperationContext operationContext)
        {
            //print out inbound identities recorded by WCF
            System.Diagnostics.Trace.WriteLine("Primary Identity is " +
         operationContext.ServiceSecurityContext.PrimaryIdentity.Name);
            System.Diagnostics.Trace.WriteLine("Windows Identity is " +
  operationContext.ServiceSecurityContext.WindowsIdentity.Name);
            //create Windows principal object from inbound Windows identity
            WindowsPrincipal p = new WindowsPrincipal(operationContext.ServiceSecurityContext.WindowsIdentity);
            //check user in role
            string[] windowsGroups = this.m_group.Split(',');

            foreach (string windowsGroup in windowsGroups)
            {

                bool isAdmin = p.IsInRole(windowsGroup);
                if (isAdmin)
                {
                    System.Diagnostics.Trace.WriteLine("User is in role. Security Accepted " +
                                    windowsGroup);
                    return true;
                }
                else
                {
                    System.Diagnostics.Trace.WriteLine("User is not in role '" +
                                windowsGroup + "'");
                }
            }
            System.Diagnostics.Trace.WriteLine("Security Rejected Request.");
            return false;
        }

        /// <summary>
        /// Performs the claims authentication.
        /// </summary>
        /// <param name="operationContext">The operation context.</param>
        /// <returns></returns>
        public bool performClaimsAuthentication(OperationContext operationContext)
        {

            string[] claimValues = this.m_group.Split(',');

            foreach (ClaimSet claimset in operationContext.ServiceSecurityContext.AuthorizationContext.ClaimSets)
            {
                foreach (Claim claim in claimset.FindClaims(this.m_claimtype, Rights.PossessProperty))
                {

                    //Add to headers
                    System.Diagnostics.Trace.WriteLine("Current Claim is " +
                                    claim.ClaimType);

                    System.Diagnostics.Trace.WriteLine("Current Claim Resource is " +
                                    claim.Resource.ToString());

                    foreach (string claimValue in claimValues)
                    {
                        if (claim.Resource.ToString() == claimValue)
                        {
                            System.Diagnostics.Trace.WriteLine(String.Format("User is in claim {0} Security Accepted ",claimValue));
                            return true;
                        }
                    }
                }
            }
            System.Diagnostics.Trace.WriteLine("User is not claim. Security Denied " +
                            this.m_group);

            return false;
        }

Now add a new class called BizTalkCustomServiceBehavior.cs and extend IServiceBehavior. This doesn’t really do very much but is necessary.

public class BizTalkCustomServiceBehavior : IServiceBehavior
    {
        private string m_group;
        private .BizTalk.CommonArtifacts.Components.BizTalkCustomServiceAuthManager.CustomServiceAuthenticationType m_authenticationType;
        private string m_claimtype;
        ///
<summary> /// Initializes a new instance of the class.
 /// </summary>
        ///The group.
        ///Type of the authentication.
        public BizTalkCustomServiceBehavior(string group, .BizTalk.CommonArtifacts.Components.BizTalkCustomServiceAuthManager.CustomServiceAuthenticationType authenticationType, string claimType)
        {
            this.m_group = group;
            this.m_authenticationType = authenticationType;
            this.m_claimtype = claimType;
        }

        ///
<summary> /// Provides the ability to change run-time property values or insert custom extension objects such as error handlers, message or parameter interceptors, security extensions, and other custom extension objects.
 /// </summary>
        ///The service description.
        ///The host that is currently being built.
        public void ApplyDispatchBehavior(ServiceDescription serviceDescription, ServiceHostBase serviceHostBase)
        {
            ServiceAuthorizationBehavior authBehavior =
                                                        serviceDescription.Behaviors.Find();
            //pass in Windows Group set during config setup
            authBehavior.ServiceAuthorizationManager = new BizTalkCustomServiceAuthManager(this.m_group, this.m_authenticationType, this.m_claimtype);
            ((IServiceBehavior)authBehavior).ApplyDispatchBehavior(serviceDescription, serviceHostBase);
        }

        /// <summary> /// Provides the ability to pass custom data to binding elements to support the contract implementation.
        ///The service description of the service.
        ///The host of the service.
        ///The service endpoints.
        ///Custom objects to which binding elements have access.
        public void AddBindingParameters(
	            ServiceDescription serviceDescription,
	            ServiceHostBase serviceHostBase,
	            Collection endpoints,
	            BindingParameterCollection bindingParameters) {

        }

        public void Validate(
                 ServiceDescription serviceDescription,
                ServiceHostBase serviceHostBase
                )
        {

        }

    }

And finally add the Behaviour Element where we define the custom properties that can be set in the receive port properties.

public class BizTalkCustomBehaviorElement : BehaviorExtensionElement

    {
        ///
        /// Want custom config property to show up in the BizTalk receive location
        ///
        [ConfigurationProperty("group", IsRequired = false, DefaultValue = "")]
        public string Group
        {
            get { return (string)base["group"]; }
            set { base["group"] = value; }
        }

        ///
<summary> /// Gets or sets the type of the auth.
 /// </summary>
        ///
        /// The type of the auth.
        ///
        [ConfigurationProperty("authenticationtype", IsRequired = false, DefaultValue = CompanyName.BizTalk.CommonArtifacts.Components.BizTalkCustomServiceAuthManager.CustomServiceAuthenticationType.WINDOWS)]
        public CompanyName.BizTalk.CommonArtifacts.Components.BizTalkCustomServiceAuthManager.CustomServiceAuthenticationType AuthType
        {
            get { return (CompanyName.BizTalk.CommonArtifacts.Components.BizTalkCustomServiceAuthManager.CustomServiceAuthenticationType)base["authenticationtype"]; }
            set { base["authenticationtype"] = value; }
        }

        ///
<summary> /// Gets or sets the type of the claim.
 /// </summary>
        ///
        /// The type of the claim.
        ///
        [ConfigurationProperty("claimtype", IsRequired = false, DefaultValue = "")]
        public string ClaimType
        {
            get { return (string)base["claimtype"]; }
            set { base["claimtype"] = value; }
        }

        ///
<summary> /// Creates a behavior extension based on the current configuration settings.
 /// </summary>
        ///
        /// The behavior extension.
        ///
        protected override object CreateBehavior()
        {
            return new BizTalkCustomServiceBehavior(Group, AuthType,ClaimType);
        }
        ///
<summary> /// Gets the type of behavior.
 /// </summary>
        /// A .
        public override Type BehaviorType
        {
            get
            {
                return typeof(BizTalkCustomServiceBehavior);
            }
        }

    }

Now build your project and add it into the GAC using a command like this "C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\bin\NETFX 4.0 Tools\x64\gacutil.exe" /if "D:\Projects\Biztalk\CompanyName.CommonArtifacts\CompanyName.CommonArtifacts.Components\bin\Debug\CompanyName.BizTalk.CommonArtifacts.Components.dll"

Next you need to open your machine.config – I know it sounds a bit scary but it’s the only way. Mine is located in C:\Windows\Microsoft.NET\Framework\v4.0.30319\Config but it depends on what version of .NET you are using and also whether you are running x86 or 64 bit applications.

Search for the behavior extensions tag <behaviorExtensions> and at the bottom add a link to your new class that has been added to the GAC. Note that if you aren’t sure what this should look like you can view the .dll meta info using something like gacutil.exe -l CompanyName.BizTalk.CommonArtifacts.Components

<add name="CustomAuthBehavior" type="CompanyName.BizTalk.CommonArtifacts.Components.BizTalkCustomBehaviorElement, CompanyName.BizTalk.CommonArtifacts.Components, Version=1.0.0.0, Culture=neutral, PublicKeyToken=6dbf238400666825" />
</behaviorExtensions>

Almost done. Now restart your BizTalk Host Instances and close and re-open the BizTalk Administration Console. Open the properties for any Windows or Federated binding WCF-Custom receive port (note that it does have to be WCF-Custom) and click Configure. In the Behavior tab, right click ServiceBehavior and select “Add extension…”. You should see your new Service Behavior class appearing there – if it isn’t check you edited the correct machine.config (note you will need to edit the machione.config within the relevant Framework64 directory as well if you are running 64bit OS) and that it is definitely referencing the right assembly.

To configure access to a windows AD group, select Windows authenticationType and specify the AD group (including the domain). Note that you can add multiple groups by separating with commas.

To use the Claims authentication you also need to specify which Claim to search on. The Group will relate to the value of that claim, e.g. in this example the claim type is a member of a SharePoint 2010 site http://schemas.company.com/identity/claims/2010/07/teammember and the value is the name of the site itself, e.g. “Team A”. Again you can specify multiple values by separating the group parameter with commas.

Now restart IIS as well and put some test cases through. If the user fails you will receive a Service Authorization fault. The Debug output might look something like this.

So perhaps not the most straightforward of solutions and I really think Microsoft should have bundled this functionality out the box. But the pain is only had once and at least you get total flexibility for how the authorization is performed.

Lost Messages in WCF-SQL Polling Receive Location

I’ve been having a problem with messages not appearing in a WCF-SQL receive location of late. The receive location polls for new data every 30 seconds using the polledDataAvailableStatement and then when it finds some rows to process, the pollingStatement extracts those rows and then returns them to BizTalk using a very simple Stored Procedure. This mostly works fine but mysteriously around 25% of the time the rows were set to processed but no message appeared in the BizTalk message box and the items were lost forever. There were no suspended or failed messages. No messages in the Event Log. Absolutely nothing. Very mysterious.

The solution is I think that whenever you have a polling statement that does anything other than a select statement which mine is as it updates the affected rows to PROCESSED = true, you have to set useAmbientTransaction to true. We had originally disabled this because we thought it might have been related to table locking behaviour we had experienced in our development environments.

This msdn article seems to support the solution.

Not performing operations in a transactional context is advisable only for operations that do not make changes to the database. For operations that update data in the database, we recommend setting the binding property to true; otherwise you might either experience message loss or duplicate messages, depending on whether you are performing inbound or outbound operations.

 

 

Search with criteria for value in different XML node in an XSLT Template

There have been a few instances where I have had to filter the incoming XML to look for a value from a different node in the XML. For the example in this article I had the parent node id but I wanted to know the name of that parent node.

This excellent article gave me a starting point for the principle of searching for values in different nodes using an XSLT template. This worked spendedly in one map I was working where I was trying to sum up all values in the current list of node items.


<xsl:template name="OutputSum">
  <xsl:param name="param1" />
  <xsl:element name="TransactionAmount">
    <xsl:variable name="var:vTranAmt"
    select="sum(//ExpenseItem[ReportEntryID=$param1]/TransactionAdjustmentAmount" />

    <xsl:value-of select="$var:vTranAmt" />

  </xsl:element>
</xsl:template>

However, when I tried to use the same principle to select the name value of a parent node for which I only knew the ID, it just wouldn’t work. I kept gettiing blank values no matter what I tried.

The source schema was from the MS Dynamics CRM v4 web services using the RetrieveMultiple method and is much more complex than one I normally use. After a lot of headscratching I noticed that it was the namespaces that were causing the problems. When I right clicked the map, selected validate and viewed the .xsl that Visual Studio was generating I noticed in my other mappings that BizTalk was adding some bizarre namespaces to the nodes, e.g. s6:RetrieveMultipleResult/s4:BusinessEntities . Eventually I got it to work using these namespaces.


<xsl:template name="ParentCompanyCode">
  <xsl:param name="param1" />
  <xsl:element name="Code">

                <xsl:value-of select="//s4:BusinessEntity[s6:new_geographyid=$param1]/s6:new_name" />

  </xsl:element>
</xsl:template>

Unfortunately I can’t explain exactly why this was required, but my advise is that if you can’t work out why custom xsl isn’t working then create something similar using functoids and have a look through the xsl that it generates via the Validate map functionality.

Flat File Assembler Encoding and Charset

I’ve had lots of problems recently with file encodings including foreign and special characters not appearing properly in target flat files. For example ø being written as ? and even worse an accented é translated as 2 characters é which meant that the column aligned file got messed up.

The reason is that the source system provides the data in UTF-8 (which is the best encoding because it should in theory deal with all character sets) but the files were being created as ANSI which is far more limited and unless you specify the charset will cause the issues I have described. However the target system I am creating files for is a little archaic and does actually require files with an ANSI charset.

BizTalk can handle this issue quite well out the box but you have to specify the charset in the flatfile component otherwise it will take a guess which will in all likelyhood be wrong. I have also found that you have to specify this at design time in visual studio otherwise the change will not be effective. So although you have the option of specifying the charset in the send port properties, this does not work properly – must be a bug in BizTalk.

In visual studio open up your pipelene and select the Flat file assembler component.

Then select the file encoding which matches what your target system expects. This might be UTF-8 but may also be ANSI in which case you should select the charset that matches e.g. Western-European (1252).

You should use a similar process when reading files using the flat file disassembler component. And also be careful when using external tools such as an otherwise excellent PGP pipeline component. I had an issue where the pipeline component was writing a file to a temporary location but did’t take the encoding into account and the encoding was lost. Here is the code I used to change the EncryptStream method int he component source code.

        /// <summary>
        /// Encrypts the stream.
        /// </summary>
        /// <param name="inStream">The in stream.</param>
        /// <param name="pubKeyFile">The pub key file.</param>
        /// <param name="sourceFilename">The source filename.</param>
        /// <param name="extension">The extension.</param>
        /// <param name="armorFlag">if set to <c>true</c> [armor flag].</param>
        /// <param name="integrityCheckFlag">if set to <c>true</c> [integrity check flag].</param>
        /// <param name="strCharset">The STR charset.</param>
        /// <param name="targetCharset">The target charset.</param>
        /// <returns></returns>
        public static Stream EncryptStream(Stream inStream, string pubKeyFile, string sourceFilename, string extension, bool armorFlag, bool integrityCheckFlag, String strCharset, String targetCharset)
        {
            string tmpEncryptedFile = sourceFilename + "." + extension;

            // View debug lines with Debugview.exe
            //System.Diagnostics.Debug.WriteLine(strCharset + " is the charset");

            try
            {

                using (StreamWriter sourceStream = new StreamWriter(sourceFilename, true, Encoding.GetEncoding(targetCharset)))
                {
                    using (StreamReader sr = new StreamReader(inStream, Encoding.GetEncoding(strCharset), true))
                    {
                        sourceStream.Write(sr.ReadToEnd());
                    }
                }

                using (FileStream encryptedStream = File.Create(tmpEncryptedFile))
                {
                    PgpPublicKey publicKey = ReadPublicKey(File.OpenRead(pubKeyFile));
                    EncryptFile(encryptedStream, sourceFilename, publicKey, armorFlag, integrityCheckFlag);

                    encryptedStream.Seek(0, SeekOrigin.Begin);
                    int myCapacity = Int32.Parse(encryptedStream.Length.ToString());
                    Byte[] byteArray = new Byte[myCapacity];

                    MemoryStream memStream = new MemoryStream();

                    encryptedStream.Read(byteArray, 0, myCapacity);
                    memStream.Write(byteArray, 0, myCapacity);

                    encryptedStream.Close();

                    memStream.Seek(0, SeekOrigin.Begin);
                    return memStream;
                }
            }
            catch (Exception ex)
            {
                //System.Diagnostics.EventLog.WriteEntry("PGPWrapper[Encrypt] Error", ex.ToString());
                throw ex;
            }
            finally
            {
                // Clean-up temp files
                if (File.Exists(sourceFilename)) { File.Delete(sourceFilename); }
                if (File.Exists(tmpEncryptedFile)) { File.Delete(tmpEncryptedFile); }

                //FileInfo fileInfo = new FileInfo(sourceFilename);
                //System.Diagnostics.EventLog.WriteEntry("PGP_Debug", fileInfo.FullName + " was deleted");

                //fileInfo = new FileInfo(tmpEncryptedFile);
                //System.Diagnostics.EventLog.WriteEntry("PGP_Debug", fileInfo.FullName + " was deleted");
            }
        }