Category Archives: BizTalk

NUnit Test Cases picking up cached version of an assembly

We all know that Test Driven Development makes development so much quicker and safer. However, I’ve had a problem for a while whereby my NUnit test cases which reference libraries built for BizTalk projects don’t update when I make a change to the code and re-run the test case. The only way around it was to restart Visual Studio – a major pain.

Today I did a bit more digging and found out what was wrong. BizTalk needs signed assemblies deployed to the GAC to be picked up at runtime so I’ve always automatically done this and also added a post build event to add my assemblies to the GAC on a successful build. It was this that was causing my NUnit test case to pick up the signed version instead.

The solution is to untick the “sign the assembly” box in the properties of the library and also REM the post build events whilst you are in the unit testing phase. This will mean the NUnit library will pick up the version of the .dll within your Visual Studio environment.

Filtering Effective Dated Messages in BizTalk

I do a lot of work integrating systems with PeopleSoft using BizTalk. One of the core concepts in PeopleSoft is effective dating, that is the ability to have a history of an entity within the database controlled by an effective date field. For example PeopleSoft may hold an address for a company or person but we know the address will change on a particular date in the future. So you will have one entry effective dated in the past and another effective dated in the future. The advantage being that on the day the person or company moves addresses, the system will automatically pick up the correct address. It also helps keep an audit of historical addresses.

When consuming component interfaces exposed as web services using Integration Broker, all historical and future dated rows are returned. So how can you program a BizTalk map to only return the current row? The answer is to apply filtering via an inline C#

In this example I am using the CUSTOMER_MAIN CI from the Accounts Receivable module in PeopleSoft Financials to filter for current addresses. The incoming message has 2 different addresses (ADDRESS_SEQ_NUM), but one of them has 3 CUST_ADDRESS entries with different effective dates. The end result I want is 2 addresses, the current address for each ADDRESS_SEQ_NUM.

Create an inline c# functoid with the following code. Note that this was based on this tutorial here.

public System.Collections.Generic.List<int> duplicateList = new System.Collections.Generic.List<int>();
public bool IsDuplicate( int addressSeqNo, string effectiveDate, string effectiveStatus )
{

	 DateTime dt = Convert.ToDateTime(effectiveDate);
	 // Not interested in future dated dates
	 if (dt.Date > DateTime.Now.Date)
	   return true;

	 // Not interested in inactive addresses either
	 if (effectiveStatus != "A")
	   return true;

     if( duplicateList.Contains( addressSeqNo ) )
        return true;
     duplicateList.Add( addressSeqNo );
     return false;
}

Add EFFDT, ADDRESSSEQ_NUM and EFF_STATUS to it and connect it to a logical equals functoid, with false hard coded as the second parameter. This is then sent to the destincation node, as per the screenshot below.

FilteringAnEffectiveDatedMessage

Essentially this is filtering out any future effective dated rows and anything that is not active. Then is is pciking the first valid row, ignoring any others that follow. It relies on the order of the dates being sequential, but this is enforced via the PeopleSoft front end.

Hyperion Planning Import Aborts – Required Column ‘Member Name’ not found

I have been involved with writing interfaces into Oracle Hyperion Planning of the past year, although mostly limited to making the data available for our specialist contractors to import into the model. We encountered an error today which had us stumped for hours as there was little or no logging that indicated where the problem was.

We were trying to import a dimension using EPMA which we had done many times before. This time however, the import would “Abort” but with no indication why. This is what the output from the EPMA job  looked like:

2014-01-16 11:57:02,068 INFO Import job has been successfully submitted with jo
b id number "18,441".
2014-01-16 11:57:02,068 INFO JobID 18,441 submitted. To view detailed job infor
mation go to https://myapp.mycompany.com/workspace/?module=awb.appmanager&bp
m.docid=bpmarj&bpm.logoff=false&jobId=18441.
2014-01-16 11:57:07,115 INFO The import job status is "Aborted".
2014-01-16 11:57:07,115 INFO Import failed.

We searched everywhere and eventually found this output in the file D:\Oracle\Middleware\user_projects\epmsystem1\diagnostics\logs\epma

[2014-01-16T11:20:00.898+00:00] [EPMADIM] [INTERNAL_ERROR:32] [EPMADIM-1] [EPMADIM.Hyperion.CommonServices.Exceptions.BaseException] [tid: 15] [ecid: disabled,0] SVR_ERR_IMPORT_COLUMN_NOT_FOUND:Required Column 'Member Name' not found at Hyperion.DimensionServer.ImportHierarchyItem.ValidateColumns()
at Hyperion.DimensionServer.ImportEngine.CleanOutBadImportItems(ImportDimension dim)
at Hyperion.DimensionServer.ImportProfile.IterateDimensions(Action`1 action)
at Hyperion.DimensionServer.ImportEngine.ParseAndExecuteImport()
at Hyperion.DimensionServer.ImportEngine.StartImport(Boolean updateFinalJobStatus)

Still didn’t help us much. Eventually a colleague noticed that in the source data table which feeds the hierarchy there was a row which had the Child field set to blank. i.e. ” . This is what was causing the issue. So simple yet so much time to find. Anyway, hope it might help someone else in the same position.

BizTalk Flat File Assembler with Untagged Header and Trailer/Footer

I needed to build an interface which converted an inbound message into a tab delimited file containing 1 header line, any number of detail lines and finally 1 footer line – essentially 3 different schemas/formats in one file E.g.

1->0000002->20130101->M
00001->A bike->300.00->Sent->true->true
00002->A ball->30.00->Placed->true->false
00003->A skipping rope->5.00->Sent->true->true
1->3->335.00

I originally tried combining the 3 different schemas into one and had some success. However the footer row was causing issues and I subsequently found out that BizTalk didn’t know when the repeating record finished and the footer row started so it was processing all the rows as the repeating row and failing with something like “Native Parsing Error: Unexpected data found while looking for:’\t'”. One solution was to add a Tag identifier to the footer schema e.g. FOOTER->1->3->335.00 which worked fine except that this was someone else’s schema (a 3rd party) so I couldn’t start adding FOOTER-> to the end of each file otherwise it wouldn’t get processed.

The clue to the solution was provided in this post by colmac73 which involves the execution of pipeline components in an orchestration. This post adds a few more details as to how I achieved it.

  1. First of all create a separate schema for each of the 3 formats, e.g. SchemaHeader, SchemaBody, SchemaTrailer. This is described in detail in a Microsoft article which explains how to do a similar thing but for receiving rather than sending messages.
  2. Create 3 separate maps to transform the source data to each of the SchemaHeader, SchemaBody, SchemaTrailer you have just created
  3. Create a pipeline object called ConvertToFlatfile containing nothing more than a flat file assembler. As colmac73 says in his article, leave the header / trailer  and document schemas to (none) as these will automatically be detected at runtime.
  4. Create an orchestration which receives your inbound message however you prefer. I like to use filter based receive objects that look for schema types, promoted properties or in this case BTS.ReceivedPortName = “MyFileReceivePort”
  5. Create 3 messages in the orchestration for each of the SchemaHeader, SchemaBody, SchemaTrailer
  6. Create a message called CombinedSendPipelineMessage of type System.Xml.XmlDocument. This will be the message you send to the outbound port.
  7. Create a variable called SendPipelineInput of type Microsoft.XLANGs.Pipeline.SendPipelineInputMessages. Note that you will need to reference Microsoft.XLANGs.Pipeline.dll which is located in the BizTalk root installation directory, e.g. D:\Program Files (x86)\Microsoft BizTalk Server 2010
  8. Execute all 3 of your transforms either in sequence or in parrallel
  9. Create a message assignment shape and enter something like this
    CombinedSendPipelineMessage = new System.Xml.XmlDocument();
    
    SendPipelineInput = new Microsoft.XLANGs.Pipeline.SendPipelineInputMessages();
    SendPipelineInput.Add(msgHeader);
    SendPipelineInput.Add(msgBody);
    SendPipelineInput.Add(msgTrailer);
    
    Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteSendPipeline(typeof(MyCompany.Namespace.Pipelines.ConvertToFlatfile),SendPipelineInput, CombinedSendPipelineMessage);
    
  10. Send the message to a send port. I usually like to use direct subscription / filter based bindings but for this I configured the port directly to the orchestration using “Specify later”
  11. Configure the send port as file / SFTP depending on your requirements and select PassThroughTransmit as the pipeline (as the pipeline has already been executed from within the orchestration).
  12. Now if everything is working correctly, you should be able to output any inbound message into the required header / body / trailer format at your send port.

Your Orchestration might look a bit like this.

BizTalk Flat File Assembler Orchestration with Header and Footer

Active Service Instance Never Finishing in BizTalk 2010

A colleague of mine was trying to get ordered delivery running on a send port but kept seeing a rogue Active Service Instance that never seemed to finish, almost as if it was hanging or was waiting for something else to happen. The process overall seemed to complete without error and there was only ever one of these service instances no matter how many messages were put through BizTalk. Of course you could manually terminate the instance but we were worried that it might create issues such as support people trying to investigate the issue. The instance itself was blank in that it had no messages, errors or properties other than an instance Id, timestamp and a send port name.

We did lots of investigation including detailed orchestration debugging and traffic monitoring in Wireshark but all seemed normal. It had us stumped for hours.

The answer to the issue is that it isn’t an issue at all and is the way Microsoft has designed a port to work with ordered messaging as described in this article.

Ordered Delivery Send Port instance works as a singleton service. Since start it stays in Running state. It will not recycle if we restart its Host Instance. We could manually terminate it, if we want.