Skip to content
+1-888-319-3663

EONE BLOG

Tech Tuesday: Importing Large Documents Using SmartConnect


When running large transactions from an eConnect application, SmartConnect would include any application referencing eConnect including the Direct Document Sender sample application included with eConnect. It is possible to get a number of different errors all related to various possible timeouts that eConnect uses.

The various errors that SmartConnect can give you are:

There was an error writing to the pipe: The pipe is being closed. (232, 0xe8).

There was an error reading from the pipe: The pipe has been ended. (109, 0x6d)

The transaction associated with the current connection has completed but has not been disposed. The transaction must be disposed before the connection can be used to execute SQL statements.

To understand how this could occur, it is helpful to know about a bit about how the architecture of eConnect works.

I “borrowed” the following screenshot of the eConnect architecture from MSDN and added 3 items into the list to show where SmartConnect fits in as well as the technology between the components.

Figure 1: eConnect architecture from https://msdn.microsoft.com/en-us/library/aa973837.aspx

Figure 1: eConnect architecture from https://msdn.microsoft.com/en-us/library/aa973837.aspx

At the top at 1, you have an eConnect application such as SmartConnect.

As we know, SmartConnect can take pretty much any source data and then transform it into an eConnect formatted XML document.

But where does it go from there?

SmartConnect then calls the local eConnect assembly Microsoft.Dynamics.GP.eConnect.dll and passes it the XML document created.

At this point, this is where it gets interesting.

In #2 above, we have the connection between the “.NET assemblies” and the “eConnect Integration Service”.

The eConnect Integration Service is a Microsoft Windows Service that runs on the local machine and listens for a WCF connection to be made to it.

When the call to the local eConnect assembly is made, we then go to step #2 above where the local eConnect assembly opens a WCF connection to the eConnect Integration Service and then passes it the XML document to process.

Because we want a document to be processed as “all or nothing”, the eConnect Integration Service starts a SQL transaction (#3 in the chart) and then proceeds to process each node in the XML as a call to an eConnect stored procedure.

Once the entire XML document is processed, the eConnect Integration Service either commits the transaction or rolls it back along with any error messages.

At that point, handling is returned back to the caller – the local eConnect assemblies– which in turn returns the results from the eConnect Integration Service back to the calling application.

Now you might be asking “OK, but how does that solve my problem?”

Well, it doesn’t directly but the specific error you are getting tells us what needs to be adjusted.

The two “pipe” error messages initially are WCF errors. Because we know that eConnect is making the WCF connection to the eConnect Integration Service – we know that we could be running across a WCF timeout of some type and so we need to adjust the WCF settings.
For a WCF connection, we need to make sure both the “sender” (SmartConnect) and the “receiver” (the eConnect Integration Service) both need to be on the same page for their WCF settings.

For the SQL transaction error, that actually is caused by a SQL Timeout happening although the error doesn’t specifically mention that happened. But it is.

You can download a zip file containing my Microsoft.Dynamics.GP.eConnect.Service.exe.config and also a new eOne.SmartConnect.UI.External.exe.config file that contains the client SmartConnect client eConnect settings HERE.

In both of the files, configuration lines have been added and/or modified and are commented to explain the changes.

You likely will not have an eOne.SmartConnect.UI.External.exe.config file so you can just copy that into your C:\Program Files (x86)\eOne Solutions\SmartConnect folder.

You WILL have a Microsoft.Dynamics.GP.eConnect.Service.exe.config file in your C:\Program Files\Microsoft Dynamics\eConnect XX.0\Service folder.

There isn’t anything unique in this configuration file (unless you manually changed/added it), so you can rename your existing Microsoft.Dynamics.GP.eConnect.Service.exe.config file to Microsoft.Dynamics.GP.eConnect.Service.exe.config.original. Then copy in the new Microsoft.Dynamics.GP.eConnect.Service.exe.config file.

Once you do this, make sure to restart the eConnect Integration Service on the machine.

Lastly, one other thing you will also need to change isn’t in those configuration files.

In the configuration files, we are changing the SQL Timeout settings. However there is a “maximum timeout” value that is set in your machine.config file that supposedly overrides our maximum value. On my own machine, I don’t seem to need it. Don’t know if that is because I’m in a VM or something but I’ve seen other customers needing to add it.

Per this blog article and others, we also need to add some lines into your machine.config.

We would only need to update the .NET 4.0 versions of this and both the 32bit & 64bit configurations just in case.

C:\Windows\Microsoft.NET\Framework\v4.0.30319\CONFIG\machine.config
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\CONFIG\machine.config

At the bottom of the machine.config, we will need to add the new lines for our settings. You won’t have these unless you modified the files already.

Below, I’ve added the new <system.transactions> section near the bottom of both machine.config files with the new maxTimeout value of 2 hours.

      </roleManager>
</system.web>
<system.transactions>
<machineSettings maxTimeout="02:00:00" />
</system.transactions>
</configuration>


This will match the eConnect & SmartConnect timeout changes that I’ve added as I’ve set them to also be 2 hours (using values of 120 minutes and 7200 seconds as appropriate).

Have any questions? Feel free to leave a comment below or email into sales@eonesolutions.com!

6 Comments

  1. Steven Priest on September 8, 2016 at 9:50 am

    Hi Patrick,

    I have a client with a large number of G/L transaction including AA transactions. They were receiving a time out and I had them apply the changes to the eConnect and the machine config files, then reboot the machine. They are now receiving the following message:

    There was an error reading from the pipe: The pipe has been ended. (109, 0x6d).
    Journal Entry with AA (xlsx format) (JOURNAL_ENTRY) processed with errors. Not all journal entry records uploaded correctly, as noted below.

    Failed Records:
    Company: BABET JE: Test Big SC Entry User: 427090
    Company: BABET JE: Test Big SC Entry User: 427090

    Would this error be related to the time out/pipe original error or is there something else we need to check?

    Thank you

  2. Patrick Roth on September 8, 2016 at 9:58 am

    As noted in the blog article “pipe” errors are WCF timeouts. So you’d want to focus on the eConnect config file changes typically.
    But there are also some sc exe changes (the other file) that you can create – just copy in mine.

  3. james pascoe on September 27, 2016 at 7:48 pm

    We have run into this issue with large imports using SmartConnect.

    I have attempted to modify my machine.config but when I do this it causes GP to crash and i assume other applications might not work either.

    I have obviously done something wrong in adding the system.transaction section.

    Below is the end of my machine.config file. Does this look correct?

  4. Dave Washabaugh on October 21, 2018 at 8:09 pm

    Patrick,
    When I add the code to the machine.config files GP crashes for me too.

  5. Patrick Roth on October 23, 2018 at 9:15 am

    James & Dave,

    You modified the config file incorrectly in some way. This would cause any application that has .NET 4.x components to fail due to a bad config file.

    I modified this article to show a screenshot of the changes in context of my existing machine.config.

  6. Dan on November 19, 2018 at 2:59 pm

    Hello,

    I have tried copied the config files shared in your link and have updated the two machine.config files for .Net 4.0, but all that did was change the error I was seeing from “There was an error reading from the pipe: The pipe has been ended. (109, 0x6d)” to “The transaction associated with the current connection has completed but has not been disposed. The transaction must be disposed before the connection can be used to execute SQL statements.” Is there something I am missing?

Leave a Comment





RECENT POSTS


eOne at Summit North America 2021
Employee Spotlight: Megan Tripp, Business Development Representative
eOne is now compatible with Microsoft Dynamics GP October 2021 Release
October 2021 Releases of Extender, SmartList Builder, SmartView, SmartPost, and Flexicoder are Now Available
Advanced SmartConnect REST Web Services Integration - October 2021

POPULAR POSTS


2016 SmartConnect Integration Bootcamps
Tech Tues: Automatically refreshing Pivot Tables in Excel Refreshable Reports
Happy Thanksgiving from the eOne team!
2017 SmartConnect Integration Bootcamps
New Releases of Extender and SmartList Builder

CATEGORIES

TAGS

Business Central CRM D365 Business Central Dynamics 365 dynamics crm Dynamics GP Dynamics NAV Econnect Employee Spotlight eone eOne News error Error Message Events Excel Excel Report Builder Extender Flexicoder GP integration Map Meet the Team Microsoft dynamics crm Microsoft Dynamics GP Navigation List Builder Office Relationships partners Popdock release Reporting SalesForce SalesForce.com SmartConnect SmartConnect.com SmartConnect Bootcamp SmartConnect Maps SmartConnect Office Hours SmartList SmartList Builder SmartPost SmartView SQL Tech Tuesday Templates training

Integrate & Automate without Any Code.

SmartList Data has Never Been Faster.

The Easiest Way to Report on GP Data.