Skip to content
+1-888-319-3663

EONE BLOG

Tech Tuesday: SmartConnect Processing Sequence


Our Director of Professional Services, Lorren Zemke, will be taking you through the SmartConnect processing sequence for this week’s Tech Tuesday!

In this article I am continuing my series of discussions on how SmartConnect processes data. Previously I’ve used examples on summing values and working with line sequence numbers. Today I will diagram the process flow of SmartConnect, using the process of sending data to Microsoft Dynamics GP but the same process is used for all maps.
 
The basic steps fall into the following categories:
  1. Pre-Map Tasks
  2. Data Read
  3. Transformation
  4. Pre-Document Tasks
  5. Document Send
  6. Post Document Tasks
  7. Post Map Tasks

Pre-Map Tasks 

Pre-Map tasks are executed before the data is read from the data source. This can be document processing tasks such as moving files, running scripts to ask the user for values, running another map and reading data from a SQL table. 

 

Data Read 

Pre-map tasks have finished executing and the data is being read from the data source. 

One specific task within SmartConnect, after the data is read, is called SQL Validation. This task is used to query SQL tables based on data from the data source. The Validation logic can verify if data exists or does not exist allowing the map to stop processing if the logic is not met. 


Transformation
This section of data is processed before it is sent to the destination. All of the column calculations are done at this time and the data is prepped for sending to the destination. In the case of Dynamics GP, SmartConnect is creating the XML file that will eventually be sent to eConnect. In a call to a web service destination, SmartConnect is generating the object needed to make the call and send data to that web service.

Pre-Document Tasks

The tasks created in this section happen immediately before each document is sent to its destination. At this point all data has been retrieved and all calculations with the map have taken place.
Use the tasks in this section for setting global variable data that will need to be used for Post Document or Post Map tasks.

If you are using Global Variables in your map, you can clear them out at this time to get them ready for the next document unless you need them for a Post Document or Post Map Task.

Document Send
SmartConnect creates a connection to the destination and sends a single document.

Post Document Tasks
Once a document has been sent to the destination, based on whether the document was successful will determine which set of Post Document Tasks will execute. Tasks can run if the document was successfully integrated or not. If a task needs to run regardless of whether the document was successful or not, the task needs to be placed.


The Pre-Document Tasks, Document Send and Post Document Tasks will continue to loop based on the value set for the Map Keys on the Data Source section of the map.



Post Map Tasks
After all data has been sent to the destination, SmartConnect determines if the map was successful or not to determine which set of tasks to execute. For all destinations but Dynamics GP it will depend if even one document or task is listed as having failed, the tasks that are part of the failed document section will execute. Only if 100% of the documents are successful will the map success tasks execute.

Happy Integrating!

Interested in learning more about SmartConnect? Reach out to our Sales Team at sales@eonesolutions.com for more information!
 

8 Comments

  1. curtis beethe on March 8, 2018 at 4:47 pm

    It appears SmartConnect fires the post map logic prior to creating it’s log entries for the run. Kinda hard to query the TaskHeader and TaskDetail tables from a script. Am I doing it wrong?

  2. Patrick Roth on March 12, 2018 at 9:08 am

    “Am I doing it wrong?”

    I doubt it – how could you?
    I would suspect it would work this way because a POST map task could also fail and therefore should be logged as well. And if SC has already written the entries, it would then need to write more.

    But yes sometimes it would be nice if SC would write the detail entries as it goes vs all at once when finished.

  3. Andrew Bynum on December 12, 2018 at 11:45 am

    So does this mean before SmartConnect sends the document to the GP server, it has to ingest the data to complete the transformation and create the document? Or does SmartConnect essentially perform a read query on the source server…with some kind of agent on that server…?…create the document, then send to the GP server?

    I’m trying to figure out if SmartConnect is reading a large dataset and has to make any transformation, is that data set moved to the SmartConnect server, then effectively pushed to the target server.

    Thanks!

  4. Patrick Roth on December 13, 2018 at 9:45 am

    Andrew,

    SmartConnect processes the source data from wherever it executes from.
    So typically the “local machine”

    It reads the data from wherever it comes from – CRM Query, SQL, Excel, Text Files.

    From there, it transforms the data however you need it to via calculations, etc and turns it into a “document”.

    And then it calls the appropriate destination method. So that might be a SQL table directly, it might be CRM on prem or the cloud, might be BC or Salesforce, or might be GP.

  5. Andrew Bynum on December 13, 2018 at 1:50 pm

    Thank you for the explanation…can you clarify if this document is created on the SmartConnect server?

    We are looking at hosting our SmartConnect server offsite. We will be charged for outgoing data transfer.

    I am trying to calculate, given the number of maps and their source & destinations, whether running the maps will add to our outgoing data usage.

    So…in my example…SmartConnect server A is in AWS…Servers B & C are on the home network. If I am moving data from B to C, is the “document” generated by SmartConnect generated on server A in AWS, then moved to server C as the load…?

    Or does SmartConnect somehow direct server C to pull from server B using instructions from the “document”…?

    Thank you in advance for your help!

  6. Chad Bruels on May 2, 2019 at 9:59 pm

    To me, this means there is not a true pre-document task. I need to write a script that gets a next transaction number from GP (and I cannot use the built-in GP rolling column function because the source dynamically indicates the target company) and placing that script on the pre-document tasks means my first document does not get a next number.

  7. Patrick Roth on May 6, 2019 at 4:50 pm

    Think of “before document” to be “Before Document Submit” which means “I have my document, here is the final “send it/stop it” option.

    I haven’t had an issue with using a GP Rolling Column not getting the “next number” from an integration regardless if I used the “Define” capability or else hard coding the company destination.

    So You might want to double check that this isn’t working for you.

    But if that doesn’t work for you or if we just want to do it ourselves then we could.

    You are correct there is no task for “I’m starting a new document now”.

    Easiest solution is to use the “Restriction” script of whatever your first transaction mapped node is.
    In that restriction script, make a .NET connection to your database and run your proc to get the next number.

    in that script, you would have something like:
    if GBL_NEXTDOCNO = “” then

    GBL_NEXTDOCNO =
    end if
    return true

    if the global is empty, then we execute the .net sql code to pull that next number. If not, we leave it set as-is.

    Now make a calculation that just returns the global

    return GBL_NEXTDOCNO

    and map that everywhere you need a document number for the document.

    Now the issue is, we need to clear it so that on the NEXT document, we find the next number again.

    And we can either use:
    BeforeDocument task (because it is ready to do)

    or instead
    AfterDocumentSuccess & AfterDocumentFailure

    clear the global in both

    GBL_NEXTDOCNO = ""
    return true

    So as we go, we find the next global and then use it in the map.
    and then before/after we've finished the document, clear it out so that we re-get it during the next cycle.

  8. Patrick Roth on May 6, 2019 at 5:01 pm

    Hmmm. Not sure why that posted in Red like that. But I do like that it looks authoritative.

Leave a Comment





RECENT POSTS


Accessing Historical Dynamics GP Data in NetSuite: Using Popdock
Popdock's Top 10 New Features
New Webinar: Getting Started with Matrix Reporting in Popdock
Tech Tuesday: Popdock User Security in D365 Business Central
Popdock Data Lake Upload Tool for Dynamics NAV Data - FREE TRAINING

POPULAR POSTS


2016 SmartConnect Integration Bootcamps
Tech Tues: Automatically refreshing Pivot Tables in Excel Refreshable Reports
Happy Thanksgiving from the eOne team!
2017 SmartConnect Integration Bootcamps
New Releases of Extender and SmartList Builder

CATEGORIES

TAGS

Business Central CRM D365 Business Central Dynamics 365 dynamics crm Dynamics GP Dynamics NAV Econnect Employee Spotlight eone eOne News error Error Message Events Excel Excel Report Builder Extender Flexicoder GP integration Map Microsoft dynamics crm Microsoft Dynamics GP Navigation List Builder Office Relationships Partner All Hands Call Popdock promotions release SalesForce SalesForce.com SmartConnect SmartConnect.com SmartConnect Bootcamp SmartConnect Maps SmartConnect Office Hours SmartList SmartList Builder SmartPost SmartView SQL Tech Tuesday Templates training Zendesk

Integrate & Automate without Any Code.

SmartList Data has Never Been Faster.

The Easiest Way to Report on GP Data.