Best practice for stacked tasks
Tom asked 7 years ago
I am setting up an import for a customer which requires several steps. Basically, three maps and then two SQL scrips need to run in succession. Let’s call them MAP1, MAP2, MAP3, SQL1, and SQL2. I had it set up so if MAP1 suceeded, a post-map task ran MAP2. In MAP2, I had a post-map task to run MAP3 (if MAP2 succeeded). Then in MAP3, I set up a post-map task to run SQL1 then SQL2.
It turned out this way because I built the entire process as I went, mostly by trial and error. After I got it all set up and working properly, I got to thinking that maybe it would be cleaner to have all maps and scripts set to run as post-map tasks in MAP1. So, I switched the process so all the post-map tasks are in MAP1. When MAP1 finished, run MAP2. When MAP2 is run successfully, run MAP3. When MAP3 is done, run SQL1, then SQL2.
Is there a Standard Operating Procedure for processing multiple maps/SQL scripts in one fell swoop? I tested it both ways and they both work equally as well. Is one method less prone to glitches than the other?
TL;DR Should succeeding processes be set up as post-map tasks inside the preceding process like a daisy chain, or should they all be post-map tasks inside the first process?
It turned out this way because I built the entire process as I went, mostly by trial and error. After I got it all set up and working properly, I got to thinking that maybe it would be cleaner to have all maps and scripts set to run as post-map tasks in MAP1. So, I switched the process so all the post-map tasks are in MAP1. When MAP1 finished, run MAP2. When MAP2 is run successfully, run MAP3. When MAP3 is done, run SQL1, then SQL2.
Is there a Standard Operating Procedure for processing multiple maps/SQL scripts in one fell swoop? I tested it both ways and they both work equally as well. Is one method less prone to glitches than the other?
TL;DR Should succeeding processes be set up as post-map tasks inside the preceding process like a daisy chain, or should they all be post-map tasks inside the first process?
Answers
Patrick Roth Staff answered 7 years ago
Tom,
Interesting question and I’ve wondered about it a bit as well.
In the end, I don’t know that it really makes a difference. You have one map calling a second map. You can choose to pass the data with it or not, Variables or not.
In either case, one map calling another.
There isn’t a defined limit in this daisy chain so from that perspective doesn’t seem like it would matter either.
Personally, I like Map1 calling map2,map3, map4 – that way I can easily see when I open map1 what is going to happen.
if I have map1->map2->map3->map4, then it isn’t so obvious on map1 this is what is going to happen.
patrick
Interesting question and I’ve wondered about it a bit as well.
In the end, I don’t know that it really makes a difference. You have one map calling a second map. You can choose to pass the data with it or not, Variables or not.
In either case, one map calling another.
There isn’t a defined limit in this daisy chain so from that perspective doesn’t seem like it would matter either.
Personally, I like Map1 calling map2,map3, map4 – that way I can easily see when I open map1 what is going to happen.
if I have map1->map2->map3->map4, then it isn’t so obvious on map1 this is what is going to happen.
patrick
Tom replied 7 years ago Thanks Patrick! I was a bit worried I was missing something and would look like a doofus asking a question with an obvious answer! 😉 I share your view that it is easier to follow the process pattern when all tasks are inside the first one. I plan to leave it that way unless something else turns up.